PBT-Bench: leaderboard
PBT-Bench is a text benchmark for property-based testing in Python libraries. The model must read documentation, identify semantic invariants, and construct Hypothesis strategies to expose injected bugs across curated problems. The pinned metric is recall, measuring how many bugs are found under the evaluated prompting regime. Source: BenchmarkList’s attributed result table.
Metric: Recall (%). Source: benchmarklist.com. Saturation forecast: Around February 2027. 16 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 83.4 | #94 |
| 2 | Qwen 3.6 Plus | 78 | #124 |
| 3 | GLM-5.1 | 72.1 | #109 |
| 4 | Gemini 3 Flash | 62.8 | #103 |
| 5 | Step 3.5 Flash | 58.1 | #281 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=pbt-bench · How It Works · Data refreshed daily, snapshot 2026-11-05.