HiL-Bench Original Paper - SWE - ASK-F1: leaderboard
Useful clarification questions on SQL and SWE tasks. Printed Table 1, page 6 of the original four-model HiL-Bench paper; distinct from the larger live Pass@3 roster. Recall finds blockers, precision measures useful questions, and ASK-F1 balances the two. Values are printed percentages, not reconstructed prose summaries. This series reports ASK-F1 (%). Source: https://static.scale.com/uploads/67a153343e046988406ef320/HiL_Bench.pdf.
Metric: ASK-F1 (%). Source: static.scale.com. Saturation forecast: Around November 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 41.1 | #64 |
| 2 | GPT-5.4 | 35.6 | #83 |
| 3 | GPT-5.3 Codex | 33.2 | #72 |
| 4 | Claude Opus 4.6 | 29.9 | #67 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=hil-bench-original-paper-swe-ask-f1 · How It Works · Data refreshed daily, snapshot 2026-11-05.