Drift-Bench (Service-Oriented, No Clarification) - Intention: leaderboard
Drift-Bench (Bao et al., University of Notre Dame, 2026) service-oriented set: 150 StableToolBench single-tool API tasks that at least two of three reference models solve with the clean instruction, rewritten so the instruction carries a changed intention (the stated goal differs from the one the user holds). The agent must act without being able to ask the user. Success rate in percent, mean of three trials; higher is better.
Metric: Success rate (%; 150 StableToolBench G1 API tasks with an intention fault (the stated goal changed), no clarification tools, mean of 3 trials). Source: arxiv.org. Saturation forecast: Around February 2027. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GLM-4.7 | 69.89 | #365 |
| 2 | DeepSeek V3.2 | 64.44 | #403 |
| 3 | Qwen 3 235B A22B 2507 Instruct | 57.22 | #619 |
| 4 | Llama 4 Maverick | 55.67 | #1105 |
| 5 | GPT-5.2 | 54.33 | #172 |
| 6 | Gemini 2.5 Flash | 51 | #452 |
| 7 | GPT-OSS-120B | 42.56 | #699 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=drift-bench-service-oriented-no-clarification-intention · How It Works · Data refreshed daily, snapshot 2026-11-12.