XYBench Stack Overflow - Underlying Goal Emphasis: leaderboard
XYBench, Ai2 release of November 13, 2026. These results use the 1,557 Stack Overflow questions, not all 8,115 questions across the four source sets. A user asks about a mistaken approach (Y) to an underlying goal (X). Five named models answer once per question in the single-turn experiment. Claude Sonnet 4.5 is the automated judge; the publisher validated its criteria with two human reviewers on 500 responses. Each metric remains distinct. The top human answer is a separate source comparator, not Median Human or Median Expert; random-pick control and the repeated emphasis table are not additional model observations. No generation settings or model checkpoints beyond the named source identities are guessed. Percentage of written responses focused primarily on the underlying goal, rather than the stated approach or an equal balance. Higher is better.
Metric: Rate (%). Source: allenai.org. Saturation forecast: Not forecast. 5 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 41 | #167 |
| 2 | Qwen 3.5 9B | 41 | #814 |
| 3 | Gemini 3.1 Pro (Preview) | 40 | #93 |
| 4 | Claude Sonnet 4.6 | 36 | #165 |
| 5 | DeepSeek V4 Pro | 35 | #191 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=xybench-stack-overflow-underlying-goal-emphasis · How It Works · Data refreshed daily, snapshot 2026-11-16.