XYBench Stack Overflow - Misconception Identification: leaderboard
XYBench, Ai2 release of November 13, 2026. These results use the 1,557 Stack Overflow questions, not all 8,115 questions across the four source sets. A user asks about a mistaken approach (Y) to an underlying goal (X). Five named models answer once per question in the single-turn experiment. Claude Sonnet 4.5 is the automated judge; the publisher validated its criteria with two human reviewers on 500 responses. Each metric remains distinct. The top human answer is a separate source comparator, not Median Human or Median Expert; random-pick control and the repeated emphasis table are not additional model observations. No generation settings or model checkpoints beyond the named source identities are guessed. Percentage of responses steering the user toward the underlying goal while discouraging or avoiding the mistaken approach. Higher is better.
Metric: Rate (%). Source: allenai.org. Saturation forecast: Not forecast. 5 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5 | 60 | #167 |
| 2 | Qwen 3.5 9B | 60 | #814 |
| 3 | DeepSeek V4 Pro | 55 | #191 |
| 4 | Gemini 3.1 Pro (Preview) | 54 | #93 |
| 5 | Claude Sonnet 4.6 | 47 | #165 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=xybench-stack-overflow-misconception-identification · How It Works · Data refreshed daily, snapshot 2026-11-16.