XYBench Stack Overflow - Goal-Focused Answer Selection: leaderboard

XYBench, Ai2 release of November 13, 2026. These results use the 1,557 Stack Overflow questions, not all 8,115 questions across the four source sets. A user asks about a mistaken approach (Y) to an underlying goal (X). Five named models answer once per question in the single-turn experiment. Claude Sonnet 4.5 is the automated judge; the publisher validated its criteria with two human reviewers on 500 responses. Each metric remains distinct. The top human answer is a separate source comparator, not Median Human or Median Expert; random-pick control and the repeated emphasis table are not additional model observations. No generation settings or model checkpoints beyond the named source identities are guessed. Percentage choosing the goal-focused answer when shown two alternatives; this is recognition, separate from writing an answer. Higher is better.

Metric: Rate (%). Source: allenai.org. Saturation forecast: Around February 2027. 5 models tracked. Updated 14 Nov 2026.

Top models

#ModelScoreOverall rank
1DeepSeek V4 Pro84#191
2Qwen 3.5 9B83#814
3Gemini 3.1 Pro (Preview)81#93
4GPT-572#167
5Claude Sonnet 4.671#165

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=xybench-stack-overflow-goal-focused-answer-selection · How It Works · Data refreshed daily, snapshot 2026-11-16.