Ambig-DS - Target (Ask): leaderboard

Target ambiguity in Ambig-DS (2026): DSBench data-modeling tasks rewritten so that the prompt and data no longer fix which variable to predict, solved by code agents in the OpenCode harness that may first ask one short question of a scoped answer-only oracle. Score is the DSBench relative performance normalized to 0-100, failed runs scoring 0; higher is better.

Metric: Normalized task score (0-100) in the Ask condition (before modeling the agent may ask one question of 50 words or fewer to a scoped answer-only oracle) on the 51 target-ambiguity tasks of Ambig-DS (DSBench data-modeling tasks whose prompt and data no longer fix the prediction target), scored by the original DSBench evaluator as the relative performance gap normalized to 0-1 (times 100, so 0-100; invalid and timed-out runs score 0), code agents in the OpenCode harness with a 2-hour budget; higher is better. Source: arxiv.org. Saturation forecast: Around May 2027. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 3 Flash61#180
2Gemini 3.1 Pro (Preview)60#102
3GPT-5.452#138
4Claude Haiku 4.552#587
5GPT-5.4 Nano45#653

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=ambig-ds-target-ask · How It Works · Data refreshed daily, snapshot 2026-11-22.