Ambig-DS - Objective (Ask): leaderboard
Objective ambiguity in Ambig-DS (2026): MLE-bench competitions rewritten so that the prompt and data no longer fix the evaluation metric, solved by code agents in the OpenCode harness that may first ask one short question of a scoped answer-only oracle. Score is the MLE-bench grade against the Kaggle median normalized to 0-100, failed runs scoring 0; higher is better.
Metric: Normalized task score (0-100) in the Ask condition (before modeling the agent may ask one question of 50 words or fewer to a scoped answer-only oracle) on the 61 objective-ambiguity tasks of Ambig-DS (MLE-bench competitions whose prompt and data no longer fix the evaluation metric), scored by the original MLE-bench grader against the Kaggle median and normalized to 0-1 (times 100, so 0-100; invalid and timed-out runs score 0), code agents in the OpenCode harness with a 24-hour budget; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash | 31 | #175 |
| 2 | Gemini 3.1 Pro (Preview) | 27 | #99 |
| 3 | GPT-5.4 | 26 | #135 |
| 4 | Claude Haiku 4.5 | 26 | #583 |
| 5 | GPT-5.4 Nano | 10 | #651 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=ambig-ds-objective-ask · How It Works · Data refreshed daily, snapshot 2026-11-21.