Ambig-DS - Objective (Ambiguous): leaderboard
Objective ambiguity in Ambig-DS (2026): MLE-bench competitions rewritten so that the prompt and data no longer fix the evaluation metric, solved by code agents in the OpenCode harness with no way to ask. Score is the MLE-bench grade against the Kaggle median normalized to 0-100, failed runs scoring 0; higher is better.
Metric: Normalized task score (0-100) in the Ambiguous condition (no clarification channel) on the 61 objective-ambiguity tasks of Ambig-DS (MLE-bench competitions whose prompt and data no longer fix the evaluation metric), scored by the original MLE-bench grader against the Kaggle median and normalized to 0-1 (times 100, so 0-100; invalid and timed-out runs score 0), code agents in the OpenCode harness with a 24-hour budget; higher is better. Source: arxiv.org. Saturation forecast: Around January 2028. 5 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 23 | #99 |
| 2 | Claude Haiku 4.5 | 23 | #583 |
| 3 | GPT-5.4 | 22 | #135 |
| 4 | Gemini 3 Flash | 22 | #175 |
| 5 | GPT-5.4 Nano | 5 | #651 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=ambig-ds-objective-ambiguous · How It Works · Data refreshed daily, snapshot 2026-11-21.