EnvironmentDuels - Hint Lift: leaderboard
EnvironmentDuels: 90 agent-created environments, 12,960 episodes; each creator makes 10, solver conditions with/without hints share seeds. Difficulty is author failure rate, solver success and hint lift are distinct axes. Native Claude Code, Codex, DeepSeek Harness, Qwen Code, ZCode, Grok Build, Gemini CLI, Kimi Code and OpenCode harnesses; maximal supported reasoning (Gemini High, Qwen/Grok XHigh, others Max). Lift (percentage points).
Metric: Lift (percentage points). Source: arxiv.org. Saturation forecast: Not forecast. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.8 27B (xHigh) | 21.2 | #118 (Qwen 3.8 27B) |
| 2 | Gemini 3.8 Flash (High) | 18.9 | #23 (Gemini 3.8 Flash) |
| 3 | DeepSeek V4.1 Flash (Max) | 14.4 | #33 (DeepSeek V4.1 Flash) |
| 4 | GLM-5.3 (Max) | 11.4 | #36 (GLM-5.3) |
| 5 | Kimi K3 (Max) | 10.8 | #30 (Kimi K3) |
| 6 | Claude Opus 5 (Max) | 9.2 | #12 (Claude Opus 5) |
| 7 | Grok 4.6 (xHigh) | 8.6 | #25 (Grok 4.6) |
| 8 | GPT-5.6 Sol (Max) | 8.3 | #18 (GPT-5.6 Sol) |
No result here: #4 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=environmentduels-hint-lift · How It Works · Data refreshed daily, snapshot 2026-10-23.