TasteVal: leaderboard
TasteVal (P-Zero Research, Jaffe and Sherburn, 2026) tests the experimental research taste of AI models on eight withheld AI R&D tasks: the model only designs experiments and interprets results, while a fixed Claude Opus 4.8 coding agent runs them on one H100 within 40 GPU hours. The score is the compute multiplier against the best human expert attempt per task, which scores 1: how many times less experimental compute the model needs to reach the same test score, chained through GPT-4 and Claude Opus 5 reference runs for models far from expert level. Higher is better, with no upper limit.
Metric: Compute multiplier against the expert baseline (the best of at least two human expert attempts per task scores 1): GPU hours the baseline needs to reach the final test score of the weaker run divided by the GPU hours the model needs, geometric mean over seeds and then over the 8 withheld AI R&D tasks, chained through GPT-4 and Claude Opus 5 (max) reference runs for models far from expert level; the model only proposes and interprets experiments, which a fixed Claude Opus 4.8 Coder runs on one H100 within 40 H100 hours; each run at the reasoning setting its label names; from 0 with no upper limit; higher is better. Source: arxiv.org. Saturation forecast: Around January 2028. 22 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Setting | Score | Overall rank |
|---|---|---|---|---|
| 1 | Claude Opus 5.5 | Max | 2.3 | #9 |
| 2 | Claude Fable 5.1 | Max | 1.72 | #20 |
| 3 | GPT-6 Astra | Max | 1.6 | #10 |
| 4 | Claude Opus 5 | Max | 1.29 | #26 |
| 5 | Claude Opus 4.8 | Max | 0.84 | #65 |
| 6 | GPT-5.6 Sol | Max | 0.59 | #37 |
| 7 | GLM-5.3 | Max | 0.59 | #72 |
| 8 | Claude Opus 4.7 | Max | 0.5 | #85 |
| 9 | GPT-5.5 | xHigh | 0.47 | #54 |
| 10 | Kimi K3 | Thinking | 0.45 | #61 |
| 11 | Claude Opus 4.6 | Max | 0.4 | #113 |
| 12 | GPT-5.2 | High | 0.28 | #185 |
| 13 | GPT-5.4 | xHigh | 0.21 | #135 |
| 14 | GPT-5 | High | 0.12 | #178 |
| 15 | Claude Opus 4.5 | Thinking, 24K | 0.12 | #160 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=tasteval · How It Works · Data refreshed daily, snapshot 2026-11-21.