TasteVal: leaderboard

TasteVal (P-Zero Research, Jaffe and Sherburn, 2026) tests the experimental research taste of AI models on eight withheld AI R&D tasks: the model only designs experiments and interprets results, while a fixed Claude Opus 4.8 coding agent runs them on one H100 within 40 GPU hours. The score is the compute multiplier against the best human expert attempt per task, which scores 1: how many times less experimental compute the model needs to reach the same test score, chained through GPT-4 and Claude Opus 5 reference runs for models far from expert level. Higher is better, with no upper limit.

Metric: Compute multiplier against the expert baseline (the best of at least two human expert attempts per task scores 1): GPU hours the baseline needs to reach the final test score of the weaker run divided by the GPU hours the model needs, geometric mean over seeds and then over the 8 withheld AI R&D tasks, chained through GPT-4 and Claude Opus 5 (max) reference runs for models far from expert level; the model only proposes and interprets experiments, which a fixed Claude Opus 4.8 Coder runs on one H100 within 40 H100 hours; each run at the reasoning setting its label names; from 0 with no upper limit; higher is better. Source: arxiv.org. Saturation forecast: Around January 2028. 22 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 5.5Max2.3#9
2Claude Fable 5.1Max1.72#20
3GPT-6 AstraMax1.6#10
4Claude Opus 5Max1.29#26
5Claude Opus 4.8Max0.84#65
6GPT-5.6 SolMax0.59#37
7GLM-5.3Max0.59#72
8Claude Opus 4.7Max0.5#85
9GPT-5.5xHigh0.47#54
10Kimi K3Thinking0.45#61
11Claude Opus 4.6Max0.4#113
12GPT-5.2High0.28#185
13GPT-5.4xHigh0.21#135
14GPT-5High0.12#178
15Claude Opus 4.5Thinking, 24K0.12#160

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=tasteval · How It Works · Data refreshed daily, snapshot 2026-11-21.