BlueFin - Manipulation: leaderboard
The 75 held-out manipulation tasks of BlueFin (Kundurthy et al., 2026): the agent receives an existing financial-model workbook and an analyst-style instruction and modifies it through a spreadsheet tool harness. Scored in percent (higher is better) as the weighted share of rubric criteria met, covering formula correctness, links to source tabs, outputs within tolerance and correct recalculation when an input changes, checked by an agentic GPT-5.4 judge.
Metric: Criteria-weighted rubric score (%; points met over maximum points of expert rubrics checked by an agentic GPT-5.4 judge for synthesis and manipulation workbooks, answer correctness for interrogation, non-completions count as zero; 20-tool spreadsheet harness, high reasoning effort; 75 held-out tasks modifying an existing financial model). Source: arxiv.org. Saturation forecast: Around June 2027. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 (High) | 49.4 | #75 (Claude Opus 4.7) |
| 2 | GPT-5.5 (High) | 48.2 | #48 (GPT-5.5) |
| 3 | Gemini 3.1 Pro (Preview) (High) | 46.1 | #91 (Gemini 3.1 Pro (Preview)) |
| 4 | Claude Sonnet 4.6 (High) | 43 | #158 (Claude Sonnet 4.6) |
| 5 | Grok 4.20 (High) | 27.8 | #296 (Grok 4.20) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=bluefin-manipulation · How It Works · Data refreshed daily, snapshot 2026-11-12.