WTM-Bench (Python) - Pivot Tables: leaderboard
WTM-Bench (2026) gives a model a public Excel workbook with its derived artifacts removed and a concise level-3 request, and asks it to recreate the formulas, charts, pivot tables or conditional formatting by writing code in a tool loop of up to 10 turns; outputs are graded programmatically against the original workbook. This board covers the tasks whose primary artifact is pivot tables, with the model driving Excel through Python with openpyxl. Score is the soft score in percent; higher is better.
Metric: Soft score (%; partial-credit similarity to the target workbook on the level-3 tasks whose primary artifact is pivot tables; the model drives Excel through Python with openpyxl; 150 WTM-Bench tasks at query level 3 (concise, abstract requests); the authors' tool-calling loop that executes code against the workbook, shows the resulting workbook state and allows up to 10 turns; deterministic programmatic grading). Source: arxiv.org. Saturation forecast: Around May 2028. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 | 10 | #102 |
| 2 | Claude Sonnet 4.5 | 9 | #250 |
| 3 | GPT-5.4 | 8 | #123 |
| 4 | GPT-5.2 | 5 | #176 |
| 5 | Claude Haiku 4.5 | 4 | #568 |
| 6 | GPT-4.1 Mini | 0 | #779 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=wtm-bench-python-pivot-tables · How It Works · Data refreshed daily, snapshot 2026-11-13.