STAGE-Eval: leaderboard
STAGE-Eval (Ahn, Son and Yu, 2026) tests extracting structured JSON from documents: 851 held-out items each give a report written around a real spreadsheet, an extraction prompt and a JSON Schema, with gold values checked against the spreadsheet. Exact match rate in percent, the share of items whose parsed output equals the gold JSON, averaged over three runs with a 3,100-token cap; higher is better. The authors' fine-tunes trained on STAGE data are in-distribution.
Metric: Exact match rate (%; share of items whose parsed JSON equals the gold object after canonicalization, a parse failure counting as a miss; the 851-item held-out STAGE-Eval test set: an LLM-written report around a real spreadsheet, an extraction prompt and a JSON Schema per item; vLLM, temperature 0.6, at most 3,100 new tokens, JSON read after any reasoning; mean of three runs, the standard deviation printed run-on in the same cell not read). Source: arxiv.org. Saturation forecast: Estimated already saturated. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 4B (Non-reasoning) | 31.37 | #2281 (Qwen 3 4B) |
| 2 | Qwen 2.5 3B Instruct | 21.5 | #5150 |
| 3 | Llama 3.2 3B Instruct | 11.32 | #7147 |
| 4 | Llama 3.2 1B Instruct | 4.07 | #9047 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=stage-eval · How It Works · Data refreshed daily, snapshot 2026-11-13.