FIRE-Bench: leaderboard
FIRE-Bench (Wang et al., 2026) asks research agents to rediscover the empirical findings of 30 published papers on LLM behaviour from their research questions alone, designing and running the experiments end to end. The agent conclusions are split into atomic claims and matched by entailment against the source findings with a GPT-5.2 judge; claim-level F1 in percent, averaged over tasks and three runs, higher is better.
Metric: Claim-level F1 (%; agent conclusions and the source paper's findings are split into atomic claims and matched by semantic entailment with a GPT-5.2 judge; mean over the 30 core LLM-behaviour rediscovery tasks and three runs; agent runs full research cycles from the research question in a GPU sandbox). Source: arxiv.org. Saturation forecast: Around April 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4 | 46.7 | #397 |
| 2 | GPT-5 (Medium) | 41.9 | #163 (GPT-5) |
| 3 | GPT-5 | 37.9 | #163 |
| 4 | O4 Mini | 31.9 | #354 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=fire-bench · How It Works · Data refreshed daily, snapshot 2026-11-12.