BI-Bench (SQL, BI-Agent Tools): leaderboard
BI-Bench (Hu et al., 2026) asks models to answer 100 business questions taken from real Power BI dashboards directly from the raw project tables, which requires finding, reshaping and joining the relevant tables before computing the answer. An answer is correct when its result table matches the ground truth in shape and values; accuracy in percent averaged over 10 runs, higher is better. SQL setting with the BI-Agent search, transform and join tools of the authors added to the code-execution tool.
Metric: Accuracy (%; 100 end-to-end business-intelligence queries curated from real Power BI dashboards (82 projects, raw source tables needing search, reshaping, joins and analysis); an answer is correct when the output table matches the ground-truth table in shape and cell values up to row and column order, with numeric tolerance; mean over 10 runs; SQL setting with the BI-Agent search, transform and join tools added to the code-execution tool). Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | O4 Mini | 61.9 | #354 |
| 2 | GPT-5.5 | 60.2 | #48 |
| 3 | GPT-5.2 | 55.2 | #172 |
| 4 | GPT-OSS-120B | 45.3 | #699 |
| 5 | Mistral Large 3 | 44.8 | #819 |
| 6 | GPT-4o | 37.7 | #741 |
| 7 | Kimi K2.6 | 36.7 | #173 |
| 8 | Llama 4 Maverick | 31.2 | #1105 |
| 9 | DeepSeek V4 Pro | 30.8 | #181 |
| 10 | Qwen 3 8B | 19.8 | #1838 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=bi-bench-sql-bi-agent-tools · How It Works · Data refreshed daily, snapshot 2026-11-12.