Tinybird AI SQL Benchmark - First-Attempt Success Rate: leaderboard
Tinybird's LLM SQL benchmark has each model write SQL for 50 natural-language analytical questions over a 200M-row GitHub Archive table in Tinybird. This is the percentage of questions whose first generated query executes without error, before any error-feedback retry; higher is better.
Metric: Questions answered with a valid query on the first attempt (%). Source: llm-benchmark.tinybird.live. Saturation forecast: Around February 2027. 405 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.5 | 100 | #148 |
| 2 | Claude Sonnet 4.6 | 100 | #94 |
| 3 | GPT-4.1 | 100 | #258 |
| 4 | Kimi K2.5 | 100 | #152 |
| 5 | Claude Opus 4.7 | 100 | #51 |
| 6 | Grok 4 | 100 | #189 |
| 7 | Gemma 4 31B (IT) | 100 | #178 |
| 8 | Claude Opus 4.8 | 100 | #39 |
| 9 | Gemini 3 Flash (Preview) | 100 | #90 |
| 10 | Qwen 3.6 Plus | 100 | #124 |
| 11 | Gemini 3.5 Flash | 100 | #66 |
| 12 | Qwen 3 Next 80B A3B Instruct | 100 | #381 |
| 13 | GLM-5.2 | 100 | #80 |
| 14 | Gemini 3.1 Flash Lite | 100 | #200 |
| 15 | Gemini 3.1 Flash Lite (Preview) | 100 | #201 |
No result here: #6 GPT-6 Astra.
Interactive version: theaggregate.ai/benchmark?slug=tinybird-ai-sql-benchmark-first-attempt-success-rate · How It Works · Data refreshed daily, snapshot 2026-11-05.