Text-to-SQL under RBAC - BIRD: leaderboard

Access-controlled F1 on the BIRD split of the role-based-access text-to-SQL evaluation. Measures answering database questions while respecting the user's access permissions. Spider and LiveSQLBench are separate boards.

Metric: Access-Controlled F1 (%, BIRD). Source: arxiv.org. Saturation forecast: Around February 2027. 13 models tracked. Updated 4 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Sonnet 4.587.65#262
2DeepSeek V3.2Thinking87.1#423
3GPT-586.32#180
4GPT-5 Mini84.03#365
5Gemini 2.5 Flash75.59#469
6Gemma 3 27B65.67#1669
7GPT-4o Mini63.18#1678
8DeepSeek V3.2 Coder57.89
9Qwen 2.5 14B Instruct54.64#1785
10Snowflake-R1-7B51.45
11Qwen 2.5 Coder 7B Instruct50.75#3987
12Llama3-SQLCoder-8B50.49
13Gemma 3 4B50.22#5301

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=text-to-sql-under-rbac-bird · How It Works · Data refreshed daily, snapshot 2026-11-20.