PilotBench: leaderboard

PilotBench (2026) evaluates LLMs as general-aviation flight predictors: from structured historical waypoints of real trajectories across nine flight phases, a model predicts the next trajectory and attitude state in a fixed output schema at temperature 0. This board is Pilot-Score on a 5 to 100 scale, 0.6 times the regression score plus 0.4 times the instruction-following score; higher is better.

Metric: Pilot-Score (5-100): 0.6 x regression score plus 0.4 x instruction-following score, on PilotBench (one-step trajectory and attitude prediction from 34-channel telemetry of 708 general-aviation trajectories in nine flight phases, fixed output schema, temperature 0); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 18 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3 32B90.23#983
2Qwen 2.5 72B Instruct88.58#1124
3DeepSeek V388.31#736
4GPT-4o Mini88.09#1680
5O3 Mini87.76#552
6Doubao-1.5-Pro-32k87.52#701
7Qwen 2.5 32B Instruct85.27#1253
8QwQ-32B84.9#913
9Qwen 2.5 14B Instruct84.44#1788
10Qwen 2.5 VL 72B Instruct77.35#849
11Qwen 2.5 7B Instruct71.11#2732
12glm-4-9B69.03#2748
13DeepSeek R1 Distill Llama 70B65#1676
14DeepSeek R1 Distill Qwen 32B59.37#1844
15Qwen 3 14B49.54#1284

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=pilotbench · How It Works · Data refreshed daily, snapshot 2026-11-22.