PilotBench - Instruction Following: leaderboard

PilotBench (2026) evaluates LLMs as general-aviation flight predictors: from structured historical waypoints of real trajectories across nine flight phases, a model predicts the next trajectory and attitude state in a fixed output schema at temperature 0. This board is the instruction-following score in percent, combining field completeness, field validity and format compliance of the predicted schema; higher is better.

Metric: Instruction-following score (%): 0.5 x field completeness plus 0.3 x field validity plus 0.2 x format compliance of the predicted schema, on PilotBench (one-step trajectory and attitude prediction from 34-channel telemetry of 708 general-aviation trajectories in nine flight phases, fixed output schema, temperature 0); higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 19 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3 32B97.43#972
2Qwen 2.5 14B Instruct96.65#1775
3Qwen 2.5 72B Instruct96.64#1117
4glm-4-9B96.45#2738
5DeepSeek V396#729
6GPT-4o Mini95.74#1667
7Qwen 2.5 7B Instruct95.32#2723
8Qwen 2.5 VL 72B Instruct94.89#841
9Qwen 2.5 32B Instruct94.66#1247
10DeepSeek-R1-Distill-Qwen-7B94.42#8055
11Qwen 3 14B94.09#1269
12Doubao-1.5-Pro-32k93.5#694
13O3 Mini93.05#546
14DeepSeek R1 Distill Qwen 32B91.07#1834
15QwQ-32B83.95#907

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=pilotbench-instruction-following · How It Works · Data refreshed daily, snapshot 2026-11-21.