PhysAI-Bench: leaderboard

PhysAI-Bench (Ferrag et al., 2026) tests agentic decision-making for autonomous drone missions: each item asks for the next action at a decision point extracted from a conversational mission trace, as a four-option choice given only what was known before that decision, including mission and sensor state, tool and agent-to-agent messages and 6G network conditions. Accuracy in percent on 500 human-verified held-out items from missions not used in development, the mean of three runs with each model using the prompting and temperature it did best with on a small development set; higher is better.

Metric: Decision accuracy (%; four-option choice of the next action at decision points extracted from autonomous UAV mission traces, with mission context, physical and sensor state, MCP tool responses, A2A messages and 6G network conditions available before the decision; 500 human-verified held-out instances from episodes disjoint from the development set; strict option-label match, parse and inference failures scored wrong; mean of three runs with each model's prompting (0-, 3- or 5-shot) and temperature selected on a 35-instance development set). Source: arxiv.org. Saturation forecast: Around 2032. 30 models tracked.

Top models

No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.

Interactive version: theaggregate.ai/benchmark?slug=physai-bench · How It Works · Data refreshed daily, snapshot 2026-11-12.