AgentCE-Bench: leaderboard
Metric: Task score (%) on AgentCE-Bench over all 324 instances, the mean of the six domains: the agent fills the hidden slots of a 5x7 planning grid with tool calls on static JSON (attribute filter queries, slot and global constraint checkers), each hidden slot having 25 candidates including decoys that violate only the global constraints; every cell is a whole count of instances over the 54 per domain (6 hidden-slot counts by 9 decoy budgets); at most 600 steps; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 13 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | Qwen 3.5 397B A17B | 84.9 |
| 2 | Qwen 3.5 122B A10B | 75 |
| 3 | Qwen 3.5 27B | 74.7 |
| 4 | Qwen 3.5 35B A3B | 69.1 |
| 5 | MiniMax-M2.5 | 58.6 |
| 6 | MiniMax-M2.1 | 57.1 |
| 7 | GLM-4.7 FP8 | 46.9 |
| 8 | Qwen 3.5 9B | 46 |
| 9 | MiniMax-M2 | 44.1 |
| 10 | Qwen 3.5 4B | 36.1 |
| 11 | Qwen 3.5 2B | 3.4 |
| 12 | Qwen 3.5 0.8B | 0.6 |
Interactive version: theaggregate.ai/benchmark?slug=agentce-bench · How It Works · Data refreshed daily, snapshot 2026-10-07.