GroupTravelBench - Hard: leaderboard

GroupTravelBench Hard reports Group Utility on the 200 hard multi-user travel planning tasks, which allow up to 25 chat rounds (GroupTravelBench, 2026). Group Utility adds points per user for satisfied preferences and subtracts points for violated ones and for splitting the group, so it has no fixed range and can be negative; higher is better.

Metric: Group Utility (GU, unnormalized points per user, unbounded and can be negative: +2 or +1 for each satisfied must or prefer item, -2 or -1 for each violated reject or avoid item, minus K-1 for each split into K subgroups, divided by the group size) on the 200 hard tasks (25 rounds), on GroupTravelBench, multi-user multi-turn group travel planning in a group-chat sandbox with 10 cached travel tools (DeepSeek-V4-Flash user simulator at temperature 0, agent temperature 0.7, mean of 3 trials per task); GPT-5.1 in instruct mode, the other models with reasoning enabled; higher is better. Source: arxiv.org. Saturation forecast: Around July 2027. 9 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1DeepSeek V4 ProThinking10.36#196
2Qwen 3.6 Max (Preview)Thinking10.33
3Qwen 3.5 PlusThinking9.2#242
4GroupTravelBench Qwen3-235B-A22B-Thinking (checkpoint unspecified)8.15
5GPT-5.1Non-reasoning7.78#240
6Qwen 3.5 FlashThinking7.65#441
7Qwen 3 4B Instruct7.1#1696
8GroupTravelBench Qwen3-30B-A3B-Thinking (checkpoint unspecified)7.07
9GroupTravelBench Qwen3-4B-Thinking (checkpoint unspecified)6.64

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=grouptravelbench-hard · How It Works · Data refreshed daily, snapshot 2026-11-21.