GroupTravelBench - Medium: leaderboard

GroupTravelBench Medium reports Group Utility on the 250 medium multi-user travel planning tasks, which allow up to 20 chat rounds (GroupTravelBench, 2026). Group Utility adds points per user for satisfied preferences and subtracts points for violated ones and for splitting the group, so it has no fixed range and can be negative; higher is better.

Metric: Group Utility (GU, unnormalized points per user, unbounded and can be negative: +2 or +1 for each satisfied must or prefer item, -2 or -1 for each violated reject or avoid item, minus K-1 for each split into K subgroups, divided by the group size) on the 250 medium tasks (20 rounds), on GroupTravelBench, multi-user multi-turn group travel planning in a group-chat sandbox with 10 cached travel tools (DeepSeek-V4-Flash user simulator at temperature 0, agent temperature 0.7, mean of 3 trials per task); GPT-5.1 in instruct mode, the other models with reasoning enabled; higher is better. Source: arxiv.org. Saturation forecast: Around July 2027. 9 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1DeepSeek V4 ProThinking11.35#201
2Qwen 3.6 Max (Preview)Thinking10.03
3Qwen 3.5 PlusThinking9.02#250
4GPT-5.1Non-reasoning7.44#248
5GroupTravelBench Qwen3-235B-A22B-Thinking (checkpoint unspecified)7.4
6Qwen 3.5 FlashThinking7.05#447
7GroupTravelBench Qwen3-30B-A3B-Thinking (checkpoint unspecified)6.26
8Qwen 3 4B Instruct6.05#1711
9GroupTravelBench Qwen3-4B-Thinking (checkpoint unspecified)5.64

No result here: #2 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=grouptravelbench-medium · How It Works · Data refreshed daily, snapshot 2026-11-22.