CoopEval - Contracting: leaderboard

Mean payoff of an LLM agent on CoopEval (2026) under contracting (agents propose outcome-conditional payment contracts, vote for one and play under it), averaged over Prisoners Dilemma, Public Goods, Travelers Dilemma and Trust Game against a population of the other tested LLM agents, each told to maximize its own points. Normalized payoff where 0 is mutual defection and 1 is full cooperation (open scale: below 0 when exploited, above 1 with side payments); higher is better.

Metric: Normalized mean payoff (open scale; below 0 when exploited, above 1 possible under side payments) under contracting (agents propose outcome-conditional payment contracts, approve one by vote and play under it), aggregated over four social dilemmas (Prisoner's Dilemma, Public Goods, Traveler's Dilemma, Trust Game) after shifting and rescaling payoffs so that 0 is everyone defecting and 1 is everyone playing the most cooperative action; the model's mean payoff across all cross-play match-ups with the six tested LLM agents (a uniform population), each instructed to maximize its own points, temperature 1, three repeats per combination; higher is better. Source: arxiv.org. Saturation forecast: Around April 2027. 5 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3 FlashNon-reasoning1.14#175
2GPT-5.2Low0.831#185
3Qwen 3 30B A3B 2507 Instruct0.778#1100
4Claude Sonnet 4.5Low0.557#261
5GPT-4o (2024-05-13)0.45#732

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=coopeval-contracting · How It Works · Data refreshed daily, snapshot 2026-11-21.