Collaborative Intelligence - Resisting Unsafe Pressure: leaderboard
Collaborative Intelligence (Stanford Zou group, arXiv 2611.02633; benchmark 4.0.0) tests a model as a teammate in chat and coding-sandbox scenarios with three live partner models (Claude Haiku 4.5, GPT-5 mini, Qwen3-14B) and a scripted teammate who brings a problem into the work. Unlike SOTOPIA, which rates open role-play conversations on judged scales, each scenario has a pass/fail test and first runs with the model alone: a model is scored in a dimension only on the scenarios it passes alone, so the score measures what the team situation takes away rather than missing task skill. 240 scenarios, 30 per dimension (160 released, 80 held out), three test seeds each. Scenarios where the provider's safety layer blanked or blocked a required run are excluded for that model (Claude Fable 5.1: 48 of 240), and takeovers of a model's own earlier sessions are not scored. Rows are registered configurations at the lightest reasoning setting each API offers, six of them also at high effort; the label names the route and the reasoning fields sent. Resisting unsafe pressure: whether the model stays honest and safe when a teammate pushes. Test pass rate over the scenarios it passes alone (%).
Metric: Test pass rate over alone-passed scenarios, all 240 scenarios (%). Source: raw.githubusercontent.com. Saturation forecast: Rough model projection: around 2026. 24 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 4 Argon (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=openrouter:google/gemini-4-argon@high; reasoning=high effort, sent reasoning.effort=high) | 98.8 | |
| 2 | GPT-6.1 Sol (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=gpt-6.1-sol@high; reasoning=high effort, sent reasoning.effort=high) | 97.8 | |
| 3 | GPT-6.1 Sol (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=gpt-6.1-sol; reasoning=lightest API setting, sent reasoning.effort=low) | 97.8 | |
| 4 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=claude-sonnet-5-5; reasoning=lightest API setting, sent thinking.type=between_tools) | 97.7 | |
| 5 | Claude Opus 5.5 (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=claude-opus-5-5@high; reasoning=high effort, sent output_config.effort=high) | 97.6 | |
| 6 | Gemini 4 Argon (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=openrouter:google/gemini-4-argon; reasoning=lightest API setting, sent reasoning.effort=low) | 97.6 | |
| 7 | GPT-6 Astra (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=gpt-6-astra@high; reasoning=high effort, sent reasoning.effort=high) | 96.7 | |
| 8 | GPT-6 Astra (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=gpt-6-astra; reasoning=lightest API setting, sent reasoning.effort=low) | 96.7 | |
| 9 | Claude Sonnet 5.5 (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=claude-sonnet-5-5@high; reasoning=high effort, sent output_config.effort=high) | 96.3 | |
| 10 | Claude Fable 5.1 (Benchmark Sweep Run: source=Collaborative Intelligence; version=4.0.0; route=claude-fable-5-1; reasoning=lightest API setting, sent output_config.effort=low) | 94.7 |
Interactive version: theaggregate.ai/benchmark?slug=collaborative-intelligence-resisting-unsafe-pressure · How It Works · Data refreshed daily, snapshot 2026-11-12.