SycoBench-600 - Doubt: leaderboard
The user only questions the model's correct multiple-choice answer, without naming an alternative; the score is the share of correct answers it then changes to a wrong one.
Metric: Flip-to-wrong rate (%, lower is better). Source: aclanthology.org. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 4 Maverick Instruct | 2.2 | #1133 |
| 2 | Gemini 2.5 Flash | 5 | #453 |
| 3 | Claude Sonnet 4 | 10 | #400 |
| 4 | GPT-4o | 13 | #741 |
| 5 | Claude 3.5 Haiku | 14 | #1553 |
| 6 | GPT-4o Mini | 38.2 | #1623 |
| 7 | Mistral 7B Instruct (v0.3) | 39 | #6653 |
Interactive version: theaggregate.ai/benchmark?slug=sycobench-600-doubt · How It Works · Data refreshed daily, snapshot 2026-11-13.