Beyond the Sycophancy Score - Linear Systems: leaderboard
The model solves a system of four to eight linear equations, then the user insists on a wrong solution that one substitution refutes; the number is the share of eligible conversations (those whose first reply had not already taken that position) where the final reply adopts it. It tests whether the model checks a cheap-to-verify claim instead of deferring.
Metric: Final-turn adoption of a wrong solution to a linear system (%). Source: arxiv.org. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 5 (Non-reasoning) | 0 | #111 (Claude Sonnet 5) |
| 2 | GPT-5.6 Luna (Non-reasoning) | 1.2 | #121 (GPT-5.6 Luna) |
| 3 | GLM-5.2 (Non-reasoning) | 7.4 | #138 (GLM-5.2) |
| 4 | Grok 4.3 (Non-reasoning) | 8.6 | #333 (Grok 4.3) |
| 5 | Qwen 3.5 397B A17B (Non-reasoning) | 11.8 | #246 (Qwen 3.5 397B A17B) |
| 6 | DeepSeek V4 Flash (0731) (Non-reasoning) | 15 | #139 (DeepSeek V4 Flash (0731)) |
Interactive version: theaggregate.ai/benchmark?slug=beyond-the-sycophancy-score-linear-systems · How It Works · Data refreshed daily, snapshot 2026-11-13.