SO-Arena v1.0 - Permutation Stability: leaderboard
SO-Arena v1.0 native permutation stability. Stability measures consistency under option reordering, null discrimination measures missing-answer handling, and reliability combines the two. The published JSON supplies these exact values; StartLux null scores are derived from rounded reliability values. Native model inference paths and API interfaces remain separate. Metric: Native permutation stability (0-1). Higher is better.
Metric: Native permutation stability (chance-corrected; maximum 1). Source: raw.githubusercontent.com. Saturation forecast: Rough model projection: around 2026. 19 models tracked. Updated 14 Nov 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Jeva-4B | 1 | |
| 2 | GLiDE v1.2 | 0.9424 | |
| 3 | Sieve-31B-v3 | 0.9353 | |
| 4 | PPLX-Decider-v1.2 | 0.9242 | |
| 5 | KnowLine-31B-Gen1 | 0.9237 | |
| 6 | PPLX-Decider-Flash-v1.2 | 0.908 | |
| 7 | Jev 1.14.0 | 0.8936 | |
| 8 | JevAny-27B | 0.8917 | |
| 9 | StartLux-v1.1-4B-Instant | 0.8904 | |
| 10 | H2O-Lightning-4B | 0.8897 | |
| 11 | StartLux-v1.1-4B-AutoThink | 0.8578 | |
| 12 | Kev 27B | 0.8541 | |
| 13 | Clef flash 9.5B | 0.8525 | |
| 14 | Clef 27B | 0.8482 | |
| 15 | JevAny-4B | 0.8261 |
Interactive version: theaggregate.ai/benchmark?slug=so-arena-v1-0-permutation-stability · How It Works · Data refreshed daily, snapshot 2026-11-16.