SO-Arena v1.0 - Permutation Stability: leaderboard

SO-Arena v1.0 native permutation stability. Stability measures consistency under option reordering, null discrimination measures missing-answer handling, and reliability combines the two. The published JSON supplies these exact values; StartLux null scores are derived from rounded reliability values. Native model inference paths and API interfaces remain separate. Metric: Native permutation stability (0-1). Higher is better.

Metric: Native permutation stability (chance-corrected; maximum 1). Source: raw.githubusercontent.com. Saturation forecast: Rough model projection: around 2026. 19 models tracked. Updated 14 Nov 2026.

Top models

#ModelScoreOverall rank
1Jeva-4B1
2GLiDE v1.20.9424
3Sieve-31B-v30.9353
4PPLX-Decider-v1.20.9242
5KnowLine-31B-Gen10.9237
6PPLX-Decider-Flash-v1.20.908
7Jev 1.14.00.8936
8JevAny-27B0.8917
9StartLux-v1.1-4B-Instant0.8904
10H2O-Lightning-4B0.8897
11StartLux-v1.1-4B-AutoThink0.8578
12Kev 27B0.8541
13Clef flash 9.5B0.8525
14Clef 27B0.8482
15JevAny-4B0.8261

Interactive version: theaggregate.ai/benchmark?slug=so-arena-v1-0-permutation-stability · How It Works · Data refreshed daily, snapshot 2026-11-16.