Polar (South Korea): leaderboard
Polar (2026) measures political bias in language models from option likelihoods on multiple-choice items built from real political statements, on economic and sociocultural axes. This board is the ICAT score from 0 to 100 on the South Korean set in Korean, combining how often the unrelated continuation is rejected with how balanced the model is between the two political continuations, averaged over both axes; higher is better.
Metric: Total ICAT (0-100), the StereoSet idealized context association score adapted to political bias: the language-model score LMS (share of items where the unrelated continuation is not chosen) times the neutrality score NS (how close the preference between the left or progressive and the right or conservative continuation is to balance, averaged over issue categories); options chosen by length-normalized log-likelihood in LM Evaluation Harness; higher means more balanced and competent; mean of the economic and sociocultural axes, South Korean political statements in Korean. Source: arxiv.org. Saturation forecast: Estimated already saturated. 38 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Llama 3.1 70B | 91.04 | #1490 |
| 2 | Midm-2.0-Mini-Instruct | 90.99 | #4735 |
| 3 | Mistral Small 3 | 90.77 | #1777 |
| 4 | Qwen 3 14B | 90.75 | #1233 |
| 5 | Qwen3-14B-Base | 90.45 | #1504 |
| 6 | Mistral-Small-24B-Base-2501 | 89.4 | #1450 |
| 7 | Llama 3.1 70B Instruct | 88.98 | #1546 |
| 8 | A.X-4.0-Light | 88.98 | #2254 |
| 9 | Qwen 3 8B | 88.7 | #1830 |
| 10 | Midm-2.0-Base-Instruct | 88.17 | #1921 |
| 11 | Qwen3-8B-Base | 87.9 | #1721 |
| 12 | Qwen 3 32B | 87.67 | #960 |
| 13 | A.X 4.0 | 87.09 | #863 |
| 14 | Llama 3.1 8B | 85.83 | #4861 |
| 15 | Qwen3-1.7B-Base | 84.93 | #4890 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=polar-south-korea · How It Works · Data refreshed daily, snapshot 2026-11-13.