TriBench-Ko - Demographic Bias: leaderboard

TriBench-Ko’s demographic-bias dimension uses paired Korean legal documents that differ in a protected attribute while legally relevant facts stay fixed. Models make yes/no judgments about the cases; the paper reports macro-F1 on the binary judgments, on a 0–100 scale here. Higher is better.

Metric: Macro-F1 (0-100) on the TriBench-Ko Demographic Bias risk items (paired document variants that change only a legally irrelevant protected attribute): yes/no verification of statements about Korean court decisions, computed over the atomic binary judgments, zero-shot, temperature 0, at most 64 output tokens; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.484.5#135
2GPT-5.4 Mini83.9#406
3GPT-4o78.4#751
4Phi-478.1#1931
5Qwen 3.5 9B75.6#828
6Qwen 3 8B74.2#1873
7Gemma 3 12B (IT)73.5#1887
8kanana-1.5-8B-instruct-250570.5#3315
9Ministral-3-8B-Instruct-251269.9#1907
10EXAONE 3.5 7.8B Instruct69.9#3410
11Midm-2.0-Base-Instruct67#1988
12Llama 3.1 8B Instruct43.2#4600
13A.X-3.1-Light35.7

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=tribench-ko-demographic-bias · How It Works · Data refreshed daily, snapshot 2026-11-21.