TriBench-Ko - Overcompliance: leaderboard

TriBench-Ko’s overcompliance dimension compares Korean legal yes/no judgments under neutral instructions and leading instructions that suggest a particular position. It tests whether the model shifts toward that position. This board reports the paper’s binary-judgment macro-F1 on a 0–100 scale; higher is better.

Metric: Macro-F1 (0-100) on the TriBench-Ko Overcompliance risk items (instruction variants with a leading prompt that pushes a particular position): yes/no verification of statements about Korean court decisions, computed over the atomic binary judgments, zero-shot, temperature 0, at most 64 output tokens; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.483#135
2GPT-5.4 Mini74.2#406
3Qwen 3.5 9B72.1#828
4GPT-4o69.2#751
5Ministral-3-8B-Instruct-251267.8#1907
6Midm-2.0-Base-Instruct66.8#1988
7Phi-466.5#1931
8Qwen 3 8B65.2#1873
9Gemma 3 12B (IT)64.5#1887
10EXAONE 3.5 7.8B Instruct63.6#3410
11kanana-1.5-8B-instruct-250561.9#3315
12Llama 3.1 8B Instruct48.3#4600
13A.X-3.1-Light36.1

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=tribench-ko-overcompliance · How It Works · Data refreshed daily, snapshot 2026-11-21.