TriBench-Ko - Nondeterminism: leaderboard
TriBench-Ko’s nondeterminism dimension repeats identical Korean legal yes/no prompts five times to probe answer stability. This board reports the paper’s macro-F1 over the resulting binary judgments on a 0–100 scale. Strict repeat accuracy is a separate complementary analysis; higher macro-F1 is better.
Metric: Macro-F1 (0-100) on the TriBench-Ko Nondeterminism risk items (identical prompts repeated five times; strict repeat accuracy is a separate complementary analysis): yes/no verification of statements about Korean court decisions, computed over the atomic binary judgments, zero-shot, temperature 0, at most 64 output tokens; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 13 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 92 | #135 |
| 2 | Qwen 3.5 9B | 88 | #828 |
| 3 | GPT-5.4 Mini | 84.9 | #406 |
| 4 | GPT-4o | 78.2 | #751 |
| 5 | Midm-2.0-Base-Instruct | 77.6 | #1988 |
| 6 | Phi-4 | 76.5 | #1931 |
| 7 | Ministral-3-8B-Instruct-2512 | 73.2 | #1907 |
| 8 | Qwen 3 8B | 59.6 | #1873 |
| 9 | Gemma 3 12B (IT) | 59.3 | #1887 |
| 10 | EXAONE 3.5 7.8B Instruct | 58.5 | #3410 |
| 11 | kanana-1.5-8B-instruct-2505 | 47.1 | #3315 |
| 12 | Llama 3.1 8B Instruct | 33.9 | #4600 |
| 13 | A.X-3.1-Light | 32.7 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=tribench-ko-nondeterminism · How It Works · Data refreshed daily, snapshot 2026-11-21.