TriBench-Ko - Nondeterminism: leaderboard

TriBench-Ko’s nondeterminism dimension repeats identical Korean legal yes/no prompts five times to probe answer stability. This board reports the paper’s macro-F1 over the resulting binary judgments on a 0–100 scale. Strict repeat accuracy is a separate complementary analysis; higher macro-F1 is better.

Metric: Macro-F1 (0-100) on the TriBench-Ko Nondeterminism risk items (identical prompts repeated five times; strict repeat accuracy is a separate complementary analysis): yes/no verification of statements about Korean court decisions, computed over the atomic binary judgments, zero-shot, temperature 0, at most 64 output tokens; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 13 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1GPT-5.492#135
2Qwen 3.5 9B88#828
3GPT-5.4 Mini84.9#406
4GPT-4o78.2#751
5Midm-2.0-Base-Instruct77.6#1988
6Phi-476.5#1931
7Ministral-3-8B-Instruct-251273.2#1907
8Qwen 3 8B59.6#1873
9Gemma 3 12B (IT)59.3#1887
10EXAONE 3.5 7.8B Instruct58.5#3410
11kanana-1.5-8B-instruct-250547.1#3315
12Llama 3.1 8B Instruct33.9#4600
13A.X-3.1-Light32.7

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=tribench-ko-nondeterminism · How It Works · Data refreshed daily, snapshot 2026-11-21.