Complement Consistency - Baseline Round Average Brier Loss: leaderboard
Complement-consistency paper v1: visible-evidence factual Brier loss across six controlled worlds after five feedback rounds. llama.cpp, 4,096-token context, 192 output tokens, greedy temperature 0/top-p 1/top-k 0/min-p 0/repetition penalty 1. Source quantization and prompt-role differences retained; higher loss is worse. Baseline and reminder are distinct prompting protocols. Symmetrization is a paired two-orientation estimator. Greedy-average rows exactly duplicate baseline and are not counted again. Baseline; Round Average. Metric: Brier loss.
Metric: Brier loss. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Complement Consistency Qwen3-4B-Instruct-2507 Q4_K_M | 0.17 | |
| 2 | Complement Consistency Llama-3.2-3B-Instruct Q4_K_M | 0.2 | |
| 3 | Complement Consistency Gemma-3-4B-IT QAT Q4_0 | 0.26 |
Interactive version: theaggregate.ai/benchmark?slug=complement-consistency-baseline-round-average-brier-loss · How It Works · Data refreshed daily, snapshot 2026-10-27.