Diagnosing LLM Arbitration Behavior over Pre-e: leaderboard
In RAG-based fact-checking, LLMs are increasingly used as verifiers to check given claims against retrieved evidence. Their parametric knowledge can induce pre-evidence tendencies that may conflict with the retrieved context, yet existing evaluation frameworks do not characterize such prior-context discrepancy or measure how verifiers arbitrate between parametric and contextual signals. We introdu
Metric: Margin (self-reported). Source: benchmarklist.com. Saturation forecast: Estimated already saturated. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | DeepSeek V3 | 23.5 | #321 |
| 2 | Phi-4 | -0.4 | #706 |
| 3 | Qwen 3 32B | -11.5 | #443 |
| 4 | Mistral 7B | -19 | #1397 |
| 5 | GPT-4o Mini | -52.1 | #599 |
| 6 | Llama 3 8B | -66.1 | #1242 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=diagnosing-llm-arbitration-behavior-over-pre-e · How It Works · Data refreshed daily, snapshot 2026-10-20.