OmniToM - Belief Labeling - Content Type: leaderboard
OmniToM (2026) evaluates theory of mind through explicit belief modeling on 895 short social stories from ToMBench, with gold belief propositions built by a human-calibrated annotation pipeline. This board is Stage 2 labeling accuracy for the content type (location, physical state, identity, epistemic, desire, emotion, trait or event) dimension of the given belief propositions; higher is better.
Metric: Belief-labeling accuracy (%): exact-match accuracy of the predicted schema label on the given gold belief propositions, macro-averaged over stories, for the content type (location, physical state, identity, epistemic, desire, emotion, trait or event) dimension, on the 895 OmniToM stories from seven ToMBench categories (22,343 gold belief propositions), zero-shot TELeR level 3 prompts; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked. Updated 7 Oct 2026.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Flash | 85.97 | #463 |
| 2 | Mistral Large 2 (Jul) | 82.83 | #1144 |
| 3 | GPT-5 | 79.96 | #178 |
| 4 | Mistral Small 3 | 76.01 | #1796 |
| 5 | Gemma 3 27B (IT) | 73.5 | #1324 |
| 6 | Llama 3.3 70B Instruct | 72.35 | #1385 |
| 7 | Qwen 3 32B | 71.27 | #972 |
| 8 | Qwen 3 8B | 51.43 | #1873 |
| 9 | Llama 3.1 8B Instruct | 48.63 | #4600 |
No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.
Interactive version: theaggregate.ai/benchmark?slug=omnitom-belief-labeling-content-type · How It Works · Data refreshed daily, snapshot 2026-11-21.