OmniToM - Belief Labeling - Content Type: leaderboard

OmniToM (2026) evaluates theory of mind through explicit belief modeling on 895 short social stories from ToMBench, with gold belief propositions built by a human-calibrated annotation pipeline. This board is Stage 2 labeling accuracy for the content type (location, physical state, identity, epistemic, desire, emotion, trait or event) dimension of the given belief propositions; higher is better.

Metric: Belief-labeling accuracy (%): exact-match accuracy of the predicted schema label on the given gold belief propositions, macro-averaged over stories, for the content type (location, physical state, identity, epistemic, desire, emotion, trait or event) dimension, on the 895 OmniToM stories from seven ToMBench categories (22,343 gold belief propositions), zero-shot TELeR level 3 prompts; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Flash85.97#463
2Mistral Large 2 (Jul)82.83#1144
3GPT-579.96#178
4Mistral Small 376.01#1796
5Gemma 3 27B (IT)73.5#1324
6Llama 3.3 70B Instruct72.35#1385
7Qwen 3 32B71.27#972
8Qwen 3 8B51.43#1873
9Llama 3.1 8B Instruct48.63#4600

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=omnitom-belief-labeling-content-type · How It Works · Data refreshed daily, snapshot 2026-11-21.