MixRea - Implicit Information Accuracy: leaderboard

MixRea (2026) tests whether LLMs attend to implicit story cues while answering an explicit hypothetical question, on 2,246 four-option questions built from the Possible Stories dataset. This board is the share of questions where the chosen option reasons correctly with the implicit context information; percent, higher is better.

Metric: Implicit information accuracy (IIA, %): share of questions on which the selected option reasons correctly with the implicit information in the story context, on the 2,246 MixRea four-option explicit-implicit reasoning questions over 673 story contexts from the Possible Stories dataset, read from which of the designed options the model selects; higher is better. Source: arxiv.org. Saturation forecast: Around March 2027. 21 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Gemini 2.5 Pro76.5#276
2Llama 3.1 405B71.1#1082
3DeepSeek R170.9#482
4Claude 3.7 Sonnet70.1#470
5Llama 3.1 70B67.6#1499
6Gemini 2.5 Flash67.1#463
7GPT-4o66.6#751
8Qwen 2.5 32B66.4#1344
9DeepSeek V365.4#729
10Llama 3 70B65.4#1655
11Gemma 3 27B65.2#1663
12Gemma 2 27B65.1#2141
13Qwen Max65#1092
14Qwen 2 7B64.2#3790
15Qwen 2.5 72B63.9#1188

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=mixrea-implicit-information-accuracy · How It Works · Data refreshed daily, snapshot 2026-11-21.