BABILong (NIAH, v0, 32k) Leaderboard · The Aggregate

BABILong (NIAH, v0, 32k): leaderboard

Metric: QA1–5 Mean Accuracy (%). Source: huggingface.co. Saturation forecast: Estimated already saturated. 25 models tracked.

Top models

#	Model	Score
1	Llama 3.1 70B Instruct	59
2	Llama 3.1 8B Instruct	56
3	Phi-3-medium-128k-instruct	53
4	GPT-4 Preview (0125)	53
5	c4ai-command-r-v01	51
6	Yi-34B-200K	48
7	Mixtral 8x22B Instruct (v0.1)	43
8	Phi-3 Mini 128K Instruct	42
9	Mixtral 8x7B Instruct (v0.1)	40
10	Mistral 7B Instruct (v0.2)	37
11	Yi-9B-200K	37
12	Yarn-Mistral-7B-128k	16
13	LongChat-7B-v1.5-32k	5
14	Llama 2 7B 32K Instruct	5
15	Yi 34B (Base)	4

Interactive version: theaggregate.ai/benchmark?slug=babilong-niah-v0-32k · How It Works · Data refreshed daily, snapshot 2026-10-09.