NARU (Open-Ended): leaderboard
The open-ended variant of NARU (2026): 500 stratified questions on long-form Japanese video with the answer options removed, so the model writes a free-form answer. GPT-5.5 splits each reference answer into atomic facts and checks which ones the answer covers; the score is the covered share in percent, averaged over the nine categories; higher is better.
Metric: Atomic-fact recall (%; x100 of the 0-1 FActScore-style recall: share of the reference answer's atomic facts that the free-form answer covers, judged by GPT-5.5, mean of the nine categories over a stratified 500-question subset of NARU with the answer options removed). Source: arxiv.org. Saturation forecast: Around February 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash | 78 | #159 |
| 2 | Gemini 3 Pro | 75 | #135 |
| 3 | Gemini 2.5 Flash | 66 | #452 |
| 4 | Qwen 3.5 9B | 56 | #803 |
| 5 | Qwen 2.5 VL 7B | 35 | #1796 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=naru-open-ended · How It Works · Data refreshed daily, snapshot 2026-11-12.