Exotic fact recall
Closed-book lookups from five public registries: proteins, ISBNs, museum objects, asteroids and network ports.
Run Sep 8–9, 2026, 85 items.
Method
Each question asks for one exact field from a public registry: a UniProt protein entry, the title behind an ISBN, the accession number of a painting at the Met, the number of a named asteroid, or the TCP port that IANA assigns to a service. Every key is exact, so no judge is involved.
Each registry contributes well-known entries and a random sample from its long tail. The score counts the 85 questions that every model in the original panel answered, 49 of them from the tail. Reasoning effort was set to high.
Results
| # | Model | Accuracy |
|---|---|---|
| 1 | GPT-5.6 Sol | 63.5% |
| 2 | Claude Fable 5.1 | 62.4% |
| 3 | Claude Fable 5 | 61.2% |
| 4 | GPT-6 Astra | 56.5% |
| 5 | Gemini 3.7 Flash | 55.3% |
| 6 | Claude Opus 5 | 50.6% |
| 7 | DeepSeek V4 Pro | 49.4% |
| 8 | GPT-5.4 | 41.2% |
| 9 | GPT-5.4 Mini | 36.5% |
| 10 | Qwen 3.6 Plus | 31.8% |
| 11 | Qwen 3.6 35B A3B | 27.1% |
| 12 | Claude Haiku 4.5 | 23.5% |
- GPT-5.6 Sol, GPT-6 Astra, Claude Fable 5.1 and Qwen 3.6 Plus were added on September 9 and scored on the same 85 questions.
- Not ranked, too many answers cut off by the output limit: GPT-OSS-120B (64 of 85).
In the main leaderboard
These results are collected for the main leaderboard alongside external benchmarks. New scores enter the rating through the regular audited data updates. Comparisons with that rating therefore share some of its inputs.
Interactive version: theaggregate.ai/our-benchmarks/exotic-recall · How It Works · Data refreshed daily, snapshot 2026-10-11.