Mat-Pref - OOD-Host: leaderboard
Mat-Pref (Leung, Kim and Park, 2026) asks which element substitution at a crystallographic site of an inorganic host best raises or lowers its formation energy or band gap, with three or four candidates graded against Materials Project DFT values. This board covers 282 logical questions on three structure families held out from training entirely (garnet, halide perovskite, NASICON), each scored by majority vote over its distractor permutations, in percent (higher is better); rows include models the authors fine-tuned on the training split.
Metric: Logical-question accuracy (%; majority vote over each logical question's distractor permutations, three- or four-choice ionic substitution questions graded against Materials Project DFT values; 282 logical questions on three structure families held out entirely: garnet, halide perovskite, NASICON). Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 235B A22B 2507 Instruct | 45.4 | #619 |
| 2 | DeepSeek V3 | 36.5 | #705 |
| 3 | Llama 3.3 70B Instruct | 35.1 | #1353 |
| 4 | Qwen 2.5 72B Instruct | 33.3 | #1093 |
| 5 | Llama 3.1 8B Instruct | 27.3 | #4277 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mat-pref-ood-host · How It Works · Data refreshed daily, snapshot 2026-11-12.