Mat-Pref - OOD-Property: leaderboard
Mat-Pref (Leung, Kim and Park, 2026) asks which element substitution at a crystallographic site of an inorganic host best raises or lowers its formation energy or band gap, with three or four candidates graded against Materials Project DFT values. This board covers 510 band-gap logical questions on hosts whose training questions covered only formation energy, each scored by majority vote over its distractor permutations, in percent (higher is better); rows include models the authors fine-tuned on the training split.
Metric: Logical-question accuracy (%; majority vote over each logical question's distractor permutations, three- or four-choice ionic substitution questions graded against Materials Project DFT values; 510 band-gap logical questions on training-family hosts). Source: arxiv.org. Saturation forecast: Estimated already saturated. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 235B A22B 2507 Instruct | 53.3 | #619 |
| 2 | DeepSeek V3 | 50.6 | #705 |
| 3 | Llama 3.3 70B Instruct | 46.3 | #1353 |
| 4 | Qwen 2.5 72B Instruct | 39.6 | #1093 |
| 5 | Llama 3.1 8B Instruct | 27.8 | #4277 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=mat-pref-ood-property · How It Works · Data refreshed daily, snapshot 2026-11-12.