MatPhaseBench: leaderboard
MatPhaseBench asks vision-language models to describe materials phase diagrams drawn from classical materials science journals, prompted with the semantic dimensions a human expert would address. This board is BERTScore recall, which measures how much of the expert description the generated text covers in meaning. The paper does not specify the optional baseline-rescaling setting. The reference is the expert description published with the diagram, and both metrics reward covering what the expert emphasized.
Metric: BERTScore recall (XLNet-large semantic recall, mean over 200 literature phase diagrams; zero-shot free-form description conditioned on annotated semantic dimensions, compared with expert source-paper descriptions; temperature 0). The paper does not specify the optional baseline-rescaling flag.. Source: arxiv.org. Saturation forecast: Around 2029. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.8 | 0.41 | #56 |
| 2 | GLM-5V Turbo | 0.4 | #344 |
| 3 | GPT-5.5 | 0.39 | #49 |
| 4 | Gemini 3.1 Pro (Preview) | 0.39 | #94 |
| 5 | Qwen 3.6 Plus | 0.39 | #222 |
| 6 | Qwen 3.6 27B | 0.38 | #341 |
| 7 | Qwen 3.6 35B A3B | 0.38 | #462 |
| 8 | Qwen 3.6 Flash | 0.38 | #424 |
| 9 | GLM-4.6V | 0.37 | #701 |
| 10 | Gemini 3.1 Flash Lite | 0.36 | #364 |
| 11 | Llama 4 Maverick | 0.3 | #1099 |
| 12 | Llama 4 Scout | 0.3 | #1776 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=matphasebench · How It Works · Data refreshed daily, snapshot 2026-11-13.