MatPhaseBench - ROUGE-1 Recall: leaderboard
MatPhaseBench asks vision-language models to describe materials phase diagrams drawn from classical materials science journals, prompted with the semantic dimensions a human expert would address. This board is ROUGE-1 recall, the share of the words in the expert description that the generated text also uses. The reference is the expert description published with the diagram, and both metrics reward covering what the expert emphasized.
Metric: ROUGE-1 recall (%; unigram recall of the expert description, mean over the 200 diagrams; free-form description of each of the 200 literature phase diagrams, prompted zero-shot with its annotated semantic dimensions, compared with the expert description from the source paper; temperature 0). Source: arxiv.org. Saturation forecast: Around August 2028. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.8 | 43.8 | #56 |
| 2 | GPT-5.5 | 43 | #49 |
| 3 | GLM-5V Turbo | 40 | #344 |
| 4 | Gemini 3.1 Pro (Preview) | 39 | #94 |
| 5 | Qwen 3.6 Plus | 35.6 | #222 |
| 6 | Qwen 3.6 35B A3B | 35 | #462 |
| 7 | Qwen 3.6 27B | 34.9 | #341 |
| 8 | Qwen 3.6 Flash | 34.9 | #424 |
| 9 | Gemini 3.1 Flash Lite | 31.1 | #364 |
| 10 | GLM-4.6V | 30.8 | #701 |
| 11 | Llama 4 Scout | 23.9 | #1776 |
| 12 | Llama 4 Maverick | 23.5 | #1099 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=matphasebench-rouge-1-recall · How It Works · Data refreshed daily, snapshot 2026-11-13.