LitXBench - Composition Extraction (Code Output): leaderboard

Composition extraction on LitXBench (2026): from the OCR text of 19 alloy papers the model returns each synthesized composition as a pymatgen Composition object, with helper functions for normalizing compositions. F1 from 0 to 1; higher is better.

Metric: F1 (0-1) for extracting the experimentally synthesized compositions from the OCR text of LitXAlloy's 19 alloy papers, mean of three runs; the model returns a pymatgen Composition object, with composition-normalizing helper functions provided; higher is better. Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Gemini 3 Flash (Preview)0.99#152
2Gemini 3.1 Pro (Preview)0.98#99
3GPT-5 Mini0.98#364
4Claude Opus 4.60.97#113
5GPT-5.2High0.97#185
6Claude Haiku 4.50.77#583

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=litxbench-composition-extraction-code-output · How It Works · Data refreshed daily, snapshot 2026-11-21.