LLMConfBench: leaderboard
Mean reliable-pair Kendall tau at a 2.0 kcal/mol energy-gap threshold, averaged over three prompt presentations then molecules. Most models cover 27 molecules; Qwen 2.5 32B and Qwen 3 32B cover 25 and 26. Source reasoning settings and Ollama identities are preserved; classical force-field reference rows are not LLMs.
Metric: Conformer Ranking Accuracy (mean reliable-pair Kendall tau). Source: arxiv.org. Saturation forecast: Around August 2027. 28 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-6 Astra (High) | 0.67 | #6 (GPT-6 Astra) |
| 2 | Claude Opus 5 (High) | 0.61 | #16 (Claude Opus 5) |
| 3 | Gemini 3.6 Flash (High) | 0.43 | #57 (Gemini 3.6 Flash) |
| 4 | GPT-5.6 Sol (High) | 0.4 | #22 (GPT-5.6 Sol) |
| 5 | Kimi K3 (High) | 0.38 | #35 (Kimi K3) |
| 6 | Claude Sonnet 5 (High) | 0.37 | #65 (Claude Sonnet 5) |
| 7 | DeepSeek V4 Flash (0731) (High) | 0.33 | #84 (DeepSeek V4 Flash (0731)) |
| 8 | Gemini 3 Flash (Preview) (High) | 0.32 | #90 (Gemini 3 Flash (Preview)) |
| 9 | Qwen 3.7 Max (Thinking) | 0.32 | #85 (Qwen 3.7 Max) |
| 10 | GPT-5 (High) | 0.3 | #105 (GPT-5) |
| 11 | Muse Glimmer 30B (High) | 0.3 | #191 (Muse Glimmer 30B) |
| 12 | Kimi K2 (Thinking) | 0.27 | #252 (Kimi K2) |
| 13 | Gemma 4 31B (High) | 0.27 | #212 (Gemma 4 31B) |
| 14 | Claude Sonnet 4.5 (Thinking) | 0.14 | #148 (Claude Sonnet 4.5) |
| 15 | DeepSeek V3.2 (Thinking) | 0.14 | #210 (DeepSeek V3.2) |
No result here: #3 Claude Opus 5.5, #7 GPT-6.1 Sol, #9 Gemini 4 Argon.
Interactive version: theaggregate.ai/benchmark?slug=llmconfbench · How It Works · Data refreshed daily, snapshot 2026-11-05.