3D-DefectBench - Geometry (Expert Labels): leaderboard
3D-DefectBench measures how well vision-language models spot defects in generated 3D assets from rendered multi-view images, marking each defect type present or absent. This board covers the five geometry defects, scored against the cells where both expert annotators agree. Scores are the Matthews correlation coefficient, macro-averaged over defect types, so zero means chance agreement and one means perfect agreement.
Metric: Macro MCC (Matthews correlation from -1 to 1, macro-averaged over the five geometry defect types) of the model's binary per-defect verdicts against expert labels on the cells both experts agree on (129-asset expert split); the fixed c004 design: six-view oblique RGB turntable renders of each text-to-3D asset and a rubric-guided checklist prompt. Source: arxiv.org. Saturation forecast: Around August 2028. 12 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3.1 Pro (Preview) | 0.3 | #94 |
| 2 | Gemini 2.5 Pro | 0.28 | #266 |
| 3 | Gemini 3.1 Flash Lite | 0.25 | #364 |
| 4 | GPT-5.4 | 0.22 | #123 |
| 5 | Claude Opus 4.7 | 0.18 | #79 |
| 6 | GPT-4o | 0.16 | #741 |
| 7 | Qwen 3.5 397B A17B | 0.16 | #246 |
| 8 | Claude Sonnet 4.6 | 0.14 | #161 |
| 9 | GPT-5 Mini | 0.14 | #348 |
| 10 | Mistral Small 3.1 | 0.05 | #1689 |
| 11 | Qwen 2.5 VL 7B | 0.02 | #1787 |
| 12 | Claude Haiku 4.5 | 0.01 | #568 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=3d-defectbench-geometry-expert-labels · How It Works · Data refreshed daily, snapshot 2026-11-13.