PointQ-Bench - Defect Diagnosis: leaderboard

Multi-label diagnosis of which of eight quality defects a point cloud has, on 3,083 point clouds from real scans, simulated distortions and AI-generated assets (PointQ-Bench, 2026), with 2D models seeing six rendered views and 3D models sampled points. Sample-level F1 in percent; higher is better.

Metric: Sample-level F1 (%) of the multi-label defect diagnosis (which of eight issue types are present; no defect is an empty set), zero-shot over 3,083 point clouds (authentic scans, simulated distortions, AI-generated assets), temperature 0; 2D models see six rendered views, native 3D models 8,192 sampled points; free-form answers mapped to the label space by an LLM parser; higher is better. Source: arxiv.org. Saturation forecast: Around 2030. 14 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-5Low42.32#178
2Qwen 3.5 9BNon-reasoning39.95#828
3GPT-5 MiniLow39.6#364
4Gemma 3 12B37.3#2059
5Claude Haiku 4.536.88#583
6InternVL3.5-8B34.83#1604
7Mistral Small 3.234.3#1641
8Qwen 3.5 27BNon-reasoning33.87#375
9ShapeLLM-13B32.42
10InternVL3.5-14B30.24#1504
11PointLLM-7B28.08
12ShapeLLM-7B26.22
13PointLLM-13B14.95
14MiniGPT-3D10.96

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=pointq-bench-defect-diagnosis · How It Works · Data refreshed daily, snapshot 2026-11-21.