IDEAL-Bench: leaderboard
IDEAL-Bench (2026) tests holistic 3D indoor layout inference from one image: on 1,000 photorealistic scenes across 10 room types, a vision-language model outputs the room type, the room size and the category, 3D position, size and yaw of every visible object as JSON. The overall score is the unweighted mean of 11 rate metrics on scene validity, physical plausibility, geometric accuracy, object recognition and grid layout, from 0 to 100; higher is better.
Metric: Overall score (0-100): unweighted mean of the 11 rate metrics on 1,000 photorealistic indoor scenes across 10 room types, where a VLM predicts the room type, room size and every visible object's category, 3D position, size and yaw as JSON from one image (zero-shot, temperature 0.1, no chain-of-thought); higher is better. Source: arxiv.org. Saturation forecast: Around September 2027. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Pro | 62.1 | #266 |
| 2 | GPT-4o | 60.4 | #741 |
| 3 | Gemma 4 31B (IT) | 59.7 | #342 |
| 4 | Claude Sonnet 4.6 | 59.2 | #161 |
| 5 | GPT-5.4 | 56.7 | #123 |
| 6 | Qwen 3 VL 8B Instruct | 55.1 | #981 |
| 7 | GLM-4.6V | 54.4 | #701 |
| 8 | Qwen 2.5 VL 72B Instruct | 52.6 | #846 |
| 9 | Qwen 3 VL 235B A22B Instruct | 51.8 | #522 |
| 10 | Qwen 3 VL 30B A3B Instruct | 50.8 | #871 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=ideal-bench · How It Works · Data refreshed daily, snapshot 2026-11-13.