BIM-Edit - Semantics: leaderboard
BIM-Edit (2026) has LLM agents edit Building Information Models in IFC format from natural-language instructions (create, update and delete; direct, spatial or topological references) over 324 tasks. This board is the semantic score on a 0 to 100 scale, whether element classes and properties match the reference; higher is better.
Metric: Semantic score (0-100): IFC class match and task-relevant property match of the edited elements against the reference (unmatched references score zero), on 324 natural-language create, update and delete tasks on 11 real and 36 synthetic IFC building models (direct, spatial and topological instructions), the model edits the IFC file as an agent that runs Python through a single IfcOpenShell execution tool with a 20-call budget, temperature 0 where settable; mean over tasks; higher is better. Source: arxiv.org. Saturation forecast: Around 2029. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Flash (Preview) | 41.81 | #144 |
| 2 | GPT-5.4 Mini | 38.8 | #389 |
| 3 | GPT-5.4 Pro (xHigh) | 38.69 | #85 (GPT-5.4 Pro) |
| 4 | Qwen 3.6 Plus | 35.93 | #222 |
| 5 | DeepSeek V3.2 | 34.8 | #406 |
| 6 | Claude Sonnet 4.6 | 33.85 | #161 |
| 7 | Gemma 4 31B (IT) | 29.94 | #342 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=bim-edit-semantics · How It Works · Data refreshed daily, snapshot 2026-11-13.