Software World Model Baselines - invoiceninja (train) - clean - Irreversible Changes: leaderboard

Native Phase0 zero-shot app-control tasks; 30-step budget and one run per task. GPT-6 Luna API uses accessibility tree+screenshot/element IDs; UI-TARS-1.5-7B uses screenshots/native coordinates via adapter and vLLM A100. Nine skills, nominal eight tasks each; UI-TARS other apps four per skill. Exact native n per row retained.12 InvoiceNinja Luna episodes affected by fallback bug are excluded; other affected episodes were rerun, originals superseded. This is source failure policy, not complete-cohort success. App train/held-out labels refer to the separate world-model track, not LLM training. GPT-6 Luna n=67; UI-TARS-1.5-7B n=72. Success is evaluator-verified task state; clean success additionally forbids collateral changes. Timeouts and wrong actions remain failures. No cost, latency, or latent transition-model scores. Metric: Native episode rate (%). Source: https://raw.githubusercontent.com/shubhamjoshipromail-svg/software-world-model/7fa10cf4f7d294ba80e8d6c357f77d3a6340c3ce/reports/phase0_results.md

Metric: Native episode rate (%). Source: raw.githubusercontent.com. Saturation forecast: Around 2029. 2 models tracked. Updated 17 Nov 2026.

Top models

#ModelScoreOverall rank
1uitars-1.5-7B3#7798
2GPT-6 Luna0#105

No result here: #1 Claude Fable 5.2, #2 GPT-6.2 Astra, #3 Gemini 4 Carbon.

Interactive version: theaggregate.ai/benchmark?slug=software-world-model-baselines-invoiceninja-train-clean-irreversible-changes · How It Works · Data refreshed daily, snapshot 2026-11-17.