Software World Model: leaderboard
Agents perform tasks in application databases. Outcomes distinguish completing the requested change from damaging unrelated state, and retain per-application and per-skill denominators.
Source: raw.githubusercontent.com.
Benchmark results
Results are published separately for each benchmark or evaluation setting.
- Software World Model Baselines - dolibarr - clean - bulk_mark_sent
- Software World Model Baselines - dolibarr - clean - composite_pay_oldest
- Software World Model Baselines - dolibarr - clean - count_report
- Software World Model Baselines - dolibarr - clean - create_client
- Software World Model Baselines - dolibarr - clean - create_invoice
- Software World Model Baselines - dolibarr - clean - cross_record_calc
- Software World Model Baselines - dolibarr - clean - edit_field
- Software World Model Baselines - dolibarr - clean - find_record
- Software World Model Baselines - dolibarr - clean - record_payment
- Software World Model Baselines - dolibarr (train) - clean - Clean Success
- Software World Model Baselines - dolibarr (train) - clean - Collateral Damage
- Software World Model Baselines - dolibarr (train) - clean - False Completion
- Software World Model Baselines - dolibarr (train) - clean - Irreversible Changes
- Software World Model Baselines - dolibarr (train) - clean - Task Success
- Software World Model Baselines - dolibarr (train) - modal - Clean Success
- Software World Model Baselines - dolibarr (train) - modal - Collateral Damage
- Software World Model Baselines - dolibarr (train) - modal - False Completion
- Software World Model Baselines - dolibarr (train) - modal - Irreversible Changes
- Software World Model Baselines - dolibarr (train) - modal - Task Success
- Software World Model Baselines - invoiceninja - clean - bulk_mark_sent
- Software World Model Baselines - invoiceninja - clean - composite_pay_oldest
- Software World Model Baselines - invoiceninja - clean - count_report
- Software World Model Baselines - invoiceninja - clean - create_client
- Software World Model Baselines - invoiceninja - clean - create_invoice
- Software World Model Baselines - invoiceninja - clean - cross_record_calc
- Software World Model Baselines - invoiceninja - clean - edit_field
- Software World Model Baselines - invoiceninja - clean - find_record
- Software World Model Baselines - invoiceninja - clean - record_payment
- Software World Model Baselines - invoiceninja (train) - clean - Clean Success
- Software World Model Baselines - invoiceninja (train) - clean - Collateral Damage
- Software World Model Baselines - invoiceninja (train) - clean - False Completion
- Software World Model Baselines - invoiceninja (train) - clean - Irreversible Changes
- Software World Model Baselines - invoiceninja (train) - clean - Task Success
- Software World Model Baselines - invoiceninja (train) - flaky_network - Clean Success
- Software World Model Baselines - invoiceninja (train) - flaky_network - Collateral Damage
- Software World Model Baselines - invoiceninja (train) - flaky_network - False Completion
- Software World Model Baselines - invoiceninja (train) - flaky_network - Irreversible Changes
- Software World Model Baselines - invoiceninja (train) - flaky_network - Task Success
- Software World Model Baselines - invoiceninja (train) - modal - Clean Success
- Software World Model Baselines - invoiceninja (train) - modal - Collateral Damage
- Software World Model Baselines - invoiceninja (train) - modal - False Completion
- Software World Model Baselines - invoiceninja (train) - modal - Irreversible Changes
- Software World Model Baselines - invoiceninja (train) - modal - Task Success
- Software World Model Baselines - invoiceninja (train) - session_expiry - Clean Success
- Software World Model Baselines - invoiceninja (train) - session_expiry - Collateral Damage
- Software World Model Baselines - invoiceninja (train) - session_expiry - False Completion
- Software World Model Baselines - invoiceninja (train) - session_expiry - Irreversible Changes
- Software World Model Baselines - invoiceninja (train) - session_expiry - Task Success
- Software World Model Baselines - odoo17 - clean - bulk_mark_sent
- Software World Model Baselines - odoo17 - clean - composite_pay_oldest
- Software World Model Baselines - odoo17 - clean - count_report
- Software World Model Baselines - odoo17 - clean - create_client
- Software World Model Baselines - odoo17 - clean - create_invoice
- Software World Model Baselines - odoo17 - clean - cross_record_calc
- Software World Model Baselines - odoo17 - clean - edit_field
- Software World Model Baselines - odoo17 - clean - find_record
- Software World Model Baselines - odoo17 - clean - record_payment
- Software World Model Baselines - odoo17 (held-out app) - clean - Clean Success
- Software World Model Baselines - odoo17 (held-out app) - clean - Collateral Damage
- Software World Model Baselines - odoo17 (held-out app) - clean - False Completion
- Software World Model Baselines - odoo17 (held-out app) - clean - Irreversible Changes
- Software World Model Baselines - odoo17 (held-out app) - clean - Task Success
- Software World Model Baselines - odoo17 (held-out app) - modal - Clean Success
- Software World Model Baselines - odoo17 (held-out app) - modal - Collateral Damage
- Software World Model Baselines - odoo17 (held-out app) - modal - False Completion
- Software World Model Baselines - odoo17 (held-out app) - modal - Irreversible Changes
- Software World Model Baselines - odoo17 (held-out app) - modal - Task Success
- Software World Model Baselines - odoo18 - clean - bulk_mark_sent
- Software World Model Baselines - odoo18 - clean - composite_pay_oldest
- Software World Model Baselines - odoo18 - clean - count_report
- Software World Model Baselines - odoo18 - clean - create_client
- Software World Model Baselines - odoo18 - clean - create_invoice
- Software World Model Baselines - odoo18 - clean - cross_record_calc
- Software World Model Baselines - odoo18 - clean - edit_field
- Software World Model Baselines - odoo18 - clean - find_record
- Software World Model Baselines - odoo18 - clean - record_payment
- Software World Model Baselines - odoo18 (held-out version) - clean - Clean Success
- Software World Model Baselines - odoo18 (held-out version) - clean - Collateral Damage
- Software World Model Baselines - odoo18 (held-out version) - clean - False Completion
- Software World Model Baselines - odoo18 (held-out version) - clean - Irreversible Changes
- Software World Model Baselines - odoo18 (held-out version) - clean - Task Success
Interactive version: theaggregate.ai/benchmark?slug=software-world-model · How It Works · Data refreshed daily, snapshot 2026-11-17.