NimCritic LOOM - Single Task Terminal Success Accuracy: leaderboard
NimCritic v1 visual manipulation-progress evaluation, not action execution. LOOM: 300 trajectories, 19,618 calibrated frames, 171 single-task and 129 multi-task; duration-weighted Spearman forward/reverse correlations, signed harmonic PRC-F1 (zero on opposite signs) and terminal completion accuracy with reference progress > 0.99. Native baseline interfaces/output definitions retained. GVL shuffles up to 30 frames, temperature 1; GPT-6 Sol extra reasoning disabled. NimCritic is a separately trained Wan2.2-TI2V-5B-initialized critic with frozen UMT5-XXL,128,000 updates and 20 Euler inference steps; Gemini 3.8 Flash decomposes subtasks. Other checkpoint revisions/efforts unreported; no aliases to untrained backbones. Metric: Terminal Success Accuracy.
Metric: Terminal Success Accuracy. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | NimCritic (Harness: Gemini 3.8 Flash subtask decomposition) | 0.88 | |
| 2 | NimCritic study Robometer 4B | 0.67 | |
| 3 | NimCritic study GPT-6 Sol (no reasoning) (Harness: GVL) | 0.65 | |
| 4 | Gemini 3.8 Flash (Harness: GVL) | 0.57 | |
| 5 | NimCritic study ProcVLM 2B | 0.52 | |
| 6 | NimCritic study VLAC-CUT | 0.51 | |
| 7 | NimCritic study LRM | 0.41 | |
| 8 | NimCritic study RoboDopamine-GRM-2 8B | 0.35 | |
| 9 | NimCritic study RynnValue 4B | 0.31 | |
| 10 | NimCritic study VLAC-Critic 8B | 0.31 |
Interactive version: theaggregate.ai/benchmark?slug=nimcritic-loom-single-task-terminal-success-accuracy · How It Works · Data refreshed daily, snapshot 2026-10-27.