NimCritic LOOM Segmentation - Adjusted Rand Index: leaderboard

NimCritic v1 visual manipulation-progress evaluation, not action execution. LOOM: 300 trajectories, 19,618 calibrated frames, 171 single-task and 129 multi-task; duration-weighted Spearman forward/reverse correlations, signed harmonic PRC-F1 (zero on opposite signs) and terminal completion accuracy with reference progress > 0.99. Native baseline interfaces/output definitions retained. GVL shuffles up to 30 frames, temperature 1; GPT-6 Sol extra reasoning disabled. NimCritic is a separately trained Wan2.2-TI2V-5B-initialized critic with frozen UMT5-XXL,128,000 updates and 20 Euler inference steps; Gemini 3.8 Flash decomposes subtasks. Other checkpoint revisions/efforts unreported; no aliases to untrained backbones. Appendix C compares identical offline subtask segmentation prompts. Boundary F1 uses adaptive median-segment tolerance; one-to-one cover uses Hungarian matching; ARI is chance-adjusted frame partition agreement. No reasoning effort for segmenter endpoints is separately reported. Metric: Adjusted Rand Index.

Metric: Adjusted Rand Index. Source: arxiv.org. Saturation forecast: Around January 2027. 3 models tracked.

Top models

No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #6 GPT-6.1 Sol.

Interactive version: theaggregate.ai/benchmark?slug=nimcritic-loom-segmentation-adjusted-rand-index · How It Works · Data refreshed daily, snapshot 2026-10-27.