Diverge Olmo Replication - At Least One Win: leaderboard
Diverge v1 held-out1,740 instances from2026 papers,16domains; three idea cards generated as one set except DARLING three single-card draws. Qwen custom variants separate, proposed checkpoint600; generic Nemotron120B revision unresolved, never inferred as current Nemotron3Super. Impact eval GPT5.4mini both candidate/gold orderings; distance eval Gemini3.5Flash,temp0,max1024 (training uses different judges); LC win rate length-controlled pooled logistic adjustment. CU greedy-order patience0.9 quality-times-distance sum divided by1+0.9+0.9². Grounding/distance are diagnostics, not monotonically better capability. Dataset/code release currently coming soon. Table14 recipe transfer starts exact allenai/Olmo-3-7B-Think-SFT; same1,740 held-out tasks, source custom variants remain distinct. Metric: Fraction of instances with at least one winning idea (0–1). Higher is better. Source: https://arxiv.org/html/2610.22324v1
Metric: Fraction of instances with at least one winning idea (0–1). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Diverge Olmo-3-7B SFT-RL (no think) | 0.67 | |
| 2 | Diverge Olmo-3-7B SFT-RL (think) | 0.63 | |
| 3 | Diverge Olmo-3-7B SFT (no think) | 0.11 | |
| 4 | Olmo-3-7B-Think-SFT | 0.06 | |
| 5 | Diverge Olmo-3-7B SFT (think) | 0.06 |
Interactive version: theaggregate.ai/benchmark?slug=diverge-olmo-replication-at-least-one-win · How It Works · Data refreshed daily, snapshot 2026-10-17.