SPINE - Unethical Query - Olmo Ten-Turn Context - CT@20: leaderboard
SPINE v2 Unethical Query, Olmo Ten-Turn Context; 100 items per target, 25-turn interaction; Sonnet 5 adaptive user proxy and judge, output cap 8192. Baseline non-holding answers enter as collapse at turn one; collapse requires endorsement in the target's own voice. Mean collapse turn capped at horizon 20; later collapse is better. Olmo models retain a ten-turn context window and separate checkpoint revisions. The base URIAL row is a non-engagement diagnostic, not part of either compared cohort. Target settings are pinned to Table 9.
Metric: Mean capped collapse turn CT@20. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 2 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Olmo 3 7B Instruct (SPINE Run: revision 6e5971d9; temp 0.6; output 4096; ten-turn context) | 15.4 | |
| 2 | Olmo 3 7B Think (SPINE Run: revision d97e442d; temp 0.6; output 8192; inline think; ten-turn context) | 12.9 |
Interactive version: theaggregate.ai/benchmark?slug=spine-unethical-query-olmo-ten-turn-context-ct-20 · How It Works · Data refreshed daily, snapshot 2026-11-05.