FOCUS - Deontic Hard: leaderboard
FOCUS v1 evaluates task-specific reinforcement-learning and compaction systems, distinct from unmodified backbones. Deontic has 338 held-out cases (150 Airline, 50 SARA-N, and 138 SARA-B, including 71 overlapping Hard cases). BrowseComp uses a fixed corpus with 249 held-out tasks from 830 test tasks; LocBench uses its held-out file-localization protocol. The generation budget is matched at one epoch. Qwen3.5-27B training uses LoRA and is limited to Deontic; its untrained reference results also appear on the other tasks. BrowseComp reports the best held-out checkpoint during training, which is not an unbiased final-checkpoint estimate. Exact source run identities are retained. LocBench GRPO File@1 and File@5 use distinct checkpoints, 1 and 89; FOCUS uses checkpoint 35. The 27B reference remains evaluated even where 27B training was outside the study. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | FOCUS Deontic reference Qwen3.5-27B | 43.7 | |
| 2 | FOCUS Deontic Qwen3.5-27B GRPO | 43.7 | |
| 3 | FOCUS Deontic Qwen3.5-27B FOCUS | 43.7 | |
| 4 | FOCUS Deontic Qwen3.5-27B VinePPO | 36.6 | |
| 5 | FOCUS Deontic Qwen3.5-9B GRPO | 28.2 | |
| 6 | FOCUS Deontic Qwen3.5-9B FOCUS | 26.8 | |
| 7 | FOCUS Deontic Qwen3.5-9B VinePPO | 24 | |
| 8 | FOCUS Deontic reference Qwen3.5-9B | 19.7 | |
| 9 | FOCUS Deontic reference Nemotron 3 Nano 30B A3B | 18.3 | |
| 10 | FOCUS Deontic reference gpt-oss-20b (medium) | 14.1 |
Interactive version: theaggregate.ai/benchmark?slug=focus-deontic-hard · How It Works · Data refreshed daily, snapshot 2026-10-28.