FOCUS - Deontic SARA-N: leaderboard
FOCUS v1 evaluates task-specific reinforcement-learning and compaction systems, distinct from unmodified backbones. Deontic has 338 held-out cases (150 Airline, 50 SARA-N, and 138 SARA-B, including 71 overlapping Hard cases). BrowseComp uses a fixed corpus with 249 held-out tasks from 830 test tasks; LocBench uses its held-out file-localization protocol. The generation budget is matched at one epoch. Qwen3.5-27B training uses LoRA and is limited to Deontic; its untrained reference results also appear on the other tasks. BrowseComp reports the best held-out checkpoint during training, which is not an unbiased final-checkpoint estimate. Exact source run identities are retained. LocBench GRPO File@1 and File@5 use distinct checkpoints, 1 and 89; FOCUS uses checkpoint 35. The 27B reference remains evaluated even where 27B training was outside the study. Metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 10 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | FOCUS Deontic Qwen3.5-27B FOCUS | 62 | |
| 2 | FOCUS Deontic reference Qwen3.5-27B | 60 | |
| 3 | FOCUS Deontic Qwen3.5-27B GRPO | 60 | |
| 4 | FOCUS Deontic Qwen3.5-27B VinePPO | 58 | |
| 5 | FOCUS Deontic Qwen3.5-9B GRPO | 56 | |
| 6 | FOCUS Deontic Qwen3.5-9B FOCUS | 52 | |
| 7 | FOCUS Deontic Qwen3.5-9B VinePPO | 50 | |
| 8 | FOCUS Deontic reference Qwen3.5-9B | 30 | |
| 9 | FOCUS Deontic reference Nemotron 3 Nano 30B A3B | 12 | |
| 10 | FOCUS Deontic reference gpt-oss-20b (medium) | 8 |
Interactive version: theaggregate.ai/benchmark?slug=focus-deontic-sara-n · How It Works · Data refreshed daily, snapshot 2026-10-28.