Decisio Frozen Readout - Suite1400 SciFact Accuracy - Plain Layout: leaderboard
Decisio Frozen Readout - Suite1400 SciFact Accuracy - Plain Layout: Accuracy fraction (0-1). Decisio official EVAL_CARD author-run frozen checkpoint evaluations. Every card figure uses historical plain state-then-question layout, not current header-layout defaults. Qwen3.6-35B-A3B-FP8@95a723d08a9490559dae23d0cff1d9466213d989; Gemma4-12B-it@707f0a3b8a3c7ad586ed01e27eafbad8a27dd0f7 bf16; Gemma4-31B-it@842da3794eaa0b77d5f08bae87a17459d91ff475 quantized FP8 on load by vLLM0.30.0 (not inferred identical to later stored FP8 artifact). No backbone training. Native letter readouts/temperatures, calibration fitting provenance and run records pinned in complete setup/fit tables. Separate231publicJevBench, DI0.2.1 per-benchmark and whole150317scored-request cohorts,1400item suite, labelled intent probes and image subsets. Temperatures fitted on private1400suite (suite ECE is in-sample);650suite questions overlap DI BANKING77/MMLUPro pools; prompts/rendering were selected partly on JevBench/DI evaluation. Taught probes use10labels/intent,150BANKING77/100CLINC150 testitems; three draws31B, six others. Compact prompt and four-question request forms remain separate. Copied board comparisons, resource axes and repeated summary values not recounted.
Metric: Accuracy fraction (0-1). Source: raw.githubusercontent.com. Saturation forecast: Estimated already saturated. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemma 4 12B (IT) | 0.83 | #688 |
| 2 | Gemma 4 31B (IT) | 0.83 | #343 |
| 3 | Qwen3.6-35B-A3B-FP8 | 0.79 | #552 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=decisio-frozen-readout-suite1400-scifact-accuracy-plain-layout · How It Works · Data refreshed daily, snapshot 2026-11-12.