OPTIC Uncompressed - PIQA: leaderboard
OPTIC Uncompressed - PIQA. The OPTIC paper evaluates unchanged released Gemma 4 E2B and E4B IT checkpoints with uncompressed per-layer embeddings. BF16 revisions are b446025c61ec and ee0ef6023621; official W4A16 revisions are 971342c08f60 and 6cd26aaa2357. W4A16 fitting checkpoints are not evaluated-model identities. Tasks are PIQA validation (1,838), eight-shot GSM8K test (1,319), GPQA Diamond (198 questions with four fixed option permutations, 792 attempts), AIME 2026 (30 problems with four generations, 120 attempts), IFBench (300), InfographicVQA validation (2,801), and MMLU-Redux v1 (3,000). AIME is the micro-average of all attempts, not pass@4. Evaluation seeds are 0, except IFBench and InfographicVQA use 42 and AIME uses 0–3. The paper uses identical prompts, generation settings, and scorers within each comparison. Compression ratios and retention statistics are not task accuracy. Reported metric: Accuracy (%).
Metric: Accuracy (%). Source: arxiv.org. Saturation forecast: Around January 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | gemma-4-E4B-it | 75.9 | #573 |
| 2 | Gemma 4 E2B (IT) | 72.4 | #787 |
No result here: #5 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=optic-uncompressed-piqa · How It Works · Data refreshed daily, snapshot 2026-10-25.