Open SLM Leaderboard - ARC-Challenge: leaderboard
The Open SLM Leaderboard (Axiomic Labs, Hugging Face Space) ranks open-weight language models under 400M parameters that their submitters pretrained from scratch, with zero-shot scores from LM-eval harness and Axiomic Labs' ArithMark-3 script that the leaderboard team checks before merging. Unlike the Hugging Face Open LLM Leaderboard, which evaluated every submitted model on its own cluster, it lists zero-shot results the submitters ran and admits only small from-scratch checkpoints (no merges). The page does not state the harness version, the accuracy variant (acc or acc_norm) or the evaluated splits. Rows are bound to their Hugging Face model card; a row without one (Mystery-5M-Preview) is withheld, and a row missing a score is absent from that board only. ARC-Challenge, the harder ARC science questions, zero-shot accuracy (%, chance 25).
Metric: Zero-shot accuracy (%, chance 25). Source: huggingface.co. Saturation forecast: Rough model projection: around 2026. 263 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | LFM-2.5-350M-Base (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=LiquidAI/LFM2.5-350M-Base) | 41.13 | |
| 2 | SmolLM2-360M (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=HuggingFaceTB/SmolLM2-360M) | 38.31 | |
| 3 | LFM-2.5-230M-Base (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=LiquidAI/LFM2.5-230M-Base) | 34.98 | |
| 4 | MobileLLM-R1-360M-base (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=facebook/MobileLLM-R1-360M-base) | 34.56 | |
| 5 | Inferno-Exp-1.7 (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=AlgoDriveAI/Inferno-Exp-1.7) | 34.04 | |
| 6 | KLM-1.6 (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=MinimaLabs/KLM-1.6) | 33.53 | |
| 7 | TinyBrainBot-350M-V3-Base (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=nkthebass/tinybrainbot-350mV3-base) | 33.28 | |
| 8 | KLM-1.5 (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=MinimaLabs/KLM-1.5) | 32.68 | |
| 9 | TinyBrainBot-360M-A75M-v4.1-MoE-Base (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=nkthebass/tinybrainbot-360m-a75m-v4.1-moe-base) | 32.34 | |
| 10 | BananaMind-2.5-Pro (Benchmark Sweep Run: source=Open SLM Leaderboard; zero-shot; card=BananaMind/BananaMind-2.5-Pro) | 31.96 |
Interactive version: theaggregate.ai/benchmark?slug=open-slm-leaderboard-arc-challenge · How It Works · Data refreshed daily, snapshot 2026-11-12.