Llama 3.1 Nemotron Ultra 253B v1: benchmark results
NVIDIA's Llama-3.1-Nemotron-Ultra-253B, a reasoning-tuned, NAS-pruned refinement of Llama 3.1 405B (April 2025). Provider: NVIDIA. Released 2025-04-07. Access: Open.
Aggregate Rating 1600 ± 19, rank #441 of 1656 rated models, from 106 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| Absurd World - Few-Shot - REAL | 100 | Determine-Outcome Accuracy (%) | 100 |
| Absurd World - Few-Shot - CAR | 99 | Determine-Outcome Accuracy (%) | 96.2 |
| Absurd World - Few-Shot - ICE CREAM | 99 | Determine-Outcome Accuracy (%) | 96.2 |
| Absurd World - Few-Shot - MISS & SWITCH | 95 | Determine-Outcome Accuracy (%) | 92.3 |
| Absurd World - Zero-Shot - MISSING | 100 | Determine-Outcome Accuracy (%) | 88.5 |
| Absurd World - Zero-Shot - MISS & SWITCH | 100 | Determine-Outcome Accuracy (%) | 84.6 |
| Absurd World - Zero-Shot - ICE CREAM | 100 | Determine-Outcome Accuracy (%) | 80.8 |
| Absurd World - Zero-Shot - LEAST | 100 | Determine-Outcome Accuracy (%) | 80.8 |
| Absurd World - Zero-Shot - SWITCH | 100 | Determine-Outcome Accuracy (%) | 80.8 |
| AA MMLU-Pro | 82.46 | Accuracy (%) | 80.7 |
| AA Omniscience - Software Engineering (SWE) - HTML | 42 | Accuracy (%) | 78.3 |
| BenchTable - STEM | 70.2 | Weighted Score (%) | 77.9 |
Interactive version: theaggregate.ai/model?slug=llama-3-1-nemotron-ultra-253b-v1 · How It Works · Data refreshed daily, snapshot 2026-10-20.