Llama-3.3-nemotron-super-49B-v1.5 (Non-reasoning): benchmark results
Provider: NVIDIA. Released 2025-07-25. Access: Open.
Aggregate Rating 1426 ± 13, rank #1365 of 2201 rated models, from 79 benchmark results.
Strongest benchmark results
| Benchmark | Score | Metric | Percentile |
|---|---|---|---|
| NVIDIA Nemotron 49B v1.5 Reproduction - bfcl_v3 - irrelevance | 88.33 | Accuracy (%) | 100 |
| NVIDIA Nemotron 49B v1.5 Reproduction - bfcl_v3 - live_irrelevance | 85.6 | Accuracy (%) | 100 |
| NVIDIA Nemotron 49B v1.5 Reproduction - ruler - ruler.nemotron_128k.cwe | 1 | Accuracy (%) | 100 |
| NVIDIA Nemotron 49B v1.5 Reproduction - ruler - ruler.nemotron_128k.qa_1 | 47.8 | Accuracy (%) | 100 |
| NVIDIA Nemotron 49B v1.5 Reproduction - ruler - ruler.nemotron_128k.qa_2 | 42.6 | Accuracy (%) | 100 |
| NVIDIA Nemotron 49B v1.5 Reproduction - ruler - ruler.nemotron_128k.vt | 79.3 | Accuracy (%) | 100 |
| AA-Omniscience Hallucination Rate | 67.26 | Hallucination Rate (%) | 62.7 |
| AA Omniscience - Software Engineering (SWE) - HTML | 30 | Accuracy (%) | 51.7 |
| NVIDIA Nemotron 49B v1.5 Reproduction - ruler - ruler.nemotron_128k.niah_single_1 | 100 | Accuracy (%) | 50 |
| AA Omniscience - Software Engineering (SWE) - Go | 16 | Accuracy (%) | 48.6 |
| AA Omniscience | -46.18 | Score | 35.1 |
| AA MMLU-Pro | 69.16 | Accuracy (%) | 33.9 |
Interactive version: theaggregate.ai/model?slug=llama-3-3-nemotron-super-49b-v1-5-non-reasoning · How It Works · Data refreshed daily, snapshot 2026-11-12.