llmbenchmarks.io Local GPU - IFEval: leaderboard
IFEval (Google, 2023) on local hardware: 541 prompts carrying verifiable instructions such as word counts, keywords and output formats, checked with the official google/IFEval data and checker, so the number is comparable with published IFEval values that report the same rate. Score: prompt-level strict accuracy (%), the share of prompts whose every instruction is followed. llmbenchmarks.io runs this on its own hardware (2026): open-weight models, mostly 4-bit and 8-bit quantised checkpoints (NVFP4, AWQ, QAT) pinned to a Hugging Face repository and revision, served by vLLM 0.29.0 on one consumer machine (2x RTX 5060 Ti, 32 GB VRAM, tensor parallel 2), each with thinking on or off. Unlike vendor or leaderboard scores for the same test, a row measures the quantised deployment a home user would run, so a quantised checkpoint is its own model, not the full-precision original. Labels carry the mode as (Thinking) or (Non-thinking); Llama 3.1 rows (no thinking mode) and Ministral 3 14B Reasoning rows (always reasoning) stay bare. A run on a subset of the test is not read. Higher is better. Source: https://llmbenchmarks.io/local-gpu-benchmarks/
Metric: Prompt-level strict accuracy (%). Source: llmbenchmarks.io. Saturation forecast: Around February 2027. 24 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3.5 4B (Thinking) | 87.8 | #514 (Qwen 3.5 4B) |
| 2 | Gemma 4 E4B (IT) (Thinking) | 87.25 | |
| 3 | Gemma 4 E4B (IT) (Non-reasoning) | 85.4 | |
| 4 | Qwen 3.5 4B (Non-reasoning) | 83.55 | #514 (Qwen 3.5 4B) |
| 5 | Llama 3.1 8B Instruct | 75.79 | #1017 |
No result here: #3 Claude Opus 5.5, #6 GPT-6 Astra, #7 GPT-6.1 Sol.
Interactive version: theaggregate.ai/benchmark?slug=llmbenchmarks-io-local-gpu-ifeval · How It Works · Data refreshed daily, snapshot 2026-11-05.