Local Agent Bench - Apple M2 - JSON Correctness: leaderboard
Local Agent Bench Apple M2 population: JSON Correctness on a public deterministic 100-task tool-use suite. Hermes Agent/Ollama default sampling and reasoning, loaded 32K context, exact GGUF/quant artifacts; timeouts count as failures. Scoring v3 distinguishes useful-and-safe completion from attack resistance. Memory display tiers are not separate scored experiments.
Metric: JSON Correctness (%). Source: localagentbench.com. Saturation forecast: Rough model projection: around 2026. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Ollama qwen2.5-coder:7b (Local Agent Bench Run: quant=Q4_K_M;gguf=60e05f2100071479f596b964f89f510f057ce397ea22f2833a0cfe029bfc2463;engine=ollama0.40.6;reasoning=default;trials=3) | 100 | |
| 2 | Ollama qwen3:8b (Local Agent Bench Run: quant=Q4_K_M;gguf=a3de86cd1c132c822487ededd47a324c50491393e6565cd14bafa40d0b8e686f;engine=ollama0.40.6;reasoning=default;trials=3) | 93.33 | |
| 3 | Ollama llama3.2:3b (Local Agent Bench Run: quant=Q4_K_M;gguf=dde5aa3fc5ffc17176b5e8bdc82f587b24b2678c6c66101bf7da77af9f7ccdff;engine=ollama0.40.6;reasoning=default;trials=3) | 93.33 | |
| 4 | Ollama qwen2.5:14b (Local Agent Bench Run: quant=Q4_K_M;gguf=2049f5674b1e92b4464e5729975c9689fcfbf0b0e4443ccf10b5339f370f9a54;engine=ollama0.40.6;reasoning=default;trials=3) | 88 | |
| 5 | Ollama hermes3:8b (Local Agent Bench Run: quant=Q4_K_M;gguf=c8985d236593f7a17da2a3da49588aa951a9b1e57ce97753364fbf59e63af84a;engine=ollama0.40.6;reasoning=default;trials=3) | 82.67 | |
| 6 | Ollama mistral-nemo:12b (Local Agent Bench Run: quant=Q4_K_M;gguf=b559938ab7a0392fc9ea9675b82280f2a15669ec3e0e0fc491c9cb0a7681cf94;engine=ollama0.40.6;reasoning=default;trials=3) | 80 | |
| 7 | Ollama llama3.1:8b (Local Agent Bench Run: quant=Q4_K_M;gguf=667b0c1932bc6ffc593ed1d03f895bf2dc8dc6df21db3042284a6f4416b06a29;engine=ollama0.40.6;reasoning=default;trials=3) | 78.67 | |
| 8 | Ollama qwen2.5:7b (Local Agent Bench Run: quant=Q4_K_M;gguf=2bada8a7450677000f678be90653b85d364de7db25eb5ea54136ada5f3933730;engine=ollama0.40.6;reasoning=default;trials=3) | 25.33 |
Interactive version: theaggregate.ai/benchmark?slug=local-agent-bench-apple-m2-json-correctness · How It Works · Data refreshed daily, snapshot 2026-11-05.