AgentJudgeBench (Hard, With Ground Truth) - Llama-3.3-70B-Instruct Outputs: leaderboard
AgentJudgeBench (ServiceNow, 2026) tests LLMs as judges of agentic tool calling: agreement of the judge verdicts with a programmatic reference on the hardest rewrite of 3,808 workflow-DAG records, judging tool calls produced by Llama-3.3-70B-Instruct; the judge also sees the ground-truth tool calls. Percent; higher is better.
Metric: Judge alignment (%; agreement of the judge's per-metric verdicts (tool selection, parameter structure, call sequence, query coverage) with the programmatic reference on the hard (most ambiguous) rewrite of 3,808 BFCL-style tool-calling records over six workflow-DAG topologies; judged outputs from Llama-3.3-70B-Instruct at temperature 0; the judge sees the ground-truth tool calls). Source: arxiv.org. Saturation forecast: Estimated already saturated. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | QwQ-32B | 87.5 | #910 |
| 2 | GPT-OSS-120B | 87.2 | #700 |
| 3 | GPT-OSS-20B | 83.9 | #1147 |
| 4 | Claude Sonnet 4.5 | 83.7 | #250 |
| 5 | GPT-5.4 | 83 | #123 |
| 6 | Gemini 2.5 Pro (Preview 05-06) | 82.2 | #275 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=agentjudgebench-hard-with-ground-truth-llama-3-3-70b-instruct-outputs · How It Works · Data refreshed daily, snapshot 2026-11-13.