GBQA: leaderboard

GBQA (Game Benchmark for Quality Assurance, 2026) asks an LLM-driven agent to find the 124 human-verified bugs implanted in 30 generated games within 500 interaction steps per game, with access to design documents and source code (Quality Assurance Mode). Recall (%) of the ground-truth bugs, matched by a critic agent; higher is better.

Metric: Recall (%) in Quality Assurance Mode (the agent also reads the game's design documents and source code) on the 124 human-verified bugs implanted in GBQA's 30 games, found by the model driving the authors' ReAct QA agent (with in-session and cross-session memory) for at most 500 interaction steps per game; a GPT-5.2 critic agent matches each report to the ground-truth bugs; higher is better. Source: arxiv.org. Saturation forecast: Around May 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 4.6Thinking48.39#113
2Qwen 3.5 397B A17BThinking41.13#264
3DeepSeek R137.9#482
4Claude Sonnet 4.5Thinking37.1#261
5Qwen 3 235B A22BThinking35.48#628
6O334.68#237
7Qwen 3 32BThinking33.87#972
8Gemini-3-Pro [GBQA checkpoint unspecified]33.06
9Kimi K2.5Thinking28.23#266
10Qwen 3 8BThinking24.19#1873
11GPT-5.2Non-reasoning22.58#185
12DeepSeek V3.2Non-reasoning20.16#419
13Gemini 3 FlashNon-reasoning19.35#175
14Llama 3.1 70B Instruct14.52#1571
15Llama 3.1 8B Instruct8.87#4600

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=gbqa · How It Works · Data refreshed daily, snapshot 2026-11-21.