GBQA (Player Exploring Mode): leaderboard

GBQA (2026) bug discovery in Player Exploring Mode: the LLM-driven agent sees only what a player sees while hunting the 124 human-verified bugs in 30 games within 500 steps per game. Recall (%) of the ground-truth bugs; higher is better.

Metric: Recall (%) in Player Exploring Mode (the agent sees only what a player sees) on the 124 human-verified bugs implanted in GBQA's 30 games, found by the model driving the authors' ReAct QA agent (with in-session and cross-session memory) for at most 500 interaction steps per game; a GPT-5.2 critic agent matches each report to the ground-truth bugs; higher is better. Source: arxiv.org. Saturation forecast: Around December 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1Claude Opus 4.6Thinking35.48#113
2Qwen 3.5 397B A17BThinking30.65#264
3DeepSeek R127.42#482
4Claude Sonnet 4.5Thinking26.61#261
5O325#237
6Qwen 3 235B A22BThinking25#628
7Qwen 3 32BThinking24.19#972
8Gemini-3-Pro [GBQA checkpoint unspecified]23.39
9Kimi K2.5Thinking20.16#266
10Qwen 3 8BThinking16.13#1873
11GPT-5.2Non-reasoning14.52#185
12DeepSeek V3.2Non-reasoning12.9#419
13Gemini 3 FlashNon-reasoning12.1#175
14Llama 3.1 70B Instruct9.68#1571
15Llama 3.1 8B Instruct5.65#4600

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=gbqa-player-exploring-mode · How It Works · Data refreshed daily, snapshot 2026-11-21.