CaptchaBench - Rotation Match: leaderboard
CaptchaBench task (CaptchaMind paper, Wang et al., EMNLP 2026) in which the agent pages through candidate images with arrow buttons to find the one where an object faces the same direction as in a reference image. Success rate in percent over 200 generated test episodes, with one ReAct tool-call prompt for every model; higher is better.
Metric: Task success rate (%; 200 test episodes). Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 3 Pro | 50 | #135 |
| 2 | Claude Sonnet 4 | 20 | #397 |
| 3 | Claude 3.7 Sonnet | 10 | #455 |
| 4 | Qwen-VL-Max | 5.5 | #1226 |
| 5 | O3 | 4 | #224 |
| 6 | GPT-5 | 3.5 | #163 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=captchabench-rotation-match · How It Works · Data refreshed daily, snapshot 2026-11-12.