AARRI-Bench (Hermes Agent) - Mindset: leaderboard
AARRI-Bench (Act As a Real Research Intern, 2026) is a set of 82 hand-crafted AI-research tasks graded by unit tests, testing whether an agent harness and model behave like a careful human researcher. This board is the percentage of the mindset tasks, which reward pushing back on flawed instructions and recognizing dead ends solved in full with the Hermes Agent harness; higher is better.
Metric: Task success rate (%) under the 0/1 reward on the mindset tasks: academic self-awareness and autonomy (disagreeing with flawed instructions, recognizing dead ends); Harbor containerized runs, Hermes Agent agent harness; a configuration's tasks without a valid run are left out of its denominator; higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 6 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.7 | 76.9 | #79 |
| 2 | Qwen 3.6 Plus | 69.2 | #222 |
| 3 | MiniMax-M2.7 | 69.2 | #518 |
| 4 | Qwen 3 235B A22B 2507 (Thinking) | 69.2 | #463 |
| 5 | Claude Sonnet 4.6 | 53.8 | #161 |
| 6 | DeepSeek V4 Flash | 46.2 | #258 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=aarri-bench-hermes-agent-mindset · How It Works · Data refreshed daily, snapshot 2026-11-13.