PatentAgentBench - Hermes - Claim Charting: leaderboard
PatentAgentBench Claim Charting evaluates patent evidence search, comparison, drafting, responses and translation under Hermes Agent 0.21.1. Its primary Macro CP-F1 (%) uses the full assigned case denominator, including final failures after up to three execution-failure retries. Harnesses and unreported model effort remain distinct source configurations. The suite has 510 patent cases; its public banks contain 1,311 records because translation's 80 cases contain 881 segment items.
Metric: Macro CP-F1 (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 7 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5 (PatentAgentBench Run: 2610.35031v1; Claude Opus 5; Hermes Agent 0.21.1; effort unreported) | 55.1 | |
| 2 | GPT-5.6 Sol (PatentAgentBench Run: 2610.35031v1; GPT-5.6-Sol; Hermes Agent 0.21.1; effort unreported) | 12.7 | |
| 3 | MiniMax-M3 (PatentAgentBench Run: 2610.35031v1; MiniMax-M3; Hermes Agent 0.21.1; effort unreported) | 8.4 | |
| 4 | DeepSeek V4 Pro (PatentAgentBench Run: 2610.35031v1; DeepSeek-V4-Pro; Hermes Agent 0.21.1; effort unreported) | 8.3 | |
| 5 | Kimi K3 (PatentAgentBench Run: 2610.35031v1; Kimi K3; Hermes Agent 0.21.1; effort unreported) | 3.2 | |
| 6 | Qwen 3.8 Max (PatentAgentBench Run: 2610.35031v1; Qwen3.8-Max; Hermes Agent 0.21.1; effort unreported) | 0 | |
| 7 | GLM-5.3 (PatentAgentBench Run: 2610.35031v1; GLM-5.3; Hermes Agent 0.21.1; effort unreported) | 0 |
Interactive version: theaggregate.ai/benchmark?slug=patentagentbench-hermes-claim-charting · How It Works · Data refreshed daily, snapshot 2026-11-05.