Herculean - Auditing (Codex): leaderboard
XBRL auditing workflow of Herculean (Peng et al., 2026): given a raw SEC filing as six interconnected XBRL documents and the US-GAAP taxonomy, the agent locates the reported value of a target concept and recomputes its correct value from the filing calculation network, over 65 audit instances. Accuracy in percent graded by the FinAuditing hierarchical LLM judge; higher is better, with each backbone running inside Codex.
Metric: Accuracy (%; share of the 65 SEC XBRL audit instances where the agent both extracts the reported value of the target concept and computes its correct value from the filing's calculation network, graded by the FinAuditing hierarchical LLM judge; Codex agent with this backbone, tools through the Herculean auditing MCP server). Source: arxiv.org. Saturation forecast: Around February 2027. 4 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Sonnet 4.6 | 63.08 | #158 |
| 2 | GPT-5.4 | 63.08 | #121 |
| 3 | Qwen 3.5 397B A17B | 49.23 | #242 |
| 4 | Qwen 3.5 27B | 27.69 | #358 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=herculean-auditing-codex · How It Works · Data refreshed daily, snapshot 2026-11-12.