Kimi K3.1 Pro Tech Blog - xdev-bench-hard: leaderboard

Moonshot AI's in-house cyber-offense benchmark of expert-curated, source-code-based vulnerability discovery and exploitation tasks, most reproducing recently disclosed vulnerabilities in software such as the Linux kernel, Node.js, JavaScriptCore, Exim and VirGL with mitigations enabled. Given the vulnerable source, a short bug hint and the binary, the agent must achieve arbitrary code execution in a separate sandbox and retrieve a secret string. pass@3 in percent over a random subset of runs, with no hard timeouts or output token limits; higher is better. Kimi K3.1 Pro launch post benchmark sheet (October 2026); Kimi K3.1 Pro ran in KimiCode at max reasoning effort, GLM-5.3 and DeepSeek-V4.1-Flash-1017 in Claude Code.

Metric: pass@3 (%). Source: www.kimi.com.

Interactive version: theaggregate.ai/benchmark?slug=kimi-k3-1-pro-tech-blog-xdev-bench-hard · How It Works · Data refreshed daily, snapshot 2026-10-23.