AgentKernelArena - HIP-to-HIP: leaderboard
GPU kernel optimization by coding agents on AMD Instinct MI300X from AgentKernelArena (Younesian et al., AMD, 2026): agents optimize 24 existing HIP kernels (activations, attention, normalization, matrix and loss operators) against the reference HIP implementation, working in an isolated workspace with compile, correctness and performance gates. Mean per-task speedup (a ratio, unbounded above), with failed compilation or correctness counted as 0, averaged over three runs; higher is better.
Metric: Mean speedup (x; arithmetic mean over the 24 HIP-to-HIP tasks of the per-task speedup of the optimized kernel over the reference HIP kernel, counting 0 for a task whose kernel fails compilation or correctness; per-task mean of 3 runs; agent in an isolated workspace, 3600 s timeout, up to 3 kernel versions; AMD Instinct MI300X). Source: arxiv.org. Saturation forecast: Around August 2027. 8 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 4.6 | 6.69 | #100 |
| 2 | Claude Opus 4.7 (High) | 5.82 | #75 (Claude Opus 4.7) |
| 3 | Claude Sonnet 4.6 | 5.37 | #158 |
| 4 | GPT-5.4 (High) | 5.19 | #121 (GPT-5.4) |
| 5 | Claude Opus 4.6 (High) | 4.62 | #100 (Claude Opus 4.6) |
| 6 | GPT-5.3 Codex (High) | 3.65 | #114 (GPT-5.3 Codex) |
| 7 | GPT-5.3 Codex | 3.61 | #114 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=agentkernelarena-hip-to-hip · How It Works · Data refreshed daily, snapshot 2026-11-12.