EdgeKernelBench - Correct@1: leaderboard
EdgeKernelBench (Zhejiang University and Westlake University, arXiv 2610.24648) asks a model to write, in one attempt, a TVM tensor program for a PyTorch reference task that must run on real edge hardware: 331 canonical tasks, from single operators through operator compositions to compact model graphs, instantiated with device-specific shapes as 650 instances (204 on a Snapdragon 8 Gen 3 CPU via LLVM, 218 on a Jetson AGX Orin GPU via CUDA, 228 on an RK3588 Mali-G610 GPU via OpenCL). Unlike KernelBench, which asks for CUDA kernels on server NVIDIA GPUs, it spans heterogeneous edge backends through one TVM-based interface and measures speed against a budget-tuned TVM MetaSchedule baseline on the same device. A program that fails to compile or fails a check counts as a miss; scores pool all 650 instances, with counts recovered from the published per-backend and per-level rates. Models run once with their providers' default inference settings. Correct@1: the program compiles and matches the PyTorch reference on inputs from three random seeds within the task's tolerance (%).
Metric: Instances whose one-shot program compiles and passes every output check (%, 650 instances). Source: pillor9.github.io. Saturation forecast: Rough model projection: around 2026. 5 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Sol (Benchmark Sweep Run: source=EdgeKernelBench; one-shot generation; provider default inference settings; TVM 0.23.0) | 64.46 | |
| 2 | DeepSeek V4 Pro (0813) (Benchmark Sweep Run: source=EdgeKernelBench; one-shot generation; provider default inference settings; TVM 0.23.0) | 57.85 | |
| 3 | Gemini 3.7 Flash (Benchmark Sweep Run: source=EdgeKernelBench; one-shot generation; provider default inference settings; TVM 0.23.0) | 56.92 | |
| 4 | GLM-5.3 (Benchmark Sweep Run: source=EdgeKernelBench; one-shot generation; provider default inference settings; TVM 0.23.0) | 33.38 | |
| 5 | Claude Sonnet 5 (Benchmark Sweep Run: source=EdgeKernelBench; one-shot generation; provider default inference settings; TVM 0.23.0) | 32.77 |
Interactive version: theaggregate.ai/benchmark?slug=edgekernelbench-correct-1 · How It Works · Data refreshed daily, snapshot 2026-11-12.