ScarfBench (Kaggle): leaderboard
Coding agents (one model at one effort inside its harness, run by Kaggle) carry out 204 migrations of enterprise Java applications between Jakarta EE, Quarkus and Spring; the score is the share of the applications' behavioral smoke tests that pass after the migration. Unlike the ScarfBench paper board (174 focused-app migrations, one skills prompt), Kaggle runs and maintains these results over focused and whole applications.
Metric: Smoke tests passed (%) over 204 Java framework migrations, each model run in a coding-agent harness by Kaggle. Source: www.kaggle.com. Saturation forecast: Around May 2027. 11 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 4 Argon (High) | 75.24 | #11 (Gemini 4 Argon) |
| 2 | Claude Opus 5.5 (Max) | 70.55 | #5 (Claude Opus 5.5) |
| 3 | Claude Opus 5 (xHigh) | 69.79 | #22 (Claude Opus 5) |
| 4 | DeepSeek V4.1 Flash (Max) | 69.03 | #59 (DeepSeek V4.1 Flash) |
| 5 | GPT-6.1 Sol (Max) | 68.42 | #10 (GPT-6.1 Sol) |
| 6 | Gemini 3.8 Flash (Medium) | 65.97 | #36 (Gemini 3.8 Flash) |
| 7 | GPT-6 Astra (High) | 65.39 | #6 (GPT-6 Astra) |
| 8 | Qwen 3.8 Max (xHigh) | 63.67 | #55 (Qwen 3.8 Max) |
| 9 | Kimi K3 (Max) | 61.29 | #54 (Kimi K3) |
| 10 | Gemini 3.1 Pro (Preview) (High) | 37.6 | #94 (Gemini 3.1 Pro (Preview)) |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon.
Interactive version: theaggregate.ai/benchmark?slug=scarfbench-kaggle · How It Works · Data refreshed daily, snapshot 2026-11-13.