Model vs Harness - AgentSkillOS 200 Skills: leaderboard
Model vs Harness v1, four matched repetitions, all scheduled attempts included even if runtime fails; official task correctness differs from valid-completion rate. Harnesses: DeepAgents Code 0.1.3, OpenCode 1.15.6, Pi CLI 0.79.3, Codex CLI (unreported version). DeepSeek V4 Flash Preview official API; MiniMax M3, Kimi K2.7 Code, DeepSeek V4 Pro 0813 local SGLang on eight H200 GPUs. Qwen3.6-35B-A3B explicit HF checkpoint. Fixed GPT-5.5 xhigh native Codex subscription and DeepSeek V4 Pro 0813 native Harness SDK 0.1.0rc6 are complete-system references. No unreported reasoning effort inferred.30 tasks, five artifact categories; mean official Task Score over 4 runs; 3,600 seconds/task, 900-second idle guard for streaming; 900-skill stress includes 19 required and 881 distractors. 200 Skills. Metric: Task Score.
Metric: Task Score. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 13 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Model vs Harness GPT-5.5 (xhigh) (Harness: Codex CLI native) | 97.57 | |
| 2 | Model vs Harness Kimi-K2.7-Code (Harness: Pi CLI 0.79.3) | 95.91 | |
| 3 | Model vs Harness MiniMax-M3 (Harness: OpenCode 1.15.6) | 93.49 | |
| 4 | Model vs Harness Kimi-K2.7-Code (Harness: DeepAgents 0.1.3) | 89.92 | |
| 5 | Model vs Harness MiniMax-M3 (Harness: Pi CLI 0.79.3) | 85.39 | |
| 6 | Model vs Harness Kimi-K2.7-Code (Harness: OpenCode 1.15.6) | 84.94 | |
| 7 | Model vs Harness MiniMax-M3 (Harness: DeepAgents 0.1.3) | 80.07 | |
| 8 | Model vs Harness Qwen3.6-35B-A3B (Harness: OpenCode 1.15.6) | 76.12 | |
| 9 | Model vs Harness Qwen3.6-35B-A3B (Harness: DeepAgents 0.1.3) | 74.01 | |
| 10 | Model vs Harness Qwen3.6-35B-A3B (Harness: Codex CLI unspecified version) | 63.06 |
Interactive version: theaggregate.ai/benchmark?slug=model-vs-harness-agentskillos-200-skills · How It Works · Data refreshed daily, snapshot 2026-10-27.