Trial by Trace - DeepSeek Harness - Agent Pass: leaderboard
Trial by Trace measures model task performance under a named native agent harness on 95 mechanism-focused tasks per model. Agent Pass requires all agent task criteria to pass; Agent Score is criterion-level performance. Harness-only scores, costs and mixed Agent/Harness composites are not admitted. arXiv v1; 95 tasks per model/harness; one selected execution including recovery replacements; DeepSeek-V4-Flash semantic judge with variable scoring budgets; native harness versions, model API snapshots and reasoning effort unreported. Metric: Agent Pass (%), higher is better.
Metric: Agent Pass (%). Source: tracelite.ai. Saturation forecast: Rough model projection: around 2026. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Claude Opus 5 (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 49.5 | |
| 2 | Gemini 3.7 Flash (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 44.2 | |
| 3 | GLM-5.3 (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 44.2 | |
| 4 | Grok 4.6 (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 43.2 | |
| 5 | Kimi K3 (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 43.2 | |
| 6 | DeepSeek V4 Pro (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 37.9 | |
| 7 | GPT-5.6 Sol (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 36.8 | |
| 8 | Qwen 3.8 Max (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 32.6 | |
| 9 | Hy3 (Benchmark Sweep Run: source=Trial-by-Trace; harness=DeepSeek Harness; snapshot-and-effort=unreported) | 28.4 |
Interactive version: theaggregate.ai/benchmark?slug=trial-by-trace-deepseek-harness-agent-pass · How It Works · Data refreshed daily, snapshot 2026-11-06.