PASE Skill Evaluation - WikiTQ: leaderboard
PASE Skill Evaluation - WikiTQ: Native benchmark success (%). PASE arXiv 2611.14019v1 Table 1: six skill configurations on Qwen3.6-27B, Qwen3.7-Plus and Gemma4-31B; all remain native ReAct evaluation runs. Same generation settings: temperature 0.6, context 131072, at most 30 inference ReAct iterations, thinking mode not explicitly configured. Evolution at most 50 iterations, 10 steps/skill, batch 8. Source Vrf, Soft, Hard, domain-Spreadsheet and TableQA metrics are distinct; no best-score pooling or new model variants.
Metric: Native benchmark success (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 18 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=EvoSkill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 86.54 | |
| 2 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=PASE; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 86.14 | |
| 3 | Qwen3.6-27B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=PASE; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 85.39 | |
| 4 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=Human Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 85 | |
| 5 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=Trace2Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 84.8 | |
| 6 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=LLM Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 84.64 | |
| 7 | Qwen3.6-27B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=EvoSkill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 84.6 | |
| 8 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=No Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 84.52 | |
| 9 | Gemma 4 31B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=LLM Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 84.36 | |
| 10 | Gemma 4 31B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=Trace2Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 83.53 |
Interactive version: theaggregate.ai/benchmark?slug=pase-skill-evaluation-wikitq · How It Works · Data refreshed daily, snapshot 2026-11-12.