PASE Skill Evaluation - HiTab: leaderboard
PASE Skill Evaluation - HiTab: Native benchmark success (%). PASE arXiv 2611.14019v1 Table 1: six skill configurations on Qwen3.6-27B, Qwen3.7-Plus and Gemma4-31B; all remain native ReAct evaluation runs. Same generation settings: temperature 0.6, context 131072, at most 30 inference ReAct iterations, thinking mode not explicitly configured. Evolution at most 50 iterations, 10 steps/skill, batch 8. Source Vrf, Soft, Hard, domain-Spreadsheet and TableQA metrics are distinct; no best-score pooling or new model variants.
Metric: Native benchmark success (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 18 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=PASE; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 76.21 | |
| 2 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=EvoSkill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 74.29 | |
| 3 | Qwen3.6-27B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=PASE; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 74.1 | |
| 4 | Gemma 4 31B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=PASE; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 73.3 | |
| 5 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=Trace2Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 72.24 | |
| 6 | Qwen3.6-27B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=EvoSkill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 72.18 | |
| 7 | Qwen3.6-27B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=Trace2Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 69.64 | |
| 8 | Gemma 4 31B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=EvoSkill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 68.96 | |
| 9 | Qwen3.7-Plus (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=LLM Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 68.03 | |
| 10 | Qwen3.6-27B (Benchmark Sweep Run: source=pase_paper; harness=ReAct; skill=LLM Skill; temperature=0.6; context=131072; iterations=30; thinking=not explicitly configured) | 67.6 |
Interactive version: theaggregate.ai/benchmark?slug=pase-skill-evaluation-hitab · How It Works · Data refreshed daily, snapshot 2026-11-12.