Evolve on Graph Five-Task Average - Hit@1: leaderboard
Evolve on Graph v1 evaluates knowledge-graph reasoning using direct answers, chain of thought, self-consistency, ReAct, retrieval, or separately trained Solver/Proposer systems. It uses Qwen2.5-3B/7B-Instruct and Llama3.1-8B-Instruct. Training lasts 140 steps, with 5 rollouts per task and at most 15 assistant turns. Hit@1 requires an exact normalized match to any gold answer; F1 measures overlap between predicted and gold answer lists. Freebase, Wikidata, and task-specific KG-R1 training scopes remain distinct. Trained weights are not aliased to the instruction backbone. Main Avg for supervised rows pools only 3 or 2 tasks, making it incomparable to the 5-task average. These remain separate task-support populations. Metric: Hit@1 (%).
Metric: Hit@1 (%). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 20 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | EoG study Qwen-2.5-7B-Instruct EoG | 61.55 | |
| 2 | EoG study LLaMA-3.1-8B-Instruct EoG | 58.59 | |
| 3 | EoG study Qwen-2.5-7B-Instruct Graph Reranker | 54.09 | |
| 4 | EoG study Qwen-2.5-7B-Instruct ToG | 51.53 | |
| 5 | EoG study Qwen-2.5-7B-Instruct Text-Based Retrieval | 50.01 | |
| 6 | EoG study Qwen-2.5-7B-Instruct CoR | 47.64 | |
| 7 | EoG study Qwen-2.5-3B-Instruct EoG | 46.02 | |
| 8 | EoG study Qwen-2.5-7B-Instruct PoG | 45.54 | |
| 9 | EoG study LLaMA-3.1-8B-Instruct SC | 44.15 | |
| 10 | EoG study Qwen-2.5-7B-Instruct SC | 38.15 |
Interactive version: theaggregate.ai/benchmark?slug=evolve-on-graph-five-task-average-hit-1 · How It Works · Data refreshed daily, snapshot 2026-10-28.