KGCE (With Knowledge Base) - Key-Step F1: leaderboard
Multimodal LLM agents operating 104 education tasks on Windows, Android or both, many in school-specific software such as the Xiaoya assistant (KGCE, Liu et al., 2026). F1 of action precision (completed over attempted actions) and recall of the key steps, averaged over task runs, in percent; higher is better. A structured JSON knowledge base describing the private software is retrieved into the prompt.
Metric: F1 of action precision (completed over attempted actions) and key-step recall (%; 104 educational GUI tasks on Windows, Android and cross-platform workflows with school-specific software, decomposed into DAG sub-goals; the multimodal model drives the agent; with the structured JSON knowledge base of the school software injected into the prompt). Source: arxiv.org. Saturation forecast: Around February 2027. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-4o | 47.71 | #741 |
| 2 | Gemini 2.0 Flash | 38.72 | #702 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=kgce-with-knowledge-base-key-step-f1 · How It Works · Data refreshed daily, snapshot 2026-11-12.