GAIA-v2-LILT - Arabic: leaderboard

GAIA-v2-LILT (2026) is a human-audited Arabic version of the GAIA agent validation tasks, with machine-translated queries and answers corrected for functional and cultural alignment, here solved by the Open Deep Research agent. Pass@1 exact-match accuracy in percent; higher is better.

Metric: Pass@1 accuracy (%) on the 165 GAIA validation tasks in Arabic after the GAIA-v2-LILT human audit (functional and cultural alignment of each machine-translated query and answer), solved by the smolagents Open Deep Research agent with web search (Exa), speech recognition, image captioning and file-reading tools, at most 12 manager steps and 20 search-subagent steps; exact match after removing spaces, lowercasing and stripping punctuation; higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Open Deep Research + Gemini 3.1 Pro52.1
2Open Deep Research + Claude Opus 4.649.1
3Open Deep Research + GPT-5.447.3

Interactive version: theaggregate.ai/benchmark?slug=gaia-v2-lilt-arabic · How It Works · Data refreshed daily, snapshot 2026-11-21.