MetaGAI - Data Card Quality: leaderboard

Writing dataset documentation from research papers on MetaGAI (2026): the model reads a paper and fills a structured data card, which three LLM judges rate from 1 to 5 on faithfulness, relevance, accuracy, consistency and usefulness on a 250-entry stratified sample. Mean Likert score; higher is better.

Metric: Qualitative average (1-5): mean judge Likert score of data cards generated zero-shot from the paper text alone, rated on faithfulness, relevance, accuracy, consistency and usefulness by GPT-OSS-120B, Llama-3.3-70B-Instruct and Qwen3-235B-A22B-2507 judges, on a stratified sample of 250 MetaGAI entries (2,541 verified paper, repository and Hugging Face triplets); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 10 models tracked. Updated 7 Oct 2026.

Top models

#ModelScoreOverall rank
1Qwen 3 30B A3B Instruct4.3#1053
2GPT-5 Mini4.06#364
3Gemini 2.5 Flash Lite3.83#964
4Gemini 2.5 Flash3.76#463
5Gemma 3 27B (IT)3.72#1324
6nemotron-3-nano-30B-a3B3.37#1946
7Qwen 3 32B3.26#972
8Mistral Small 3.23.26#1641
9GPT-5 Nano3.06#906
10GPT-OSS-20B3.06#1183

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=metagai-data-card-quality · How It Works · Data refreshed daily, snapshot 2026-11-21.