MADE - Head Labels: leaderboard

Frequent-label classification on MADE (2026), a living benchmark of FDA medical device adverse event reports: macro-F1 over the 144 head labels, each attached to more than 1 percent of training reports, with models prompted with ten kNN-retrieved training reports. Percent; higher is better.

Metric: Macro-F1 (%, the 0-1 score times 100) over the 144 head labels (more than 1 percent of training reports) with ten-shot prompting with kNN-retrieved training reports, the task instructions and the label list, on the 10,288-report stratified (truncated) MADE test set of FDA medical device adverse event reports from July 2024 to June 2025 with 1,154 hierarchical IMDRF product and patient problem labels, greedy decoding (GPT-5 at its default medium reasoning effort); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-568#178
2DeepSeek R1 052862#428
3Qwen 3 235B A22BThinking62#628
4Qwen 3 235B A22B Instruct60#691
5GPT-4.159#505
6Qwen 3 30B A3BThinking58#1156
7GPT-OSS-120B57#704
8Llama-3.3-nemotron-super-49B-v1.557#920
9GLM 4.5 Air56#886
10Qwen 3 4BReasoning53#2348
11Llama 3.1 70B Instruct50#1571
12Qwen 3 4B Instruct49#1696
13Qwen 3 30B A3B Instruct48#1053
14Llama 3.1 8B Instruct28#4600
15Kimi K218#443

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=made-head-labels · How It Works · Data refreshed daily, snapshot 2026-11-21.