MADE: leaderboard

Multi-label classification of medical device adverse event reports on MADE (2026), a living benchmark built from FDA reports with 1,154 hierarchical product and patient problem labels and a strict temporal test split (July 2024 to June 2025). Models are prompted with ten similar training reports retrieved by kNN. Macro-F1 over all labels in percent; higher is better.

Metric: Macro-F1 (%, the 0-1 score times 100) over all 1,154 labels with ten-shot prompting with kNN-retrieved training reports, the task instructions and the label list, on the 10,288-report stratified (truncated) MADE test set of FDA medical device adverse event reports from July 2024 to June 2025 with 1,154 hierarchical IMDRF product and patient problem labels, greedy decoding (GPT-5 at its default medium reasoning effort); higher is better. Source: arxiv.org. Saturation forecast: Around February 2027. 15 models tracked. Updated 7 Oct 2026.

Top models

#ModelSettingScoreOverall rank
1GPT-554#178
2Qwen 3 235B A22BThinking49#628
3DeepSeek R1 052848#428
4Qwen 3 30B A3BThinking45#1156
5Qwen 3 235B A22B Instruct44#691
6GPT-4.143#505
7GLM 4.5 Air42#886
8Llama-3.3-nemotron-super-49B-v1.542#920
9GPT-OSS-120B40#704
10Qwen 3 4BReasoning38#2348
11Llama 3.1 70B Instruct30#1571
12Qwen 3 4B Instruct29#1696
13Qwen 3 30B A3B Instruct22#1053
14Kimi K29#443
15Llama 3.1 8B Instruct8#4600

No result here: #1 Claude Fable 5.2, #3 Gemini 4 Neon, #4 GPT-6.2 Astra.

Interactive version: theaggregate.ai/benchmark?slug=made · How It Works · Data refreshed daily, snapshot 2026-11-21.