MADE - Medium Labels: leaderboard
Metric: Macro-F1 (%, the 0-1 score times 100) over the 481 medium labels (0.1 to 1 percent of training reports) with ten-shot prompting with kNN-retrieved training reports, the task instructions and the label list, on the 10,288-report stratified (truncated) MADE test set of FDA medical device adverse event reports from July 2024 to June 2025 with 1,154 hierarchical IMDRF product and patient problem labels, greedy decoding (GPT-5 at its default medium reasoning effort); higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 15 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | GPT-5 | 58 |
| 2 | Qwen 3 235B A22B (Thinking) | 52 |
| 3 | DeepSeek R1 0528 | 51 |
| 4 | Qwen 3 30B A3B (Thinking) | 49 |
| 5 | Qwen 3 235B A22B Instruct | 48 |
| 6 | GPT-4.1 | 47 |
| 7 | GLM 4.5 Air | 46 |
| 8 | GPT-OSS-120B | 45 |
| 9 | Qwen 3 4B (Reasoning) | 42 |
| 10 | Llama 3.1 70B Instruct | 35 |
| 11 | Qwen 3 4B Instruct | 35 |
| 12 | Qwen 3 30B A3B Instruct | 27 |
| 13 | Kimi K2 | 11 |
| 14 | Llama 3.1 8B Instruct | 9 |
Interactive version: theaggregate.ai/benchmark?slug=made-medium-labels · How It Works · Data refreshed daily, snapshot 2026-10-07.