EHRStruct (Synthea) - Filtering (Less Than): leaderboard
EHRStruct (Yang, Zhao and Shen, 2025) tests language models on structured electronic health record tables. This board is its D-U2 task on synthetic Synthea records: 100 zero-shot questions, each asking the model to list the IDs of the patients meeting a condition on a categorical field and a less-than condition on a numeric field from a table given as plain text. Exact-answer accuracy in percent, higher is better; models that produced no valid output are left out.
Metric: Accuracy (%; D-U2: 100 zero-shot questions on synthetic Synthea patient tables given as plain text, where each item asks to list the IDs of the patients meeting a condition on a categorical field and a less-than condition on a numeric field; an answer counts when it matches the reference exactly; the paper's check mark, a perfect score, reads 100, and its cross, no valid output, reads as missing). Source: arxiv.org. Saturation forecast: Estimated already saturated. 18 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Gemini 2.5 Flash | 58 | #453 |
| 2 | GPT-4.1 | 51 | #495 |
| 3 | Gemini 2.0 Flash | 43 | #704 |
| 4 | DeepSeek V3 | 41 | #716 |
| 5 | DeepSeek V2.5 | 41 | #1210 |
| 6 | GPT-3.5 Turbo | 15 | #2753 |
| 7 | Llama3-Med42-70B | 3 | #1769 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=ehrstruct-synthea-filtering-less-than · How It Works · Data refreshed daily, snapshot 2026-11-13.