ALLMTS Raw LLM - 50 Repeated Weighted Reference Kappa: leaderboard
ALLMTS paper v1 raw-LLM standalone baseline only, Azure Responses GPT-5.2 low effort. No guideline retrieval/execution, web search or ALLMTS system scores attributed to the model. 50curated or566full synthetic expert-validated scenarios are distinct cohorts. Ad-hoc views prompt direct labels; post-hoc views map predicted disposition labels. Nativefraction accuracy/support-weightedF1 higherisbetter; under/over-triagefractions lowerisbetter. Different labeltaxonomies remain separate. Ten independentruns of50curatedscenarios, source-reported acuityview (mappingroute not restated in Table3). Referenceagreement, not selfconsistency. Metric: Weighted Reference Kappa.
Metric: Weighted Reference Kappa. Source: arxiv.org. 1 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.2 (low) | 0.71 | #115 (GPT-5.2) |
Interactive version: theaggregate.ai/benchmark?slug=allmts-raw-llm-50-repeated-weighted-reference-kappa · How It Works · Data refreshed daily, snapshot 2026-10-27.