VARM-Bench (Taxonomy-Guided): leaderboard
Metric: Joint record exact match (%; outputs that parse, reach target character-F1 of at least 0.5 and match the reference on the other five fields; deduplicated 1,600-item test set of Chinese social-media posts and comments; the model writes one rationale with six anchored decisions (target, target type, target explicitness, author stance, harmfulness label, fine-grained category) that a deterministic parser scores, unparseable outputs counting as wrong; deterministic decoding; zero-shot prompt with the category taxonomy). Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | VARM-Bench Qwen2.5-7B (checkpoint unspecified) | 36.4 |
| 2 | VARM-Bench Llama-3.1-8B (checkpoint unspecified) | 30.4 |
| 3 | VARM-Bench InternLM3-8B (checkpoint unspecified) | 12.7 |
Interactive version: theaggregate.ai/benchmark?slug=varm-bench-taxonomy-guided · How It Works · Data refreshed daily, snapshot 2026-09-29.