htkgh-polecat (Entity-Filtered History): leaderboard
Relation prediction on htkgh-polecat, a hyper-relational temporal knowledge hypergraph built from the POLECAT geopolitical event database (Ahrabian, Boxer and Pujara, 2026). Given the 100 most recent earlier events that share an actor with a held-out event, the model chooses its relation among 42 event-type and mode combinations; accuracy in percent on 5,562 test events, higher is better.
Metric: Relation prediction accuracy (%; choose the relation of a held-out POLECAT event among 42 candidates, PLOVER event types combined with their modes, given the 100 most recent earlier events that share a primary entity with it (entity filter on, location and context filters off); stratified test set of 5,562 events from 2019 to July 2024; non-thinking models answer in at most 14 tokens, thinking models in up to 16,384). Source: arxiv.org. Saturation forecast: Around February 2027. 9 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | Qwen 3 4B 2507 (Thinking) | 40.6 | #1388 |
| 2 | Gemma 3 12B (IT) | 40 | #1849 |
| 3 | Qwen 3 8B (Thinking) | 38.9 | #1838 (Qwen 3 8B) |
| 4 | Qwen 3 4B 2507 Instruct | 36.3 | #2174 |
| 5 | GPT-OSS-20B (Medium) | 34.7 | #1150 (GPT-OSS-20B) |
| 6 | Gemma 3 4B (IT) | 32.7 | #3848 |
| 7 | Qwen 3 8B (Non-reasoning) | 30.9 | #1838 (Qwen 3 8B) |
| 8 | Llama 3.1 8B Instruct | 24.3 | #4277 |
| 9 | DeepSeek-R1-Distill-Qwen-7B | 12.2 | #8045 |
No result here: #1 Claude Fable 5.2, #2 Gemini 4 Carbon, #5 Claude Opus 5.5.
Interactive version: theaggregate.ai/benchmark?slug=htkgh-polecat-entity-filtered-history · How It Works · Data refreshed daily, snapshot 2026-11-12.