TeleResilienceBench - 3GPP TSG: leaderboard
Metric: Correct flip rate (CFR, %): share of the 90 3GPP TSG instances (classifying 3GPP documents by working group, GSMA Open-Telco ot-lite) where the model, given the question, the options and the first half of a flawed reasoning trace from qwen3.5:2b that ended in a wrong answer, continues the reasoning in its thinking stream and returns the correct option; one continuation prompt, Ollama builds on one RTX 4090; higher is better. Source: arxiv.org. Saturation forecast: Rough model projection: around 2026. 8 models tracked.
Top models
| # | Model | Score |
|---|---|---|
| 1 | gemma4:26b (Ollama) | 16.7 |
| 2 | gemma4:31b (Ollama) | 16.7 |
| 3 | nemotron-3-nano:4b (Ollama) | 11.1 |
| 4 | qwen3.5:9b (Ollama) | 5.6 |
| 5 | gemma4:e2b (Ollama) | 5.6 |
| 6 | gemma4:e4b (Ollama) | 5.6 |
| 7 | qwen3.5:27b (Ollama) | 4.4 |
| 8 | qwen3.5:4b (Ollama) | 3.3 |
Interactive version: theaggregate.ai/benchmark?slug=teleresiliencebench-3gpp-tsg · How It Works · Data refreshed daily, snapshot 2026-10-07.