TOSSS (Java): leaderboard
Metric: TOSSS security score (0-1, times 100): the share of 500 Java function pairs (the function before and after a CVE fix, mined from MegaVul) for which the model picks the secure version when shown both in random order with a neutral prompt that does not mention security; a constant or random choice scores about 50; higher is better. Source: arxiv.org. Saturation forecast: Around January 2027. 14 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.4 | 84.6 | #76 |
| 2 | GLM-5 | 83.6 | #137 |
| 3 | Claude Opus 4.6 | 83.4 | #60 |
| 4 | Kimi K2.5 | 81.4 | #139 |
| 5 | Gemini 3 Flash (Preview) | 80.2 | #78 |
| 6 | MiniMax-M2.5 | 79.2 | #295 |
| 7 | Claude Sonnet 4.6 | 78.2 | #85 |
| 8 | Gemini 3.1 Flash Lite (Preview) | 74.2 | #186 |
| 9 | Claude 3.5 Sonnet | 73.8 | #337 |
| 10 | Llama 3 70B Instruct | 73.2 | #624 |
| 11 | Qwen3 Coder Next | 65.8 | #321 |
| 12 | DeepSeek V3.2 | 65.6 | #198 |
| 13 | codestral-2508 | 64.6 | #518 |
| 14 | Mistral Large 3 | 48.8 | #388 |
No result here: #3 Claude Opus 5.5, #5 GPT-6 Astra, #8 Claude Fable 5.1.
Interactive version: theaggregate.ai/benchmark?slug=tosss-java · How It Works · Data refreshed daily, snapshot 2026-10-11.