Probability Operator Inference - Competence Floor: leaderboard

Metric: Competence floor (%; the lower of the accuracy on items whose correct answer is Yes and on items whose correct answer is No, each the mean of its valid and invalid template cells, so a constant responder scores 0 and a coin flip 50, on 14,320 procedurally generated English yes or no prompts over fifteen inference templates with gradable epistemic modals (must, probably, might), varying question form, negation and surface content; zero-shot, temperature 0, first answer token parsed, uncertain responses counted as incorrect; higher is better). Source: arxiv.org. Saturation forecast: Around December 2026. 29 models tracked.

Top models

Interactive version: theaggregate.ai/benchmark?slug=probability-operator-inference-competence-floor · How It Works · Data refreshed daily, snapshot 2026-09-29.