GEITje-7B-ultra: benchmark results

BramVanroy's GEITje 7B ultra, a Dutch DPO chat model built on Rijgersberg's Mistral-based GEITje-7B, trained on 56M tokens of GPT-4-turbo feedback. Provider: BramVanroy. Released 2024-01-27. Access: Open.

Aggregate Rating 1310 ± 24, rank #1435 of 1635 rated models, from 25 benchmark results.

Strongest benchmark results

BenchmarkScoreMetricPercentile
Grip on LLMs - Dutch Simplification - Amsterdam46.91SARI (0-100)100
Grip on LLMs - Dutch Simplification - INT Duidelijke Taal44.63SARI (0-100)100
Grip on LLMs - Dutch Simplification Average45.77SARI (0-100; mean of two datasets)100
Grip on LLMs - HonestCityBench - Narrow Expertise5Appropriate acknowledgement rate (%; LLM judge)91.7
Grip on LLMs - HonestCityBench - Outdated Information32Appropriate acknowledgement rate (%; LLM judge)70
Grip on LLMs - HonestCityBench - Insufficient Information35Appropriate acknowledgement rate (%; LLM judge)66.7
Open LLM Leaderboard v1 - TruthfulQA MC253.36MC2 (%) (0-shot)59.3
Grip on LLMs - HonestCityBench - Average26Appropriate acknowledgement rate (%; mean of five categories50
Grip on LLMs - HonestCityBench - Non-Text Modality18Appropriate acknowledgement rate (%; LLM judge)48.3
Open LLM Leaderboard v1 - GSM8K19.41Accuracy (%) (5-shot)40.6
Open LLM Leaderboard - IFEval37.23Score36.1
Grip on LLMs - HonestCityBench - Incorrect Premise41Appropriate acknowledgement rate (%; LLM judge)30

Interactive version: theaggregate.ai/model?slug=geitje-7b-ultra · How It Works · Data refreshed daily, snapshot 2026-11-05.