DolphinBench (Built-in memory): leaderboard
DolphinBench (mem0ai, arXiv 2609.24971) tests whether an agent uses long-term memory to act correctly for a user. The agent first ingests about 500,000 tokens of dated messages from each of three simulated knowledge workers through its memory, then completes 200 tool-using tasks per persona in fresh conversations against simulated apps. Unlike LoCoMo or LongMemEval, whose questions signal which fact to retrieve, DolphinBench grades the app calls the agent makes (a recipient, a date, a tool argument, the content of an email or document) with deterministic and semantic checks, and every task was certified to pass with its source history and fail without it. This board admits only configurations that use the harness's own built-in memory; rows name the harness (Hermes or Claude Code), the model and the reasoning effort its run configuration records. Pass rates count all 600 tests, so a task that is not passed for any reason counts against the configuration. Configurations with the Mem0, Honcho, Hindsight or Supermemory memory products rank those products rather than models and are not admitted.
Metric: Tests passed with the harness's built-in memory (%, 600 tests). Source: dolphinbench.ai. Saturation forecast: Rough model projection: around 2026. 3 models tracked.
Top models
| # | Model | Score | Overall rank |
|---|---|---|---|
| 1 | GPT-5.6 Luna (Benchmark Sweep Run: source=DolphinBench; harness=Hermes, version unreported; memory=built-in; source-model=gpt-5.6-luna-high; effort=high; provider=azure-foundry; max turns=60; configuration=hermes-luna-high-builtin-20260916) | 65.67 | |
| 2 | MiniMax-M3 (Benchmark Sweep Run: source=DolphinBench; harness=Hermes, version unreported; memory=built-in; source-model=minimax-m3-high; effort=high; provider=openrouter; max turns=60; configuration=hermes-minimax-m3-high-builtin-20260921) | 26.5 | |
| 3 | Claude Sonnet 5 (Benchmark Sweep Run: source=DolphinBench; harness=Claude Code, version unreported; memory=built-in; source-model=claude-sonnet-5; effort=medium; configuration=claude-code-sonnet-5-builtin-20260921) | 26.33 |
Interactive version: theaggregate.ai/benchmark?slug=dolphinbench-built-in-memory · How It Works · Data refreshed daily, snapshot 2026-11-12.