state-drift benchmark · local-first memory · Apache-2.0
dreamd
A neutral benchmark for AI memory systems under changing facts. Does your memory layer update when facts change, or does it confidently answer with yesterday's truth?
The benchmark publishes October 2026. Code is available now.
the benchmark
State-Drift: what happens when a fact changes?
Tell a memory system "I live in London." Weeks later, tell it "I moved to Tokyo." Then ask where you live. Some systems update. Others answer London, with full confidence. That failure mode is state drift, and no published evaluation measured it before this one.
The State-Drift Benchmark runs Mem0, Zep, Letta, Anthropic's managed memory, and dreamd through the same scenarios with the same scoring. dreamd is one row in the table, not the winner. The harness, the scenarios, and the scoring are all in the repo, so any number in the table can be reproduced before it is believed.
Open source. Every major system tested. One honest table.
the memory layer
Drop a folder. Your agents share it.
The other half of the project: a local-first memory layer that multiple coding agents read and write. Single binary, MCP server, one .agent/ folder per repository. It is also the reference implementation the benchmark was built against.
Shared across every agent
Your preferences and project decisions live here and are read at session start. Every supported agent writes corrections and preferences back to the same plain files automatically.
Every observation captured
Raw entries accumulate in AGENT_LEARNINGS.jsonl, an append-only log. Nothing your agent learns is discarded before the dream cycle evaluates it.
High-signal lessons promoted
The dream cycle distills the log into LESSONS.md. The salience formula scores by age, pain, importance, and recurrence. High scorers are promoted; the rest decay.
supported tools
One memory layer, every harness you use.
One MCP server. Every harness connects to it. Drop the .agent/ folder once, point every agent at it.
trust signals
Everything you need to know, in numbers.
No marketing claims. Every statement is a number you can verify.
.agent/, your repo install
One command, one folder, zero config.
Run this in your project root. The daemon starts, the .agent/ folder appears, and every supported agent discovers it automatically.
npx dreamd-mcp Requires Node.js 20+. One-time setup. The daemon starts automatically and runs silently in the background.
project status
Where things stand.
npx dreamd-mcp A Botzr Research project · botzr.com
faq
The objections answered plainly.
Why trust a benchmark run by one of the systems in it?
The right default is not to. The harness, the scenarios, and the scoring are all open source, every system runs the same scenarios, and dreamd's row is computed exactly like every other row. Where dreamd scores badly, the table says so. Reproduce any number yourself before you believe it.
Isn't the memory layer just CLAUDE.md with extra steps?
CLAUDE.md is a static file you hand-edit. dreamd is a daemon that reads and writes structured memory across every session and every harness, automatically. If you only use one agent in one repo and you're happy editing markdown by hand, you don't need dreamd.
Why not a vector database?
Vector DBs solve retrieval over unstructured text at scale. dreamd targets structured, high-signal memory for coding agents: preferences, corrections, project state, external references. The data model is closer to a typed key-value store than a vector index. The salience formula scores each entry; recall is deterministic, not approximate.
Why Rust?
Single binary, no runtime dependency, predictable latency under load. dreamd runs as a sidecar and should never be the slow thing in your agent loop. Rust's type system also maps cleanly to the memory schema enforced across files.
How does dreamd relate to Mem0, Zep, and Letta?
They are all memory systems, and they all run the same State-Drift scenarios in the benchmark. Architecturally they differ: Mem0 and Zep are hosted APIs, Letta is an agent platform with managed memory, dreamd is a local daemon writing to a .agent/ folder in your repo. Which architecture holds up under changing facts is what the benchmark measures. Architecture notes are on the comparison page; quality claims wait for the results.
Does this work with OpenCode / Aider / Continue / Goose / Roo Code?
v0.1 ships with a native MCP server for Claude Code, Cursor, and Cline. OpenCode, Aider, Continue, Goose, and Roo Code adapters are planned for v0.1.1 and will follow the same .agent/ folder contract. The spec is open; contributions welcome.