Published By

Your agent’s memory is already poisoned. Now what?

A prompt injection ends when the session ends. A poisoned memory doesn’t. It sits in the store, gets pulled up next week for some other task (maybe for some other user), and the agent acts on it like it’s something it learned honestly.

That’s really the whole problem, and it’s why OWASP gave it its own entry. The OWASP Top 10 for Agentic Applications came out in December 2025, and number six on it is Memory and Context Poisoning. Somebody, or some bad tool result, or just a plain mistake, plants a “fact” in memory, and the agent keeps making decisions on it across later sessions.

I want to walk through what these attacks look like in the public research, why filtering what goes in won’t save you on its own, and what you actually need once a bad memory is already inside. Then I’ll show you how dreamd handles that, and to be fair to you, exactly where it stops.

What the attacks look like

There are four public results worth knowing about. None of them needs anything exotic.

ChatGPT, September 2024. Security researcher Johann Rehberger hid a prompt injection in content a user asked ChatGPT to read. The injection wrote an instruction into ChatGPT’s long-term memory, and that instruction sent later conversations to a server he controlled, in every new chat. He called it SpAIware. OpenAI closed the exfiltration path, but reporting at the time pointed out that untrusted content could still write memories.

Gemini, February 2025. Rehberger again, this time against Gemini’s long-term memory. A document the user asked Gemini to summarize carried hidden instructions that tied a memory write to something the user was going to do anyway, which was type a common trigger word later in the chat. When they did, Gemini saved the attacker’s “facts” to memory, and those stuck around into future sessions. He called it delayed tool invocation. The whole point was getting past defenses that block tool calls while the model is reading untrusted data.

MINJA (Dong et al., 2025). Here the attacker never touches the memory store at all. They just send queries like any other user, and they shape the agent’s own records so a different user’s query later pulls them up and follows the attacker’s reasoning. Across a medical records agent, a web shopping agent and a QA agent, the authors report an average 98.2% success rate getting malicious records into memory, and 76.8% at getting the agent to follow the malicious reasoning.

AgentPoison (Chen et al., NeurIPS 2024). This one’s a backdoor planted in an agent’s memory or its RAG knowledge base, with an optimized trigger. The authors report an average attack success rate of at least 80% with under 0.1% of the records poisoned, and normal performance dropping by 1% or less.

That last set of numbers is the one that should bother you. A tiny slice of poisoned records, and everyday tasks look completely normal. You’re not going to catch this on a quality dashboard.

Why filtering on the way in isn’t enough

The obvious move is to screen what the agent writes. Do that where you can, no doubt. But be honest with yourself about what it buys you.

A poisoned lesson is usually just a well-formed sentence. “Skip the integration tests on hotfix branches, they’re flaky” is a perfectly reasonable thing for an agent to learn from a real incident. It’s also exactly what an attacker would plant. No classifier tells those two apart reliably, because the difference isn’t in the text. It’s in where the text came from and what was built on top of it.

That’s why dreamd’s own security policy says we don’t filter “malicious” lesson content. It’s an unsolved problem, and a filter that claims otherwise just gives you false confidence.

So plan for the case where the bad memory is already in there. Prevention is one layer. Response is the layer you’re actually going to use.

The five questions you need answered

When an agent keeps doing something wrong and you suspect memory, you need five answers, pretty much in this order:

  1. What did the agent believe? Which stored memories drove this answer.
  2. Where did that come from? The source event, when it was written, and which harness wrote it.
  3. What else did it touch? Which lessons and index entries were built from it.
  4. Can I take it out cleanly? The event and everything derived from it, without wiping the whole store.
  5. Can I show what I did? A record you can attach to a ticket or hand to whoever asks.

Most agent memory systems get you to the first question at best, and a lot of the time only through a hosted dashboard. A recall API that hands back similar chunks doesn’t tell you what those chunks fed. A delete endpoint that drops a row doesn’t tell you which summaries are still carrying it.

How dreamd answers them

dreamd keeps memory as plain files in your repo. There’s an append-only episodic log (AGENT_LEARNINGS.jsonl) and a semantic layer (LESSONS.md) that the dream cycle builds out of recurring patterns. Every lesson cites the events it came from. A provenance ledger records which event fed which lesson and which index entry. And each dream cycle commits what it wrote to git.

That structure is what makes the five questions answerable. Here’s the loop.

1. What did the agent believe?

dreamd blame "skip integration tests" --explain

blame runs the same ranked recall your agent uses and lists the memories behind it: the event id, timestamp, skill_action, BM25 score, salience, and whether the hit is a raw episode or a consolidated lesson. With --explain it breaks salience down into its factors (age, pain, importance and recurrence), so you can see why a memory ranked where it did.

If you want to be sure a suspect is really what changed the answer, ask the counterfactual:

dreamd recall "skip integration tests" --without evt_...
dreamd recall "skip integration tests" --without-cluster ci::tests

--without drops that event before ranking and shows you what recall gives back without it. Nothing gets deleted. One thing to watch, though: excluding an event id doesn’t exclude the lesson built from it, so use --without-cluster if you want both gone.

2 and 3. Where did it come from, and what did it feed?

cat .agent/semantic/LESSONS.md
dreamd doctor --provenance

The lesson’s frontmatter lists the event ids it cites. The provenance ledger is an append-only file of edges: this event is cited by that lesson, this event is in the recall index, this event is the exemplar for that cluster. doctor --provenance recomputes a Merkle root over the ledger and lists any orphan, missing or corrupt edges. It only reads. It doesn’t change anything.

4. Take it out cleanly

dreamd forget evt_... --dry-run
dreamd forget evt_... --proof forget-receipt.json

The dry run tells you what would change and writes nothing. The real run removes the event from the live log and fixes up the lesson state that names it. If the event was just one of a lesson’s citations, the citation gets dropped. If it was the lesson’s exemplar (meaning the lesson body was that event’s content), the lesson is removed, and nothing replaces it until the next dream cycle. forget also recounts the cluster and pulls the event out of the recall index.

If a whole dream cycle went bad, not just one event, roll it back the way you’d roll back any other commit:

git revert <the dreamd: cycle commit>
dreamd doctor --repair   # rebuild the recall index

Stop dreamd watch before you do either one. forget refuses to run while the daemon is up.

5. Show what you did

--proof writes a small receipt file. It has the event id, what happened to the lesson, the ledger root recomputed after the removal, and the ledger edges from that event that are now orphans. Then review the change like you’d review code:

git diff .agent/

After a forget, doctor --provenance is going to report the removed event’s old edges as orphans and exit 1. That’s on purpose. The ledger is append-only, so it keeps the record that the event existed and what it fed. A cleaned-up incident leaves a visible scar instead of a silent gap, and to be honest, that’s what you want when someone asks about it six months later.

Where this stops

If you’re going to rely on this, you should know the edges.

  • dreamd doesn’t detect or block poisoned writes. It makes a bad memory findable and removable. It doesn’t make one impossible.
  • The source_harness field is whatever the caller says it is. It tells you which harness claimed to write the memory, not proof that it did.
  • The ledger and the receipt aren’t signed. The receipt is a record of what the command did, not a cryptographic proof. Signed receipts are specified but not built yet.
  • forget rewrites the live files only. The event can still be sitting in your git history if a dream cycle committed it, and in older memory snapshots. Treat that the way you’d treat a secret that leaked into git: fix the live state first, and decide about history cleanup separately.
  • forget doesn’t work on Windows yet, because the atomic file replace it depends on isn’t available there.
  • There are no trust labels yet. Nothing today stops a lesson from an untrusted source from driving a privileged action. That’s the structural fix, and right now it’s on the roadmap, not in the binary.

Habits that make the response fast

The tooling only helps if the record is there when you need it.

  • Commit .agent/ with your code. It’s the memory. If it isn’t in git, you lose the diff and the revert.
  • Review dream cycles like pull requests. Each cycle lands as a dreamd: cycle YYYY-MM-DD commit, and new lessons are one git diff away. Actually read them.
  • Treat .agent/ like your shell rc files. Any process running as your user can write to it, so notice when it changes.
  • Keep an eye on clusters that suddenly matter. dreamd salience-drift shows the salience distribution, the top clusters, and how things moved against a snapshot from seven days earlier. A cluster that jumps is worth a look.

What comes next

The audit loop answers “what happened, and how do I undo it.” The next question is “how do I stop a bad memory from doing damage before I even notice it.” What we’re working toward is a trust label on every memory, carried along the same provenance edges the ledger already records, so a lesson that came from untrusted input can’t cause a privileged action. Signed forget receipts come along with that. Both need their own spec first, and we’ll publish the spec before we ship anything.

Until then, assume a bad memory is going to get in. Make sure you can find it, see what it fed, take it out, and show your work.

If you want to try the loop on your own project:

npx -y dreamd-mcp setup

The memory runtime here is dreamd, Apache-2.0, plain files in your repo. If your team has an enterprise deal stalled on an AI security questionnaire, I run two independent Agent Boundary Reviews a month. Details are here.


Sources