How much should an agent remember, and where should that live

Agent memory is five separate stores that teams tend to build as one. The conversation window holds the current thread, a summary compresses what fell out of it, long-term memory holds durable facts about a person, a search index holds documents, and your own database holds the authoritative rows. A fact filed in the wrong one goes stale, crosses a tenant boundary, or gets paid for on every turn.

Teams arrive here from one of two directions. One appends every turn to the prompt until a long thread costs more than the answer is worth and the model starts missing facts that are technically in front of it. Another writes down whatever seemed important during a conversation, and six weeks later the assistant greets a customer with the plan tier they left in March.

How do I give my AI agent memory across conversations

Nothing in a language model persists between requests. Memory across conversations is a read you perform while assembling the prompt and a write you decide on after the turn ends, and keeping those two decisions apart is most of the design. Recall is per turn and recoverable, since pulling the wrong three facts only makes today's answer worse. A write persists, so a bad one is wrong in every conversation until somebody notices.

A turn's prompt gets assembled from parts with different owners: the system prompt, facts recalled for this identity key, documents retrieved for this query, authoritative state read through a tool, the running summary, and the last few verbatim turns. Deciding which part a given piece of information belongs in is the work. Where those parts come from and who may write them sits inside the wider agent orchestration platform question.

Five kinds of memory, and where each belongs

The window is the cheapest of the five and the most abused. Every turn resends it, so turn 40 pays for turns 1 through 39 again, and prompt caching helps only while the prefix stays byte-identical. A rewritten summary or a timestamp near the top of the prompt invalidates that prefix and quietly returns you to full input price on every turn. Sizing and trimming the window is its own problem, covered in managing the context window.

A summary is a lossy write, and it is the store people forget they built. Once the first thirty turns become four sentences, whatever those sentences omit is gone for the rest of the thread, including the correction the user made in turn 12. Summarise on a schedule you control, keep the last few turns verbatim underneath, and store the summary somewhere you can read it back.

Long-term facts should be retrieved by key and then filtered, rather than by similarity over an embedding of every sentence a user ever said. Similarity search is how a note about a customer's brother being a lawyer surfaces in a shipping question: it is the nearest neighbour to nothing in particular, and the model treats anything in the prompt as relevant because it is there. A few dozen keyed facts, read in full, beat a vector index of thousands for this store.

Documents and application state should not be filed as memory, and doing so is the most expensive of the five mistakes. A subscription tier, an order status, a seat count and an open ticket each have an owner elsewhere, and a remembered copy is a cache with no invalidation. Read them at turn time through a tool, so the model sees today's row rather than what was true when somebody wrote it down.

What earns a durable write

The useful default is that nothing is written unless a rule says so. Four tests, all of which have to pass:

• The person stated it about themselves. An inference from tone (seems frustrated) is a judgement, and it will colour every conversation that follows.
• It will still be true in a month. A shipping address for one order belongs to the order.
• No system of record owns it. If a table answers the question, read the table.
• You would be comfortable showing it to them in a list, because you are going to have to.

Give every fact a source and a clock, or you cannot audit it later:

{
  "key": { "tenantId": "acct_8f21", "endUserId": "u_4471", "agentId": "support-assistant" },
  "fact": "Prefers replies in Spanish.",
  "source": { "conversationId": "conv_9d02", "turn": 4 },
  "writtenAt": "2026-08-14T09:12:03Z",
  "expiresAt": "2027-08-14T09:12:03Z",
  "origin": "stated"
}

Cap the store per key in the tens rather than the thousands, and make a new write evict the oldest or lowest-value fact. An append-only memory becomes a second system prompt that nobody reviews, and its cost per turn grows with the customer's tenure.

What should never be remembered

• Credentials, card numbers and one-time codes. Redact at the write, not at the display.
• Health, religion, ethnicity, sexuality, immigration status and similar characteristics, even when the person volunteers them in passing. Agreeing to say something once is a long way from agreeing to have it recalled forever.
• Characterisations of a person rather than statements from them: angry, difficult, unlikely to renew.
• Anything the agent learned through a tool call against tenant data.

That last one deserves a check in code. A tool call is authorised at call time, and a memory write copies its result into a store that recall never re-authorises, so the data escapes its permission check by being remembered. The fix is a write step that can see only the turn's messages, or an explicit allowlist of fields a tool result may contribute.

Keying memory by tenant and by end user

The key is a tuple. At minimum it is tenant, end user and agent, and every part earns its place. Drop the agent and a support assistant's notes surface inside a sales assistant. Drop the tenant and you get the failure that is hardest to find in testing.

One human can belong to two tenants. A contractor working for two of your customers, an agency running three accounts, a person with a work login and a personal one: each is a single end-user identity living in two isolated contexts. Key on the end user alone and a fact learned in one customer's workspace is recalled in another's, which is a disclosure even when the fact is dull.

Key on identity the request proved rather than identity it asserted. An end-user id read out of a request body is a claim from the caller, and memory keyed on a claim is memory any caller can address. The wider version of this is in multi-tenant AI agents, and the per-resource checks are in the tenant isolation checklist.

Expiry, correction and deletion

Every fact carries an expiry chosen when it is written, not a single global retention setting. A stated language preference can live a year. A note about the project someone is working on this quarter should be gone in ninety days, because an assistant that asks about a finished project reads worse than one that never knew.

Correction happens more often than deletion and is usually handled worse. When a user contradicts a stored fact, the turn overwrites it rather than adding a second row, or recall starts returning both and the model believes whichever it reads first.

Deletion has to reach every copy. Erasing a row while a rolling summary still carries the sentence, or while thirty transcripts sit in a retrieval index, means the fact keeps coming back while your deletion log says it is gone.

A test for memory leakage

Leakage stays invisible in normal testing, because the agent behaves correctly with one user in one tenant. It needs a probe with a distinctive token, run across boundaries:

1. In tenant A, as end user 1, state an invented codename that no plausible answer could contain by accident.
2. Open a fresh conversation in tenant B with the same human identity and ask the question that would surface it.
3. Repeat inside one tenant with a second end user, then with the same end user against a different agent.
4. Assert on the recalled set in the execution trace, not on the reply text. An agent that retrieves the fact and declines to mention it has still leaked it, and the next prompt revision will say it out loud.
5. Run the probe on every deploy. Keying regressions arrive as refactors, not as features.

The same probe covers deletion. Delete the fact, re-run steps 2 and 4, and check the summaries and the retrieval index alongside the keyed store.

Where this gets easier

Keeping five stores apart is mostly a matter of having five places to put things, and platforms differ in how many they give you. Runtype provides records and collections for structured application data, and opt-in long-term memory keyed per agent, per tenant or per end user through {{_endUser.id}}, so a fact written under one customer's key is not addressable from another's. Conversation history is a separate store again: each thread is persisted as its own conversation record, addressable by conversation id, and nothing said in it reaches long-term memory unless memory is switched on for that agent. The identity that the keying depends on is documented at end-user identity.