RAG vs agent memory

Both put retrieved text into a prompt. That is where the likeness ends.

The short version

	RAG	Agent memory
Source	Documents written beforehand	Facts learned during conversations
Who writes it	Authors, outside the agent	The agent itself
Belongs to	Usually everyone, or a team	Usually one user or one project
Changes	When documents are re-indexed	After any conversation
Must handle	Chunking, freshness of the index	Contradiction, updating, deletion
Answers	“What does the policy say?”	“What did this person tell me?”

What RAG is for

Retrieval-augmented generation gives a model knowledge it was not trained on: your handbook, your product docs, last week’s reports. The documents exist whether or not anyone talks to the agent. The work is in cutting them into chunks, indexing them and keeping the index current.

What memory is for

Memory gives a model knowledge about the people and tasks it has dealt with. Nobody writes these facts in advance. The agent has to notice them, write them down, and later realise that “moved to Porto” replaces “lives in Lisbon”.

That last part is the real difference. A document index is read-mostly. A memory is rewritten constantly, by a model, about a person. So memory needs things RAG can skip: a rule for what is worth keeping, a check for duplicates and contradictions, a view for the user, and a delete that actually deletes.

When you need both

A support agent answering “can I get a refund?” needs the refund policy (RAG) and the fact that this customer bought the annual plan three weeks ago (memory, or the customer record). Retrieve from each, give each its own heading and its own token budget in the prompt, and tell the model which is which.

A common mistake

Indexing every chat transcript and calling it memory. It works in a demo. In production it recalls stale statements with the same confidence as current ones, returns three versions of the same fact, and cannot honour a request to forget. Extract first, then store.