Vector Memory

aka Semantic Memory, Embedding-Indexed Memory

category: memory · status: mature

Intent

Store memories as embeddings in a vector index and retrieve the most semantically similar items at query time.

Context

Long-running agents accumulate facts/observations whose relevance is best judged by similarity to the current context.

Problem

Append-only logs grow unboundedly; without semantic retrieval the agent cannot find the relevant past.

Forces

Embedding choice constrains retrieval quality.
Index updates have non-trivial latency.
Forgetting is achieved by deletion or decay; both have failure modes.

Solution

Each memory item is embedded and indexed. At query time, embed the query (or a summary of current state), retrieve top-k most similar memories, prepend to context. Optional decay (boost recent, age old) and salience weighting.

Constrains

The agent reads memory only through the retriever; full-store scans are not part of the loop.

Consequences

Benefits

Semantically relevant past surfaces automatically.
Scales to memory stores too large for context.

Liabilities

Misses purely temporal queries ('what did I do yesterday?').
Embedding drift on schema changes.

Known Uses

MemGPT / Letta archival memory
Generative Agents memory stream (Park et al.)
LangChain VectorStoreRetrieverMemory

Related Patterns

used-by → memgpt-paging
specialises → naive-rag — Vector Memory is RAG over the agent's own past.
alternative-to → knowledge-graph-memory

References

Generative Agents: Interactive Simulacra of Human Behavior — Park et al. (2023) paper