Can a Vector Database Be Agent Memory? What pgvector, Pinecone and Qdrant Give You, and What You Still Build

Published 27 October 2026 by Gaurav Dadhich. Every vendor capability below was checked against that vendor's own documentation on 27 October 2026.

A vector database can be the storage and retrieval layer of agent memory, but it cannot be the memory on its own, because everything that keeps memory correct happens on the write path, and a vector database writes only what you hand it. pgvector, Pinecone and Qdrant all answer one question well: which stored records sit closest to this query, inside this filter. None of them decides what is worth keeping from a conversation, which existing record a new statement replaces, which tenant or person a statement belongs to, when a fact stopped being true, or how a deletion reaches the summaries and extracted facts built from the original. Those five operations are the difference between a retrieval index and a memory, and building on a vector database means writing all five yourself.

For some agents a vector database alone is genuinely the right answer. The decision underneath is who owns the write path, not which index is fastest, and most pgvector, Pinecone and Qdrant comparisons stop before reaching it.

What does a vector database actually give you for agent memory?

You get approximate nearest-neighbour search over embeddings, filters on that search, upserts and deletes by ID, and increasingly a keyword channel fused with the vector one. That covers the read path of memory, and all three handle it well at the scale most agents run at. They differ in tenant isolation, in how filters interact with the index, and in how deletes behave, which are exactly where memory puts its load.

pgvector

pgvector is a Postgres extension, at v0.8.7 today, with two index types, HNSW and IVFFlat, and indexable vectors up to 2,000 dimensions (4,000 with halfvec). Its biggest advantage for memory has little to do with vectors: memories live in the same database as your users, accounts and permissions, under the same transactions and backups, and a filter is a SQL WHERE clause.

Read the filtering note twice. The README says that with an approximate index, "filtering is applied after the index is scanned", so with the default hnsw.ef_search of 40 and a condition that matches 10% of rows, a query returns about four rows on average. A per-user memory lookup is exactly that kind of selective filter. Version 0.8.0 added iterative index scans (hnsw.iterative_scan set to strict_order or relaxed_order), which keep scanning until enough rows pass the filter; turn them on for any memory table that more than one user shares.

On tenants the README is blunt: a shared approximate index lets one tenant's vectors affect another tenant's recall and speed, and "for tenant isolation, use list partitioning or separate tables." Postgres adds row-level security, which restricts the rows a role can read or modify; superusers and roles with BYPASSRLS always bypass it, and so do table owners unless you set FORCE ROW LEVEL SECURITY. Hybrid search is Postgres full-text search fused with the vector result through Reciprocal Rank Fusion, which you assemble, and vacuuming an HNSW index after deletes can take a while.

Pinecone

Pinecone is a managed service, and its answer to multitenancy is the namespace. Its multitenancy guide says namespaces provide "physical isolation of data between tenants/customers", that "reads and writes always target a single namespace", and that offboarding a tenant means deleting its namespace, "a lightweight and almost instant operation." Serverless indexes allow from 100 namespaces on Starter to 1,000,000 on Enterprise.

The alternative, one namespace with a tenant field in metadata, carries two warnings from Pinecone itself: queries "scan the entire namespace regardless of filters", so you pay to scan every tenant's data, and the $in and $nin operators take at most 10,000 values each. Deletes work by ID (up to 1,000 per request), by metadata filter (5 requests per second per namespace), or for a whole namespace. Pinecone is eventually consistent, so a changed or deleted record can stay visible to queries for a short delay. Hybrid search is either two indexes, dense and sparse, linked by ID and merged client-side with a method such as RRF, or one hybrid index where you set the balance by scaling the query vectors. I found no time-to-live setting in the data-management documentation.

Qdrant

Qdrant is open source and runs self-hosted or managed, and its documented best practice for multitenancy is a single collection with payload-based partitioning. A keyword payload index marked is_tenant stores each tenant's points together, and setting m=0 with payload_m=16 builds a separate HNSW graph per tenant instead of one global graph; the docs warn against those settings if you will ever search across tenants. The Query API runs prefetch sub-queries and fuses them with RRF (from v1.10) or DBSF (from v1.11), so dense plus sparse hybrid search is native, and formula queries (from v1.14) can apply an exponential datetime decay so that older points score lower. Points are deleted by ID list or by filter, and the documentation describes no time-to-live: expiry means a datetime payload, an index on it, and a scheduled delete by filter.

Pgvector vs Pinecone vs Qdrant for AI agent memory: which should you use?

Pick the one that matches what you already operate, then judge it on the four things memory stresses: recall under a selective per-user filter, tenant isolation, delete behaviour and hybrid search. Raw nearest-neighbour speed rarely decides it at memory scale.

pgvector is the default if Postgres is already your system of record, because memories next to relational data, inside the same transaction, with row-level security, is an advantage neither of the others offers; turn on iterative scans and partition by tenant before the second customer arrives. Pinecone fits when you want nothing to operate and a namespace per tenant, or per user, matches your shape; plan around eventual consistency on deletes and the namespace ceiling on your plan. Qdrant fits teams that self-host, filter heavily, and want hybrid retrieval and recency decay expressed inside the query itself.

Give whichever you choose a keyword channel, because memories are full of names, product codes and dates that embeddings handle poorly. In our retrieval study published in June 2026, we indexed 50,000 documents from five datasets (including CodeXGLUE, SciQ and HotpotQA) and ran 5,000 queries, scored on MRR@10 by exact document-ID match with no judge model. Vector search (ChromaDB) led the five-dataset average 0.6320 to 0.5325, but with the code dataset (CodeXGLUE) removed, keyword search (Tantivy) won the remaining four 0.5931 to 0.5614, and indexed in 2.11 seconds against 161.6 seconds with embedding generation (build versus buy agent memory).

What does a vector database not do for agent memory?

Five operations sit on the write path, and none of the three databases performs any of them. Each one can be built, and each is where homegrown memory tends to break.

Deciding what to keep

Whatever you upsert gets stored. One support conversation holds a durable fact ("we run on Azure now"), a preference ("send me summaries, not calls"), small talk, a pasted error log and the agent's own replies; embed all of it as chunks and every future retrieval competes with filler. Deciding which parts are durable, rewriting each as a statement that stands on its own, and attaching a type and a confidence is an extraction step, usually a model call run in the background.

Updating a fact that changed

All three support an update by ID, and that is the easy half; the hard half is finding the ID. When a customer says in June that they moved from AWS to Azure, nothing tells the database that this replaces what they said in March. The two statements embed close together, so a similarity search returns both, ranked by similarity with no view on which one is still true, and the model guesses on every call. You need a step that pulls candidates, decides whether the new statement replaces, adds to or duplicates each one, and records that decision. Overwriting is not safe by default either: "takes a cholesterol medication" must not replace "takes 20mg of Atorvastatin daily", because the newer statement carries less detail than the one it resembles. Keep the old record, mark it historical and link it to its replacement, so the agent stops repeating what is no longer true and you can still say why it changed.

Scoping by tenant at write time

Pinecone namespaces, Qdrant's is_tenant index and Postgres partitions all let you fix the tenant at write time, which is far safer than a filter someone must remember on every read. What none of them can do is decide which scope a statement belongs to. "Our company is moving to Azure" belongs to the account and every user in it; "I prefer Python examples" belongs to one person; "refunds are allowed within 30 days" belongs to your whole product. Hand the store the wrong partition key and it files the fact in the wrong place without complaint. Where the scope is only a read-time filter on a shared index, a dropped filter does not throw an error either: the agent produces a fluent answer from another customer's data, and nothing in your monitoring shows it. Most agent memory failures in production are scoping failures, not recall failures.

Forgetting that reaches derived data

Deleting by filter removes the vectors that carry a user's ID. It leaves the conversation summary that mentioned them, the fact copied into an account-level profile, the cached context block, the separate keyword index and last week's backup. Under GDPR Article 17 a person can ask you to erase their personal data, and the embeddings are part of it: Morris and co-authors reconstructed 92% of 32-token inputs exactly from their embeddings, and recovered full names from a dataset of clinical notes. Erasure in a memory system is a fan-out across every store that derived something from the original, which requires lineage from each derived record back to its source. Eventual consistency and vacuum timing are details on top; the lineage is the real work.

Knowing when a fact was true

A timestamp in metadata records when a statement was stored. Memory needs two dates, when something was said and when it is true ("my contract renews on 1 March" is stored in January and true in March), plus a rule that retires a fact once a newer one supersedes it. Qdrant's datetime decay ranks newer points higher, which helps, but it does not mark an older point invalid, so "lives in Chicago" still comes back beside "moved to Berlin", only lower. Recency weighting is not validity. Entity resolution is the sixth gap ("my manager", "Sarah" and "Sarah Chen" are three unrelated strings to a vector index), and our knowledge graph versus vector search comparison covers it with bi-temporal modelling.

When is a vector database alone the right answer?

A vector database alone is right more often than memory vendors admit. If the content is shared and changes only when someone publishes a new version, such as a product manual, a policy set or a codebase, a vector database with hybrid search is the right tool and memory is the wrong one; that is retrieval-augmented generation. If a user's history is append-only and the agent is asked about it rather than corrected by it ("what did I say about the venue?"), retrieval over the raw record holds up well on today's benchmarks, a result the agent memory versus RAG guide walks through with the October 2026 papers behind it. So are a one-user prototype and a device with no network.

Qdrant's Reachy Mini write-up from 14 October 2026 is a clean example of that last case. The robot runs on a Raspberry Pi with 4 GB of RAM, Qdrant Edge loads as a library inside the process with no server, and three shards hold conversation exchanges and camera frames, one point per known person, and 62 facts about Qdrant and the robot; the authors report about half a millisecond per search over 500 memories. Writes follow simple rules: when the context fills, the oldest half of the exchanges moves into memory, and a face is saved when Reachy learns a name. The knowledge shard is read-only, restored from a 0.7 MB snapshot at every start, and the post describes no updating or forgetting. For one robot in one house that is the right design: nobody files an erasure request, and a stale memory costs an odd remark. Put the same design behind a support agent serving a thousand companies and every missing operation turns into an incident.

When should you use an agent memory framework instead of just a vector database?

Switch when you notice you are writing the write path. An extraction prompt, a "does this replace that" step, a scope lookup before each upsert, a nightly deduplication job or an erasure script that touches more than one store all mean you are already building a memory layer on top of the vector database, and the real question is whether to keep building it. Two conditions make it urgent: facts about a person change and the agent must use the current one across sessions, and more than one tenant shares the system. A memory layer does not remove the vector database; most keep one, often pgvector, Pinecone or Qdrant, for the read path. Migrating usually means replaying stored conversations through the new write path, then moving reads across once isolation and deletion tests pass. A minimal Postgres version of that write loop, one table and one function, is in how to give an LLM long-term memory.

Memory API vs building custom RAG infrastructure: what do you own?

Building means owning the extraction prompts and their evaluation set, the update decision, the scope model, erasure fan-out, validity rules, consolidation as the store grows, and the on-call rota for all of it. The database bill is the small line. Extraction-based memory makes two to three model calls to store a single memory, roughly $1,000 to $3,000 of inference a month at 100,000 memories before any retrieval, and consolidation done naively compares every memory with every other, according to the build versus buy page cited above. A memory API moves that work and its failure modes to a vendor, in exchange for a dependency and less control over extraction. Build when memory is the product, or your data cannot leave your infrastructure and you have people to run the write path for years. Buy when memory is a feature of a product whose value sits elsewhere.

What database should you use for AI agent memory?

For most teams, Postgres with pgvector: memories sit beside the relational data they describe, partitions and row-level security handle tenants, one transaction can retire an old statement and insert its replacement, and you already know how to back it up. Our memory versus vector RAG page reaches the same default. Move the vector index to Pinecone or Qdrant when scale, filter-heavy queries or native hybrid retrieval make Postgres the bottleneck, and keep fast working state such as the current session, locks and rate limits in a store built for it, such as Redis, with expiry. A graph database earns its place when the agent needs multi-hop questions across people and companies. Whichever store holds the bytes, the memory logic sits above it, and that is the part to budget for.

Where Maximem Synap fits

Maximem Synap is the write path as a hosted service. It sits beside your model rather than in front of it: you send conversations as they happen, and before the agent replies you ask what is known about this person and get back a short ranked set of statements ready for the prompt. Synap reads each conversation first and decides what is worth keeping, then stores self-contained statements such as facts, preferences, episodes and temporal events, each with a confidence score, a scope and the date the thing described actually happened, instead of chunks of transcript. Conscious & Lossless Forgetting retires a statement that stopped being true without letting a poorer statement overwrite a more detailed one, and Provenance records what each memory replaced and why. Scope is assigned at write time across client, customer, user and conversation, with nested hierarchies to any depth, and a read sees its own level and those above it, never below and never sideways. Erasure is per person and previewed before it runs, and payment card numbers, security codes, passwords and API keys are never stored at all. It is not a vector database you operate, and it ships Python and TypeScript SDKs, a REST API and adapters for 23 agent frameworks.

Frequently asked questions

Can a vector database be agent memory?

It can be the storage and retrieval layer of agent memory, not the whole of it. A vector database finds the stored records closest to a query inside a filter; memory also needs to decide what to keep, update facts that change, assign each statement to the right tenant or person, erase a person everywhere their data was copied, and know when a fact stopped being true.

Is pgvector enough for AI agent memory?

pgvector is enough as the store, and for teams already on Postgres it is usually the best one. Turn on iterative index scans so selective per-user filters return enough rows, and isolate tenants with partitioning or row-level security. Extraction, updates, scoping and erasure still have to be written on top.

Does deleting a vector delete the memory?

No. Deleting by ID or by filter removes those vectors, but summaries, facts extracted into other records, cached prompts, secondary indexes and backups can still hold the same information, and research has shown text can be reconstructed from embeddings. Erasure needs lineage from every derived record back to its source.

Do Pinecone or Qdrant support TTL for agent memory?

Neither documents a per-record time-to-live as of 27 October 2026, and core Postgres has none either. Expiry means storing a datetime with each record and running a scheduled delete by filter. Qdrant's datetime decay can rank older points lower, but it does not remove them or mark them invalid.

Do I need a memory framework if I already use Pinecone or Qdrant?

Not for shared documents, stateless sessions or a single-user prototype. You need one, built or bought, once facts about a person change over time, more than one tenant shares the system, or users can ask to be forgotten.

Sources

Every page below was read on 27 October 2026.

• pgvector README, v0.8.7
• Row Security Policies, PostgreSQL 18 documentation
• Implement multitenancy using namespaces, Pinecone docs
• Delete data, Pinecone docs
• Hybrid search, Pinecone docs
• Multitenancy with Qdrant, Qdrant docs
• Hybrid and multi-stage queries, Qdrant docs
• Points, Qdrant docs
• Reachy Remembers, Sasha Denisov and Chadha Sridi, Qdrant blog, 14 October 2026
• Text Embeddings Reveal (Almost) As Much As Text, Morris, Kuleshov, Shmatikov and Rush, arXiv 2310.06816
• Article 17 GDPR, Right to erasure
• Build versus buy agent memory, Maximem