Ten rows requested. Four rows returned. No exception, no warning, nothing in the Postgres log, and the query was an ordinary nearest neighbour search with one WHERE clause on a metadata column.

That single line of output is the most expensive thing I have learned about vector stores. Which of the three you pick matters far less than whether you understand what its defaults do the moment you add a filter, and all three behave differently at exactly that point. This part runs the same corpus through pgvector, Chroma and Qdrant, measures them, and names a pick.

Where the assistant stands, and what a vector store must now do

Last part the assistant picked an embedding model and produced a 50,000 row float32 array, 1536 wide, sitting in a numpy file next to a JSONL of chunk text and metadata. This part gives those vectors a home that survives a restart and answers a query in single digit milliseconds. Nothing about the retrieval quality changes here. What changes is whether the thing can be operated.

Four jobs define the requirement, and only the first one gets discussed in most tutorials. Approximate nearest neighbour search over 50,000 vectors is the easy job; every store here does it well. Filtering by metadata is the hard one, because our support assistant has to answer questions scoped to a product area, a document version, or a customer tier, and a filter interacts with an approximate index in ways that are not obvious. Incremental upsert and delete matter because product docs change weekly and a store that needs a full rebuild per edit is a store you will grow to resent. Durability across restarts is the fourth, and it quietly disqualifies more prototypes than anything else.

Everything measured below ran on one machine: 8 vCPU, 32 GB RAM, NVMe disk, Postgres 17 and Qdrant 1.18 in Docker, Chroma embedded in the Python process. Same 50,000 chunks, same 1536 dimension vectors, same 60 question evaluation set carried forward from Part 9. Numbers from a single box are not a benchmark of the projects; they are a benchmark of what a reader with one box will see, which is more useful at this stage.

Three stores, three different bets

Reading these as three interchangeable products is a mistake. Each one is a bet about where vector search belongs in your architecture, and that bet is what you inherit.

pgvector is a Postgres extension, so its bet is that vectors belong beside your relational data. You get transactions, joins against your existing tables, your existing backup and replication story, and your existing on call rotation. Chroma is a library first, running inside your Python process and writing to a local directory, betting that a single application does not need a separate database tier at all. Qdrant is a standalone service written in Rust, betting that search is its own concern with its own memory profile and deserves its own box.

Running the same corpus through each

All three scripts read the same two files: chunks.jsonl with id, doc_id, area and body, and chunk_vectors.npy holding a 50000 by 1536 float32 array in the same order. Credentials come from environment variables in every case, never from the file.

# tested with pgvector 0.8.2 on PostgreSQL 17, psycopg 3.2.9, numpy 2.3.4, Python 3.10.12
import os, json, time
import numpy as np, psycopg
from pgvector.psycopg import register_vector

conn = psycopg.connect(os.environ['PG_DSN'], autocommit=True)
conn.execute('CREATE EXTENSION IF NOT EXISTS vector')
register_vector(conn)

conn.execute('''CREATE TABLE chunks (
  id        bigint PRIMARY KEY,
  doc_id    text,
  area      text,
  body      text,
  embedding vector(1536))''')

rows = [json.loads(l) for l in open('chunks.jsonl')]
vecs = np.load('chunk_vectors.npy')
print('rows:', len(rows), 'vectors:', vecs.shape, vecs.dtype)

with conn.cursor().copy('COPY chunks (id, doc_id, area, body, embedding) '
                        'FROM STDIN WITH (FORMAT BINARY)') as cp:
    cp.set_types(['bigint', 'text', 'text', 'text', 'vector'])
    for r, v in zip(rows, vecs):
        cp.write_row([r['id'], r['doc_id'], r['area'], r['body'], v])

conn.execute("SET maintenance_work_mem = '4GB'")
t0 = time.perf_counter()
conn.execute('CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops) '
             'WITH (m = 16, ef_construction = 64)')
print('index build seconds:', round(time.perf_counter() - t0, 1))
print(conn.execute("SELECT pg_size_pretty(pg_total_relation_size('chunks'))").fetchone())

rows: 50000 vectors: (50000, 1536) float32
index build seconds: 252.4
('1084 MB',)

Chroma is the shortest script of the three, and that is genuinely its selling point. Note the explicit space setting. Leaving it out gives you squared L2 on embeddings that were trained for cosine, and no error tells you so.

# tested with chromadb 1.5.9, numpy 2.3.4, Python 3.10.12
import json, time
import numpy as np, chromadb

client = chromadb.PersistentClient(path='./chroma_docs')
col = client.create_collection(
    name='docs',
    configuration={'hnsw': {'space': 'cosine',
                            'ef_construction': 200,
                            'ef_search': 100,
                            'max_neighbors': 16}})

rows = [json.loads(l) for l in open('chunks.jsonl')]
vecs = np.load('chunk_vectors.npy')

t0 = time.perf_counter()
for i in range(0, len(rows), 2000):
    b = rows[i:i + 2000]
    col.add(ids=[str(r['id']) for r in b],
            embeddings=vecs[i:i + 2000].tolist(),
            documents=[r['body'] for r in b],
            metadatas=[{'doc_id': r['doc_id'], 'area': r['area']} for r in b])
print('count:', col.count(), 'load seconds:', round(time.perf_counter() - t0, 1))

q = np.load('query_vector.npy').tolist()
res = col.query(query_embeddings=[q], n_results=10, where={'area': 'billing'})
print('returned:', len(res['ids'][0]),
      'top distance:', round(res['distances'][0][0], 4))

count: 50000 load seconds: 168.3
returned: 10 top distance: 0.1832

Qdrant needs the most configuration and repays it. Two lines do most of the work: on_disk storage with int8 scalar quantization kept in RAM, which is what drops resident memory from about 610 MB to 290 MB, and an explicit payload index on the field you filter by. Skip that payload index and filtered queries fall back to a much slower path.

# tested with qdrant-client 1.18.0 against qdrant 1.18 in docker, Python 3.10.12
import os, json, time
import numpy as np
from qdrant_client import QdrantClient, models

client = QdrantClient(url=os.environ['QDRANT_URL'],
                      api_key=os.environ.get('QDRANT_API_KEY'))

client.create_collection(
    collection_name='docs',
    vectors_config=models.VectorParams(
        size=1536, distance=models.Distance.COSINE, on_disk=True),
    quantization_config=models.ScalarQuantization(
        scalar=models.ScalarQuantizationConfig(
            type=models.ScalarType.INT8, quantile=0.99, always_ram=True)),
    hnsw_config=models.HnswConfigDiff(m=16, ef_construct=200))

client.create_payload_index(
    collection_name='docs', field_name='area',
    field_schema=models.PayloadSchemaType.KEYWORD)

rows = [json.loads(l) for l in open('chunks.jsonl')]
vecs = np.load('chunk_vectors.npy')

t0 = time.perf_counter()
client.upload_collection(
    collection_name='docs',
    vectors=vecs,
    payload=[{'doc_id': r['doc_id'], 'area': r['area']} for r in rows],
    ids=[r['id'] for r in rows],
    batch_size=512)
print('load seconds:', round(time.perf_counter() - t0, 1))

q = np.load('query_vector.npy')
hits = client.query_points(
    collection_name='docs', query=q.tolist(), limit=10,
    query_filter=models.Filter(must=[models.FieldCondition(
        key='area', match=models.MatchValue(value='billing'))])).points
print('returned:', len(hits), 'top score:', round(hits[0].score, 4))

load seconds: 185.2
returned: 10 top score: 0.8214

# and the failure you will hit within an hour of switching embedding models:
>>> client.query_points(collection_name='docs',
...                     query=np.random.rand(768).tolist(), limit=5)
Traceback (most recent call last):
  ...
qdrant_client.http.exceptions.UnexpectedResponse: Unexpected Response: 400 (Bad Request)
Raw response content:
b'{"status":{"error":"Wrong input: Vector dimension error: expected dim: 1536, got 768"}}'

Metadata filtering, where pgvector quietly loses rows

Here is the war story. We shipped a product area filter on the assistant so that a question tagged billing only retrieved billing documentation, which is an obviously correct thing to do and took about forty minutes. Answer quality complaints started a week later, vague ones, the kind that are easy to attribute to the model rather than to retrieval. I spent two days tuning the prompt. Three weeks after the filter shipped I finally ran the evaluation set with the filter applied instead of without it, and recall@10 came back at 0.41 against 0.93 unfiltered. Same index, same embeddings, same questions. Only the WHERE clause differed.

Cause is documented and completely unintuitive. With an approximate index, pgvector scans the graph first and applies your WHERE clause to whatever came back. Default hnsw.ef_search is 40, so the scan surfaces roughly 40 candidates and then filters them. If your predicate matches ten percent of rows, about four survive. Asking for ten gets you four, and nothing anywhere says so.

sql = '''SELECT id, area, 1 - (embedding <=> %s) AS score
         FROM chunks
         WHERE area = 'billing'
         ORDER BY embedding <=> %s
         LIMIT 10'''
q = np.load('query_vector.npy')

print('default settings ->', len(conn.execute(sql, (q, q)).fetchall()), 'rows')

# available from pgvector 0.8.0: keep scanning until LIMIT is satisfied
conn.execute('SET hnsw.iterative_scan = strict_order')
conn.execute('SET hnsw.ef_search = 100')
print('iterative scan  ->', len(conn.execute(sql, (q, q)).fetchall()), 'rows')

default settings -> 4 rows
iterative scan  -> 10 rows

Memory, latency and cost on 50,000 chunks

Latency figures below are p50 and p95 over 2,000 filtered queries after a warm cache, each configured correctly rather than left on defaults, so pgvector here includes the iterative scan fix and pays for it.

Two things in that table deserve attention beyond the headline. First, a 3.8x memory gap between Qdrant with int8 and pgvector is not a rounding error at scale; multiply the corpus by ten and it becomes the difference between a 4 GB instance and a 16 GB one. Second, pgvector paid 33 ms of p95 latency to recover its recall, which is a real and reasonable trade, but it is a trade nobody tells you about because the broken configuration is the fast one. Benchmarks that compare unfiltered latency are comparing pgvector at 0.41 recall against Qdrant at 0.96.

Defaults that will bite you

Keep this table somewhere you will find it again. Every row cost me or a colleague at least half a day, and none of them announces itself as an error.

Migration cost, and what actually locks you in

Choosing wrong here is cheaper than it feels, and knowing that should make you decide faster rather than slower. Vectors are portable: a numpy array and a JSONL of metadata reload into any of these three in under four minutes for 50,000 chunks. Moving our assistant from Chroma to Qdrant took one afternoon, and most of that afternoon went on rewriting filter syntax, not on data movement.

What genuinely locks you in is the embedding model, because re-embedding is what costs real money and real time, and that decision was Part 10. Second is any query logic you have scattered through the codebase. Wrap retrieval behind one function that takes a query vector, a filter dictionary and a k, and returns chunk ids with scores. That interface is about fifteen lines per backend and it turns a rewrite into a swap. Chroma and Qdrant both offer an in memory or local mode, so your tests can run against the real client without a server, which is worth structuring for from the start.

Recommendation for a docs assistant at this scale

Qdrant is what I would run, and it is what our assistant runs now. Filtered retrieval is the normal case rather than the exception in any product with tenants, versions or product areas, and Qdrant is the only one of the three that treats a filter as part of the search rather than as a post processing step. Int8 quantization dropping resident memory to 290 MB means it fits on a 4 GB instance at 32 USD a month, roughly a third of what the equivalent Postgres costs. Price of admission is one more service to run.

pgvector is a defensible second choice under two conditions, both of which must hold: your corpus stays under roughly 200,000 chunks, and someone on the team will own the index configuration rather than trusting defaults. Its real advantage is one nobody else can match, which is joining retrieval results to your business tables inside a single transaction. That is worth a lot in an internal tool.

What I would avoid is inheriting single node Chroma as a production store. Not because it is bad, because it is not: it had the fastest load, the shortest script, and perfectly respectable filtered recall out of the box, and I would still start every prototype with it. Problem is that single node Chroma has no compression, no clustering, and a distance function you cannot change after creation, so the day the corpus doubles you are migrating anyway. Start there deliberately and leave on your own schedule, rather than discovering the ceiling during an incident.

One caveat worth stating plainly. Every recall number in this part came from dense vector search alone, and dense search misses exact identifiers, error codes and product names with reliability that will annoy you. Part 12 adds keyword search alongside the vectors and a reranking pass on top, which is where filtered recall of 0.96 becomes something you would put in front of customers. If retrieval augmented generation is still a fuzzy idea rather than a pipeline in your head, what RAG is is the twenty minute version, and monitoring models in production covers the drift habits that apply just as much to a retrieval index as to a classifier. Go and run that filtered evaluation this week.

References

• pgvector, open source vector similarity search for Postgres, v0.8.2 README, for HNSW parameters, dimension limits and iterative index scans.
• Configure Collections, Chroma documentation, for HNSW defaults and the space parameter.
• Qdrant Python client, v1.18.0, for create_collection, query_points and filter syntax.
• Quantization, Qdrant documentation, for scalar int8 configuration and its memory effect.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts