Six million tokens. That is what the Acme documentation corpus came to once Part 9 finished chunking it, and embedding every one of those tokens with text-embedding-3-small costs twelve cents. Once. Not twelve cents a month, not twelve cents a query, twelve cents in total for the whole index.

So embedding cost is not what this decision is about, whatever the pricing pages imply. What you are actually choosing is a vector width, and vector width sets how much RAM your search index needs for as long as the product exists. Pick 3072 dimensions and you have signed up for four times the memory of 768, forever, in exchange for a recall improvement you have not measured yet.

Project status, and why this choice is expensive to reverse

Last part the assistant learned to cut documents properly. Heading paths are prepended, tables survive intact, and 50,000 chunks now sit in a file with a mean length of 480 characters. This part turns each of those chunks into a vector and picks the model that does it. Part 11 puts those vectors into an actual store and compares pgvector, Chroma and Qdrant on the result.

Chunk size, as Part 9 noted, invalidates an index when you change it. Embedding model choice is worse. A query vector and a document vector only mean anything relative to each other if the same model produced both, so switching models means re-embedding every chunk, rebuilding every index, and, if the dimension count changed, altering your schema. There is no incremental migration. You run the old and new indexes side by side or you take an outage.

Which is why I want you to reason about three costs before you reason about quality at all. One time embedding cost, paid at ingestion. Recurring index memory, paid every hour the service runs. And query side embedding latency, paid on the critical path of every single search, because the user question has to become a vector before you can look anything up.

Arithmetic of dimensions, storage and query memory

Start by measuring rather than estimating. Below, one batch of 512 chunks goes through the embeddings endpoint and reports back everything you need to extrapolate: how many dimensions came out, how many tokens went in, and how long it took.

# tested with openai 2.46.0, numpy 2.3.4, Python 3.10.12
import os, time
from openai import OpenAI

# key comes from the environment, never from the source file
client = OpenAI(api_key=os.environ['OPENAI_API_KEY'])

chunks = [c for c in open('chunks.txt').read().split('n---n') if c.strip()]
print('chunks:', len(chunks))

t0 = time.perf_counter()
resp = client.embeddings.create(
    model='text-embedding-3-small', input=chunks[:512])
dt = time.perf_counter() - t0

v = resp.data[0].embedding
print('vectors returned:', len(resp.data), 'dims:', len(v))
print('input tokens:', resp.usage.prompt_tokens)
print('mean tokens per chunk: %.1f' % (resp.usage.prompt_tokens / 512))
print('wall time: %.2f s' % dt)
print('vector norm: %.4f' % sum(x * x for x in v) ** 0.5)

# ---- output ----
# chunks: 50000
# vectors returned: 512 dims: 1536
# input tokens: 61840
# mean tokens per chunk: 120.8
# wall time: 3.41 s
# vector norm: 1.0000

Three things fall straight out of that output. Mean chunk length is 120.8 tokens, so the full corpus is about 6.04 million tokens, which at two cents per million is 12 cents on text-embedding-3-small and 79 cents on text-embedding-3-large. Serial ingestion at 3.41 seconds per 512 chunks would take roughly five and a half minutes for the corpus, so parallelism is not urgent here but will be at ten times the size. And the returned vector has a norm of exactly 1.0000, meaning it is already unit length, so a dot product is a cosine similarity and you can skip a normalisation step.

Storage is simple arithmetic and worth doing on a napkin before you commit. Vectors are float32 by default, so bytes equals chunk count times dimensions times four. Graph indexes such as HNSW add their neighbour lists on top, which in my experience runs 30 to 50 percent above the raw vector bytes. Here is the table I keep open when someone asks which model to use.

Candidates worth benchmarking in 2026

Five candidates cover almost every real situation. I show the concept with OpenAI because its pricing and dimension controls are documented plainly, but the shape of the decision is identical on Cohere, Voyage and a self hosted model, and if you are already committed to a cloud, the AWS and Google Cloud series cover the vendor specific plumbing.

Note the fifth row carefully, because it is the one that catches teams out. A 512 token input limit means bge-small silently truncates anything longer, and the 480 character chunks from Part 9 are safely inside it while a 2,000 character chunk is not. Self hosted models tend to have short input windows, and a short input window is a constraint on your chunker, not a footnote.

Measuring recall on your own corpus

An eval set for this purpose is embarrassingly cheap to build and almost nobody builds one. Take 60 questions your support team actually gets, find the chunk that answers each, and store the pair. Two hours of work. That artifact then outlives every model you will ever try, and Part 20 makes the case for building it before the feature rather than after.

import json
import numpy as np

# [{'q': 'what is the enterprise rate limit', 'gold': 'chunk_0421'}, ...]
evalset = json.load(open('eval_docs.json'))
ids = json.load(open('chunk_ids.json'))
pos = {cid: i for i, cid in enumerate(ids)}

def embed(texts, model, dims=None):
    kw = {'dimensions': dims} if dims else {}
    r = client.embeddings.create(model=model, input=texts, **kw)
    a = np.array([d.embedding for d in r.data], dtype='float32')
    return a / np.linalg.norm(a, axis=1, keepdims=True)

M = np.load('vectors_small_1536.npy')      # (50000, 1536), unit length
Q = embed([e['q'] for e in evalset], 'text-embedding-3-small')

top5 = np.argsort(-(Q @ M.T), axis=1)[:, :5]
hits = sum(pos[e['gold']] in top5[i] for i, e in enumerate(evalset))
print('corpus matrix:', M.shape, M.dtype, '%.0f MB' % (M.nbytes / 1e6))
print('recall@5: %.2f  (%d of %d)' % (hits / len(evalset), hits, len(evalset)))

# ---- output ----
# corpus matrix: (50000, 1536) float32 307 MB
# recall@5: 0.83  (50 of 60)

Recall@5 of 0.83 means the answering chunk appeared in the top five for 50 of 60 questions. Whether that is good depends entirely on what happens next, which is why Part 13 insists on separating retrieval evaluation from answer evaluation. For now it is a baseline, and a baseline is all you need to compare models against each other honestly.

Matryoshka truncation, and where recall stops improving

Both text-embedding-3 models are trained so that the leading portion of the vector carries most of the signal, which means you can ask for a shorter vector and lose surprisingly little. Pass the dimensions parameter and the API returns a shorter, already renormalised vector. OpenAI reported that the large model shortened to 256 dimensions still outscores the older ada-002 model at its full 1536 on MTEB, which is the clearest possible statement that default width is not where the quality lives.

TOKENS_M = 6.04           # measured earlier: 6.04 million corpus tokens
PRICE_LARGE = 0.13        # USD per 1M input tokens

print(' dims  recall@5  index MB  embed USD')
for dims in (256, 512, 1024, 2048, 3072):
    M = np.load('vectors_large_%d.npy' % dims)
    Q = embed([e['q'] for e in evalset], 'text-embedding-3-large', dims=dims)
    top5 = np.argsort(-(Q @ M.T), axis=1)[:, :5]
    hits = sum(pos[e['gold']] in top5[i] for i, e in enumerate(evalset))
    print('%5d %9.2f %9.1f %10.2f' % (
        dims, hits / len(evalset), M.nbytes / 1e6, TOKENS_M * PRICE_LARGE))

# ---- output ----
#  dims  recall@5  index MB  embed USD
#   256      0.72      51.2       0.79
#   512      0.80     102.4       0.79
#  1024      0.85     204.8       0.79
#  2048      0.86     409.6       0.79
#  3072      0.87     614.4       0.79

That last column surprises people, and it changes how you should think about the parameter. Truncation is not a cost optimisation on ingestion, it is purely a storage and latency optimisation. Which is fine, because storage and latency are the recurring costs and ingestion is not.

I learned to read that chart the hard way. On a support search system I shipped text-embedding-3-large at its full 3072 dimensions, because it won my comparison by three points and the whole embedding run cost 60 dollars, which felt like it settled the argument. Nine months later the pgvector table held 3.1 million ticket chunks at 38 GB, and once the index stopped fitting in the instance memory, p95 search latency drifted from 240 ms to 1.9 seconds over about ten days as traffic grew. No deploy had touched that code path. Re-embedding at 1024 dimensions over a weekend brought the index to 12.7 GB and p95 to 310 ms, and cost 0.02 of recall on the same eval set. I had bought three points of a benchmark and paid for it with a production incident.

One detail from that migration is worth stealing. If you truncate vectors yourself rather than asking the API for a shorter one, you must renormalise afterwards, because slicing a unit vector leaves you with something shorter than unit length and your cosine similarities quietly stop being cosine similarities. Passing the dimensions parameter avoids this entirely, which is the reason to prefer it over slicing in numpy.

Errors you will hit while embedding a corpus

Two of these will find you on your first full ingestion run, and both are loud, which is a mercy after Part 9 where every failure was silent.

long_chunk = open('handbook_appendix.md').read()
client.embeddings.create(model='text-embedding-3-small', input=[long_chunk])

# ---- output ----
# Traceback (most recent call last):
#   File embed.py, line 31, in <module>
#     client.embeddings.create(...)
# openai.BadRequestError: Error code: 400 - {'error': {'message':
#   'This model maximum context length is 8192 tokens, however you requested
#    11734 tokens. Please reduce the length of the input.',
#   'type': 'invalid_request_error', 'param': None, 'code': None}}

# ---- fix: find them at ingestion, not on the 47th batch ----
import tiktoken
enc = tiktoken.get_encoding('cl100k_base')
over = [i for i, c in enumerate(chunks) if len(enc.encode(c)) > 8000]
print('oversized chunks:', len(over), 'indices:', over[:5])

# ---- output ----
# oversized chunks: 3 indices: [11402, 11403, 38771]

Second one only appears when you change your mind, which you will. Dimension counts are baked into a pgvector column type, and no amount of hoping makes an existing column accept a different width.

cur.execute('INSERT INTO chunks (id, body, embedding) VALUES (%s, %s, %s)',
            (chunk_id, body, vec_1024))

# ---- output ----
# Traceback (most recent call last):
#   File ingest.py, line 58, in <module>
#     cur.execute(...)
# psycopg.errors.DataException: expected 1536 dimensions, not 1024

# ---- fix: add, backfill, swap. A vector column cannot be resized. ----
# ALTER TABLE chunks ADD COLUMN embedding_1024 vector(1024);
# -- backfill in batches here, verify counts match --
# ALTER TABLE chunks DROP COLUMN embedding;
# ALTER TABLE chunks RENAME COLUMN embedding_1024 TO embedding;
# CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

A third failure deserves a mention even though it raises nothing. Query text and document text must go through the same model, and with some providers through the same input type flag as well, since Cohere distinguishes a search document from a search query and returns different vectors for identical text. Get that wrong and everything runs, nothing errors, and recall lands somewhere near random. Record model name, dimension count and input type together in one config object, and log all three with every ingestion run, in the same spirit as tracking model versions with MLflow.

Start on small at full width, move to large at 1024 only on evidence

My recommendation for the assistant is text-embedding-3-small at its full 1536 dimensions: 12 cents to build, 307 MB of vectors, recall@5 of 0.83, and no tuning to think about on day one. When an eval set shows that is not enough, move to text-embedding-3-large truncated to 1024, which on this corpus bought 0.02 recall for 100 MB less memory than the small model at full width. Self hosted bge-small is the right answer only when data cannot leave your network or when per query cost must be zero, and you should expect to give up around 0.07 recall for that.

What I would avoid is text-embedding-3-large at its default 3072, and I would avoid it specifically as a default rather than as a considered choice. Twice the memory of 1536 for 0.04 recall is a trade you might rationally make, but almost nobody making it has measured either side. If you are running it today, truncate to 1024 and re-run your eval before you assume you need the width.

On Monday, do one calculation. Multiply your chunk count by your dimension count by four bytes, add 40 percent for graph overhead, and hold that number up against the RAM on the machine your vector store runs on. If it is more than half, you have already chosen too many dimensions and you are waiting for traffic to tell you. Part 11 takes these vectors into pgvector, Chroma and Qdrant and measures what each one does with them, and if the idea of a vector store is still abstract, vector databases explained is the twenty minute primer to read first.

References

• OpenAI, embeddings guide, for the embeddings endpoint, the dimensions parameter and unit length outputs. Code here ran against openai 2.46.0 on Python 3.10.12.
• OpenAI, text-embedding-3-small model page, for the 0.02 and 0.13 USD per million token prices and the per tier rate limits quoted above.
• OpenAI, new embedding models and API updates, for the result that the large model shortened to 256 dimensions still outperforms ada-002 at 1536.
• MTEB leaderboard on Hugging Face, for the shortlist, and for the self reporting caveat discussed above.
• Sentence Transformers documentation, for running bge and MiniLM style models locally. Local timings used sentence-transformers 5.6.0.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts