You have 8,000 support PDFs and a model that has never seen a single one of them. Fine tuning would bake the content into weights and go stale the next time a policy changes. RAG takes the other road: the model reads the relevant passages at question time, pulled fresh from a store you control. Bedrock Knowledge Bases is the managed version of that road, and it hides most of the plumbing.
It hides the plumbing so well that the first real surprise is the invoice. A knowledge base nobody queried all month can still cost more than the model calls on a busy one. This part walks the whole thing from the ground up: what a knowledge base is, how ingestion turns files into vectors, the three choices you actually make, how to query it in code, and where the money goes. I define each term the first time it shows up, so you can read this cold.
What a knowledge base does that a bare model cannot
A foundation model knows only what it saw in training. Ask it about your refund policy and it guesses, which is where the hallucinations from earlier in the series come from. Retrieval augmented generation fixes that by putting the answer in front of the model at request time. You retrieve the passages that likely hold the answer, paste them into the prompt as context, and ask the model to answer from that context. The model stops guessing because the facts are right there in the window.
The hard part is the retrieve step. To find the right passages you cannot rely on keyword search alone, because a question rarely uses the same words as the document. So each passage gets turned into an embedding, a list of numbers that captures its meaning, and the question gets turned into an embedding too. Passages whose vectors sit closest to the question vector are the ones you pull back. That similarity math needs a home built for it, a vector store, which is a database that indexes vectors and answers nearest neighbor queries fast.
A Bedrock knowledge base is the managed assembly of those pieces. You point it at documents in Amazon S3 or another source, and it handles the ingestion pipeline: reading the files, splitting them into chunks, calling an embeddings model on each chunk, and writing the vectors into a store. At query time it embeds the question, runs the nearest neighbor search, and hands the top passages back to you or straight to a model. You still make three decisions that shape quality and cost. Everything else Bedrock runs.
How ingestion turns your documents into vectors
Ingestion is the job that reads your source and fills the vector store. You attach a data source to the knowledge base, most often an S3 bucket, though Bedrock also connects to a web crawler, Confluence, Salesforce, and SharePoint. When you start a sync, Bedrock reads each file, parses it into text, applies your chunking strategy to break the text into passages, calls the embeddings model on every chunk, and writes each vector plus its source metadata into the store. Change a file and the next sync re-embeds only what moved, so you pay the embedding cost again for the changed chunks, not the whole corpus.
The embeddings model matters because its output dimension and its price are fixed for the life of the index. Amazon Titan Text Embeddings V2 is the current default and produces vectors you can configure to 1024, 512, or 256 dimensions, trading a little recall for a smaller, cheaper index. Cohere Embed is the other common pick, strong on multilingual corpora. Whichever you choose, the query path has to use the same model, because a question embedded by one model does not sit in the same space as chunks embedded by another. Switch the embeddings model later and you re-embed everything from scratch.
One newer feature earns its place on messy corpora. Advanced parsing lets Bedrock run a foundation model over PDFs that carry tables and images, so a number trapped inside a table becomes retrievable text rather than a blur the parser skips. It costs extra per page and slows the sync, so I turn it on only for the documents that need it, not the whole bucket.
Picking a vector store
This is the choice with the biggest cost consequence, so treat it as a real decision, not a default you click through. Bedrock supports several vector stores, and the quick create flow silently provisions Amazon OpenSearch Serverless for you. That is fine for scale and painful for a small or idle workload, because OpenSearch Serverless bills a capacity floor whether or not anyone queries it. The alternatives change that math.
| Vector store | Model | Low idle cost | When I reach for it |
|---|---|---|---|
| OpenSearch Serverless | Serverless, auto scaled | No, capacity floor | Large corpora, steady query volume, need scale |
| Aurora PostgreSQL, pgvector | Serverless v2, scales down | Yes, low when idle | Small to mid corpora, cost sensitive, already on Postgres |
| Neptune Analytics | Managed graph plus vector | No | GraphRAG, relationship aware retrieval |
| Pinecone | Third party SaaS | Plan dependent | You already run Pinecone |
| MongoDB Atlas | Third party SaaS | Plan dependent | Your data already lives in Atlas |
| Redis Enterprise Cloud | Third party SaaS | Plan dependent | Low latency need, already on Redis |
All of these answer nearest neighbor queries. They differ on who runs them, how they price idle capacity, and whether they add graph or keyword search alongside vectors.
My rule is short. If the corpus is large and queried steadily, OpenSearch Serverless earns its floor and gives you room to grow. If the corpus is small, or the traffic is spiky, or this is a proof of concept that might sit idle for weeks, Aurora Serverless v2 with pgvector scales its compute down when nothing is happening and saves you the fixed monthly bill. Do not accept the quick create default without asking which of those you are.
Chunking decides what retrieval can find
Chunking is how Bedrock cuts each document into the passages it embeds. Get it wrong and retrieval suffers no matter how good the model is, because the chunk is the unit that comes back. Too large and each passage carries noise that dilutes the match. Too small and the answer gets split across chunks so no single one is a clean hit. The strategy is set per data source, and changing it means a re-sync, so it is worth a minute of thought up front.
| Strategy | What it does | Best for |
|---|---|---|
| Fixed size | Split every N tokens with overlap | Uniform text, a quick start |
| No chunking | One chunk per file | Short files already sized right |
| Hierarchical | Retrieve small child, return larger parent | Long structured reports and manuals |
| Semantic | Split at meaning boundaries | Mixed topic documents |
| Custom, Lambda | Your own splitter runs in a function | Special formats, code, tables |
Fixed size is the honest default. Hierarchical and semantic buy accuracy on long or mixed documents at the cost of a slower, pricier ingest.
I start almost every knowledge base on fixed size chunking with a moderate overlap, measure retrieval on a set of real questions, and only move to hierarchical when the documents are long and structured, like policy manuals where the answer needs its surrounding section for context. Hierarchical retrieves a tight child chunk for the match, then hands the model the parent chunk so it has the full paragraph around the hit. Semantic chunking helps when one file mixes unrelated topics. Both cost more to ingest, so I reach for them after fixed size proves not good enough, not before.
Query the knowledge base two ways
Once the store is filled, you read it with one of two APIs on the bedrock-agent-runtime client. RetrieveAndGenerate is the managed path: you pass a question and a generation model, and Bedrock embeds the question, retrieves the top passages, builds the prompt, calls the model, and returns a written answer with citations pointing at the source documents. It is the fastest way to a working answer, and where I start every project.
import boto3
agent_rt = boto3.client('bedrock-agent-runtime', region_name='us-east-1')
resp = agent_rt.retrieve_and_generate(
input={'text': 'What is our refund window for damaged items?'},
retrieveAndGenerateConfiguration={
'type': 'KNOWLEDGE_BASE',
'knowledgeBaseConfiguration': {
'knowledgeBaseId': 'KB1234ABCD',
'modelArn': 'arn:aws:bedrock:us-east-1::foundation-model/amazon.nova-pro-v1:0',
},
},
)
print(resp['output']['text'])
for c in resp['citations']:
for ref in c['retrievedReferences']:
print(ref['location'])ResourceNotFoundException on the knowledge base id means the id is wrong or lives in another Region. A ValidationException naming the model ARN means that generation model is not enabled in this Region, so request model access first, then retry.The Retrieve API is the lower level door. It returns the ranked passages and their similarity scores and stops there, leaving the prompt building and the model call to you. That is what you want once you need hybrid search, a reranker, or custom prompt assembly, or when you want to feed the passages into a Converse call you already control from Part 11. In production I usually graduate to Retrieve so the generation step is mine to tune, but only after RetrieveAndGenerate proved the retrieval is good.
r = agent_rt.retrieve(
knowledgeBaseId='KB1234ABCD',
retrievalQuery={'text': 'refund window for damaged items'},
retrievalConfiguration={'vectorSearchConfiguration': {'numberOfResults': 5}},
)
for item in r['retrievalResults']:
print(round(item['score'], 3), item['content']['text'][:80])Where the cost actually comes from
Three lines make up a knowledge base bill: embedding tokens at ingest, the vector store, and generation tokens at query time. People budget for the tokens and forget the vector store, which is exactly backward for most workloads. The token lines are cheap. The store is a fixed monthly floor that runs whether traffic is heavy or zero.
Worked example
A knowledge base over 8,000 documents, about 20 million tokens after chunking, embedded with Titan Text Embeddings V2 at roughly 0.20 dollars per million tokens [VERIFY current rate], costs about 4 dollars to ingest, and again only for changed chunks on later syncs. The vector store is OpenSearch Serverless at a redundant production floor near 350 dollars a month, fixed. Serve 10,000 RetrieveAndGenerate calls in the month, each about 2,500 input and 400 output tokens through Nova Pro, and generation runs near 33 dollars.
Month one lands around 387 dollars, and about 350 of it is the store nobody queries when the app is idle. Ten times the traffic adds roughly 300 dollars of generation, not another store. The floor is the number to design around, not the tokens.
Plot the total against query volume and the shape of the problem is obvious. The line barely moves from 1,000 queries to 10,000, because the floor drowns the token cost, and only starts to climb once generation catches up near 100,000 queries a month. For a busy app the floor is a rounding error. For a quiet internal tool it is almost the entire bill, which is the case for a cheaper store that scales down.
What breaks in production
Three failure modes show up again and again. The first is stale answers: someone updates a document in S3 and forgets that a knowledge base only knows what the last sync ingested, so the model keeps citing the old version until you re-sync. Automate the sync on document change rather than trusting a human to remember. The second is empty or wrong retrieval that looks like a model problem but is a chunking problem, which the Retrieve scores will tell you if you look at them instead of only reading the generated answer.
The third is the money one, and it is the reason I lead with cost. When you delete a knowledge base that used the quick create flow, Bedrock does not delete the OpenSearch Serverless collection it created for you. The collection keeps billing its floor, quietly, with no knowledge base attached to explain the charge. I have seen the line item outlive the project by months.
Before you change production
Creating a knowledge base with the quick create flow provisions a billable vector store immediately. Before you run it in a shared account, confirm which store it will create, set a budget alarm on the collection, and decide the teardown steps in advance so a proof of concept does not leave a floor charge behind. This is general guidance, not a substitute for your own cost and change review. The bill that catches teams out is an OpenSearch Serverless collection left running from a proof of concept nobody tore down. It carries a minimum OCU charge whether or not a single query hits it, so it sits on the invoice for months as a line item no one recognizes.
My take
For anything that is not clearly a high traffic production system on day one, I start on Aurora Serverless v2 with pgvector, not the quick create OpenSearch collection. The compute scales down when the app is quiet, which matches how a new knowledge base actually behaves in its first months, mostly idle while you tune it. I move to OpenSearch Serverless when query volume is steady enough that its floor is cheaper than the alternative, and when I need its scale. Choosing the store for the workload you have, not the one the wizard defaults to, is the single decision that most often keeps this bill sane.
Aurora for small corpora, OpenSearch Serverless when you need the scale
Here is the posture I ship. Use RetrieveAndGenerate to get a working answer and prove the retrieval is good, then graduate to the Retrieve API when you need to own the generation step. Start on fixed size chunking, measure on real questions, and only pay for hierarchical or semantic where long or mixed documents demand it. Keep the source bucket, the embeddings model, and the query path aligned, because a mismatch there fails quietly. And pick the vector store for your actual traffic: Aurora Serverless v2 with pgvector when the corpus is small or the app will sit idle, OpenSearch Serverless when the volume is steady and the scale is real. Whichever you choose, put a budget alarm on the store the day you create it, and write the teardown steps down so nothing outlives the project on the bill.
Next in the series I take this retrieval layer and hand it to a model that can act, with Bedrock Agents, where a knowledge base becomes one tool an agent calls among several. If you also build on other clouds, the Azure Gen AI guide covers the same RAG pattern through Azure AI Search. Before you read on, open the OpenSearch Serverless console in your main Region and check for a collection with no knowledge base attached. If you find one, you have found a charge you can delete today.
Related reading: the vendor-neutral GenAI guide explains retrieval augmented generation at the concept level, without the Bedrock specifics.
References
• Retrieve data and generate responses with Amazon Bedrock Knowledge Bases
• How content chunking works for knowledge bases
• Amazon OpenSearch Service pricing
• Amazon Bedrock pricing
About The Author
DrJha