A retrieval system fails at the corpus long before it fails at the model. I once watched a team swap three foundation models trying to fix wrong answers from a support assistant, when the real fault sat in the knowledge base: half sentences cut mid clause, page headers glued onto the wrong paragraph, tables flattened into word soup. The model was answering faithfully from garbage. No bigger model fixes that.
Part 12 built a Bedrock Knowledge Base end to end and showed managed retrieval augmented generation working. This part is about what you feed it. Four choices, parsing, chunking, embeddings, and where the vectors live, decide whether a query returns the exact passage that answers it or a confident near miss. Get that pipeline right and a mid tier model on a clean index beats a frontier model on a messy one.
What grounding data means on Bedrock
Grounding means giving the model your own facts at query time so it answers from your data instead of from whatever it absorbed in training. On Bedrock the mechanism is the Knowledge Base. You point it at documents in Amazon S3, it converts them into vectors and stores them, and at query time it embeds the question, retrieves the closest chunks, and hands them to the model as context. Data prep is everything that happens before that query: getting the source into clean text, splitting it sensibly, embedding it, and indexing it.
Each stage can quietly wreck retrieval, and the damage compounds. A parser that drops a table means the answer never enters the index. A chunk that splits a procedure in half means retrieval returns step three without step two. An embedding model that does not understand your domain scores the wrong passage as closest. By the time the model sees the context, the mistake is already made, and no prompt engineering recovers a fact that was never retrieved. That is why I treat the four stages below as the real product, and the model as the easy part.
Turn messy documents into clean text
Parsing is pulling readable text out of a source file. Plain text, HTML, and Markdown are easy, and the default parser handles them well. PDFs are where it breaks. A two column layout gets read straight across, so the end of a left column runs into the start of a right one. A table becomes a run of numbers with no row or column sense. A chart or a scanned page has no text layer at all, so it comes out empty. If the meaning of your document lives in its tables and figures, the default parser throws that meaning away.
Bedrock gives you two heavier options for exactly those documents. Foundation model parsing sends each page to a model such as Claude that reads the layout, so tables keep their structure and figures get described in text. It currently applies to PDF files and costs extra, because you pay for the tokens the model spends reading each page. Bedrock Data Automation is the other path, built for multimodal extraction across document types. My rule is narrow: default parser for anything already close to plain text, and foundation model parsing or Data Automation only for the subset of documents where a table or a figure carries the answer. Turning smart parsing on for a corpus of plain support articles just burns tokens for no accuracy gain.
Which chunking strategy fits your documents
Chunking splits the parsed text into the pieces that get embedded and retrieved. It is the single highest leverage choice in this pipeline, and the one teams skip. Make chunks too large and each one carries the answer buried in a page of unrelated text, so the embedding is blurred and retrieval gets vague. Make them too small and you slice a procedure or a definition in half, so retrieval returns a fragment that reads like an answer but is missing the part that mattered. The right size depends on how your documents are written, which is why Bedrock offers several strategies rather than one.
| Strategy | What it does | Best for |
|---|---|---|
| Default | About 300 tokens, respects sentence ends | A sane first pass on most text |
| Fixed size | You set max tokens plus an overlap percent | Uniform docs, tuning chunk size by hand |
| Hierarchical | Parents near 1,500 tokens over 300 token children, retrieve child, return parent | Manuals, legal, nested structure |
| Semantic | Splits where meaning shifts, using embedding similarity | Prose with topic changes, slower to ingest |
| None | One file becomes one chunk | Already small records, FAQ rows |
| Custom | Your own logic in a Lambda transform | Odd formats the built ins mishandle |
Hierarchical chunking is worth understanding because it solves the size tension directly. It builds two layers: small child chunks near 300 tokens that make retrieval precise, and larger parent chunks near 1,500 tokens that carry surrounding context. At query time Bedrock matches on the sharp child, then hands the model the broader parent, so you get precise matching and full context at once. For a technical manual where a step means nothing without the section around it, that behavior is the difference between a usable answer and a fragment.
Picking an embeddings model and vector dimension
An embedding is a list of numbers that captures the meaning of a chunk, arranged so that text with similar meaning lands at nearby points. That is what makes semantic search work: the query becomes a vector, and retrieval finds the chunk vectors closest to it. On Bedrock the common choice is Amazon Titan Text Embeddings V2, which takes up to 8,192 input tokens, covers more than 100 languages, and, unusually, lets you choose the output dimension. Cohere Embed English and Multilingual are the other well used options, and Titan Multimodal Embeddings covers image plus text when your corpus is not all words.
The dimension choice on Titan V2 is a real cost lever. The default is 1,024 numbers per vector. Drop to 512 and AWS reports you keep about 99 percent of the 1,024 dimension accuracy while halving the storage each vector needs. Drop to 256 and you hold about 97 percent. Fewer numbers per vector means less storage, faster nearest neighbor search, and a smaller index bill, and for most corpora the accuracy you give up at 512 is inside the noise. I start at 512 and only move to 1,024 if a measured retrieval miss actually improves when I do. A multilingual corpus is the main reason I reach past Titan to Cohere Multilingual, though Titan V2 already spans many languages.
Worked example
Take a 5,000 page PDF corpus at roughly 500 tokens a page, so about 2.5 million tokens of text. Chunked at the 300 token default, that is roughly 8,300 chunks, which means 8,300 vectors to store and 2.5 million tokens to embed once. The embedding run bills at the Titan V2 per 1,000 token rate [VERIFY current rate], a one time cost you pay again only when you re-sync changed files. Storage is where the ongoing bill sits, and 8,300 vectors is small, so the number that decides your monthly cost is not the vectors, it is which store you put them in. At 512 dimensions those vectors need half the bytes of 1,024, but the store minimum matters far more than the byte count at this scale, which is the next section.
Where your vectors live
The vector store is where the embedded chunks sit and where retrieval runs its nearest neighbor search. Bedrock Knowledge Bases can wire up to several, and the choice sets both your latency and your monthly floor. Amazon S3 Vectors is the newest and the cheapest: Bedrock creates an S3 vector bucket and index for you, and because it is storage first, the floor is a few dollars a month rather than a running cluster. OpenSearch Serverless is the fast, high query rate option, but it carries a much higher minimum whether you store a thousand vectors or a million. OpenSearch managed clusters, Amazon Aurora PostgreSQL with the pgvector extension, and Amazon Neptune Analytics round out the AWS native list, and Pinecone, MongoDB Atlas, and Redis Enterprise are supported third parties.
| Store | Strength | Best when |
|---|---|---|
| S3 Vectors | Lowest floor cost, storage first | Cost sensitive, moderate query volume |
| OpenSearch Serverless | Low latency, high query rate, hybrid search | Heavy traffic, latency budget is tight |
| Aurora PostgreSQL pgvector | Reuses a database you already run | Under about 10 million vectors, Aurora in stack |
| OpenSearch managed | Full control, binary vector support | You already operate OpenSearch |
My default now is S3 Vectors for anything cost sensitive at moderate query volume, which covers most internal knowledge bases. I move to OpenSearch Serverless when latency is tight or query rate is high enough that its speed and hybrid search earn the floor, and to Aurora pgvector when the team already runs Aurora and the corpus sits under roughly 10 million vectors, since reusing a database beats standing up a new service. Do not reach for OpenSearch Serverless on day one for a small internal tool. You will pay a cluster sized bill to search a few thousand vectors.
Keeping the knowledge base fresh
A knowledge base is only as good as its last sync. When source documents change, you run an ingestion job, and Bedrock re-parses, re-chunks, and re-embeds the files that changed, updating the index. Incremental sync only touches changed files, so a nightly job on a large corpus is cheap as long as most of it is stable. The second tool that earns its keep is metadata. Attach a metadata file alongside each document, and queries can filter on attributes like department, product, or date before the semantic search runs, which stops the retriever from pulling a right sounding chunk out of the wrong context.
Here is the operational shape of it: attach a chunking configuration to a data source, then start an ingestion job so the changed files flow through parse, chunk, and embed. The example uses the bedrock-agent API to set hierarchical chunking and kick off a sync.
import boto3
agent = boto3.client("bedrock-agent")
# attach hierarchical chunking to an S3 data source
ds = agent.create_data_source(
knowledgeBaseId="KB1234567",
name="support-docs",
dataSourceConfiguration={
"type": "S3",
"s3Configuration": {"bucketArn": "arn:aws:s3:::my-support-corpus"},
},
vectorIngestionConfiguration={
"chunkingConfiguration": {
"chunkingStrategy": "HIERARCHICAL",
"hierarchicalChunkingConfiguration": {
"levelConfigurations": [{"maxTokens": 1500}, {"maxTokens": 300}],
"overlapTokens": 60,
},
}
},
)
ds_id = ds["dataSource"]["dataSourceId"]
# re-parse, re-chunk, and re-embed the changed files
job = agent.start_ingestion_job(knowledgeBaseId="KB1234567", dataSourceId=ds_id)
print(job["ingestionJob"]["status"])Expected output: create_data_source returns the new data source id, and start_ingestion_job prints a status of STARTING, which moves through IN_PROGRESS to COMPLETE as the documents are embedded.
Failure mode: a ValidationException if the level token sizes are out of order, since the parent must be larger than the child, or if a chunk exceeds the embedding model input limit. On a very large corpus you can hit a ThrottlingException as the embedding calls are rate limited, so ingest in stages. Confirm the exact field names against the current bedrock-agent API before you script this. [VERIFY API field names]
What data prep costs
Four meters run in this pipeline, and they do not all matter at the same scale. Embedding tokens are a one time cost at ingest, repeated only when you re-sync, and they dominate on a huge first load of a large archive. Foundation model parsing tokens stack on top of that, but only for the documents where you enabled it. The vector store floor is a monthly cost that runs whether or not anyone queries, and at low volume it is the biggest line on the bill, which is why the store choice from the last section is really a cost decision. Query time embedding of each question is tiny and rarely worth thinking about.
The practical read is simple. On a small internal knowledge base, the store minimum is your whole cost, so pick S3 Vectors and stop optimizing embeddings. On a massive one time ingest, the embedding token bill is the number to model, so chunk and parse deliberately and use 512 dimensions to keep the index lean. Enable foundation model parsing only on the document subset that needs it, because paying a model to read every plain text page is the quiet waste I see most often.
Clean the corpus first, then tune retrieval
Where I start on a new knowledge base is deliberately boring: the default parser, fixed chunking near 300 to 512 tokens, Titan V2 at 512 dimensions, and S3 Vectors underneath. That baseline is cheap, fast to stand up, and good enough to measure. Then I change one variable at a time and re-measure, because if you switch parser, chunking, and store together you learn nothing about which one moved the needle. Retrieval quality is an experiment, not a config you get right on the first try.
When not to reach for the heavy options: skip semantic chunking, foundation model parsing, and OpenSearch Serverless on day one. Each adds cost or a failure mode you cannot yet attribute, and you should only add them once a measured miss tells you the baseline is the reason. What to validate first is the retrieval itself, not the final answer. Pull 20 to 30 real questions, run them, and read the chunks the retriever returned, not just what the model said. Nine times out of ten the failure is visible right there: a chunk cut in half points at chunking, an empty table points at parsing, a plausible but off topic passage points at embeddings or missing metadata.
Part 21 moves up the stack to multi-agent collaboration on Bedrock, where several agents share the grounded knowledge you just built, so the quality of this index sets the ceiling for everything above it. If you work across clouds, the same data prep decisions show up as Azure AI Search and its RAG indexing, and the retrieval concepts themselves are laid out vendor neutral in the GenAI series on why data beats model size. This week, take 10 wrong answers from your knowledge base, open the retrieved chunks behind each one, and label the fault as parse, chunk, or embed. That one exercise will tell you where to spend, and it costs nothing but an afternoon.
References
• How content chunking works for knowledge bases
• Advanced parsing, chunking, and query reformulation in Bedrock Knowledge Bases
• Amazon Titan Text Embeddings models
• Using S3 Vectors with Amazon Bedrock Knowledge Bases
About The Author
DrJha