Most teams reach for fine-tuning too early. They have a prompt that is almost right, they add more examples to it, the prompt grows to 6,000 tokens, latency climbs, and someone says the word fine-tuning as if it were the obvious next move. Usually it is not. Fine-tuning is the tool you pick up after retrieval and a well-built prompt have stopped paying off, not before. This part is about that decision, and about the mechanics once you have made it.

Two ways to change a model’s weights

Everything up to now in this series put information in front of the model at inference time. A prompt carries instructions. A knowledge base retrieves passages and pastes them into the context window. The weights of the model never move. Customization is different in one specific way: it changes the weights, so the knowledge lives inside the model instead of being handed to it on every call.

Two methods sit under that idea. Fine-tuning, more precisely supervised fine-tuning, trains the model on labeled examples: an input and the output you want back. You are teaching a task by demonstration. Continued pre-training trains on unlabeled text, raw documents with no answer attached, so the model absorbs the vocabulary and patterns of a domain without being pointed at a specific task. Fine-tuning sharpens behavior. Continued pre-training broadens background. Most projects want the first one.

Bedrock does not retrain the whole model for you. It uses parameter-efficient fine-tuning, which adds a small matrix of new weights, an adapter, and trains only that. The base model stays frozen. This is cheaper, faster, and far less likely to wreck the model’s general ability than full fine-tuning would be. You do not choose this or configure it for Nova models; Bedrock applies it automatically.

When fine-tuning beats a bigger prompt

Fine-tuning earns its cost in a narrow, common shape: a high-volume, well-defined task where you can assemble clean labeled examples. Intent classification is the textbook case. So is forcing a consistent output format, teaching a house style or tone, or replacing an older machine-learning classifier with a small model that also carries world knowledge. In each of these the task is stable, the right answers are knowable, and you run it enough that shaving tokens and latency off every call adds up.

It is the wrong tool when the knowledge changes often, because retraining to update a fact is absurd next to editing a document in a knowledge base. It is wrong when you cannot produce quality labels, since a fine-tune on noisy data bakes the noise in. And it is wrong when a clearer prompt or a better retrieval step would have closed the gap, which is more often than people admit. Fine-tuning does not add facts you can look up. It changes behavior you can demonstrate.

The payoff, when the shape fits, is real. AWS reports fine-tuning Nova Micro on a standard intent benchmark lifting accuracy from 41.4 percent to 97 percent, while cutting the long prompt down to a short one and dropping latency with it. A model that internalized the task no longer needs the instructions re-read on every request.

Which models you can customize today

Not every model on Bedrock can be customized, and the ones that can differ in which method they accept. The Nova family is where the current story is. Nova Micro, Nova Lite, and Nova Pro accept supervised fine-tuning; Nova 2 Lite adds reinforcement fine-tuning on top; and Nova Premier serves as a teacher for distillation rather than a fine-tuning target. Reinforcement fine-tuning trains against a reward function, custom code or a model acting as judge, instead of fixed labels, and it suits tasks where you can score an answer more easily than you can write the perfect one.

Older models are still customizable but on tighter terms. Amazon Titan Text Express and Titan Text Lite take both fine-tuning and continued pre-training. Cohere Command and Command Light, and Meta Llama 2 in its 13B and 70B chat variants, take fine-tuning only. A few of these sit behind a preview or a single Region. The table below is the shape as of this writing; because AWS moves models in and out of these lists, confirm your exact model and Region in the docs before you plan around it.

Availability and Regions shift; verify against the Bedrock supported-models page before planning. [VERIFY]

What a training job actually does

A fine-tuning job is more ordinary than it sounds. You put training data in Amazon S3, you start a job that points at a base model and that data, Bedrock trains an adapter on managed infrastructure you never see, and it hands back a custom model. No cluster, no GPU wrangling, no distributed training to set up. Your job is the data and a few settings.

The data format is JSONL, one JSON object per line, each a training example. For a conversational Nova fine-tune, every line carries a system prompt, a user turn, and the assistant response you want. One rule catches almost everyone the first time: the system prompt in your training data must match the system prompt you send at inference. The model learns that prompt as the trigger for its trained behavior, so if you drop it or reword it in production, the fine-tune underperforms and you will not get an error telling you why.

{"schemaVersion": "bedrock-conversation-2024",
 "system": [{"text": "Classify the airline query intent. Reply with one label only."}],
 "messages": [
   {"role": "user", "content": [{"text": "show me morning flights from boston to philadelphia"}]},
   {"role": "assistant", "content": [{"text": "flight"}]}
 ]}

One training line. In a real file this is a single physical row; it wraps here only for reading.

You kick the job off with three settings that matter. epochCount is how many full passes the model makes over your data, Nova allows 1 to 5 with a default of 2. learningRateMultiplier controls how hard each correction pulls the weights. learningRateWarmupSteps ramps the rate up gently at the start to avoid early instability. Here is the job in boto3.

import boto3

bedrock = boto3.client('bedrock', region_name='us-east-1')

job = bedrock.create_model_customization_job(
    jobName='nova-micro-intent-v1',
    customModelName='nova-micro-intent',
    roleArn='arn:aws:iam::123456789012:role/BedrockCustomizationRole',
    baseModelIdentifier='amazon.nova-micro-v1:0',
    customizationType='FINE_TUNING',
    trainingDataConfig={'s3Uri': 's3://my-bucket/training-data/intent.jsonl'},
    outputDataConfig={'s3Uri': 's3://my-bucket/output/'},
    hyperParameters={'epochCount': '3',
                     'learningRateMultiplier': '0.00001',
                     'learningRateWarmupSteps': '10'},
)
print(job['jobArn'])

Reading the loss curve

Bedrock writes training metrics and a loss curve to your output bucket and plots the curve in the console. Loss is a single number for how wrong the model’s predictions are on the training data, and it should fall as training proceeds. One number tells you most of what you need. A smooth downward trend that flattens near the end is convergence, the model has learned what the data has to teach.

The failure shapes are readable too. A curve that oscillates, swinging up and down instead of settling, means the learning rate is too high; halve learningRateMultiplier and rerun. A curve that barely declines means the rate is too low or the data is thin; try doubling the rate. A curve that flattens early, well above a good accuracy, usually wants another epoch or two. Watch for the training loss and a validation loss drifting apart, the training loss dropping while validation stalls or climbs, which is overfitting: the model is memorizing your examples rather than learning the task.

What customization costs, and where the bill hides

Training is cheap and the number surprises people. AWS trained a Nova Micro intent classifier on roughly 4,978 examples, three epochs, about 1.75 million tokens, in about an hour and a half for 2.18 dollars, plus 1.75 dollars a month to store the model. Training bills on tokens processed, which is examples times epochs, so a bigger dataset or more epochs costs more, but the whole operation stays in the range of pocket change for most tasks.

The bill hides in hosting, not training, and this is where 2026 split the models into two camps. Customized Nova models serve on-demand at the same per-token rate as the base Nova model, so a fine-tuned Nova Micro costs what plain Nova Micro costs to call, and nothing when idle. Older customizable models, Titan, Cohere, Llama 2, do not offer on-demand for custom versions. To serve them you buy Provisioned Throughput, billed per model unit per hour on a 1-month or 6-month commitment, which runs from roughly 20 dollars an hour per unit upward depending on the model. That is a standing cost whether or not a single request arrives.

The choice of base model quietly decides your hosting cost. Pick the model that gives you on-demand custom hosting unless volume clearly justifies the commitment.

There is a second trade-off worth plotting, and it is the one I actually use to decide. A long context prompt pays tokens on every request forever. A fine-tune pays a training cost up front and then a shorter prompt on every request. Cross those two lines and you get a break-even in request volume: below it, keep the long prompt; above it, the fine-tune wins. The exact crossing depends on your token counts, but the shape is always this.

Continued pre-training and Nova Forge

Continued pre-training is the other lever, and it answers a different question. Instead of teaching a task with labeled pairs, you feed the model a large body of unlabeled text from your domain, contracts, clinical notes, telecom logs, and let it absorb the vocabulary and structure. There are no right answers in the data, only raw material. The model comes out more fluent in the domain, which then makes fine-tuning or prompting on top of it more effective.

It is a heavier commitment and a smaller club. In managed Bedrock the Titan Text models are the pair that accept it, and it wants a genuinely large corpus to be worth the run. For deep, strategic domain adaptation of Nova, AWS points to Nova Forge, which does continued pre-training with extras that matter at that scale: data mixing to fight catastrophic forgetting, where the model loses general ability as it over-specializes, checkpoint selection, and a bring-your-own-optimizer path. Forge is an annual subscription and a longer training cycle, aimed at organizations building a model as a lasting asset rather than tuning one for a task. For most readers of this series, continued pre-training is good to understand and rarely the thing you reach for first.

Where fine-tuning trips you in production

The system prompt mismatch is the first and most common trap, and I named it above because it recurs. Train with one system prompt, serve with another, and the model quietly reverts toward base behavior. Store the exact training system prompt with the model and send it verbatim at inference. Treat it as part of the artifact, not a detail.

Overfitting is the second. Run too many epochs on too few examples and the model memorizes your training set, scoring beautifully on data it has seen and poorly on anything new. Hold out a test set the model never trains on, evaluate against that, and stop adding epochs once the loss flattens. A smaller, cleaner dataset beats a big noisy one almost every time, so spend your effort curating examples rather than piling them up.

The third is the hosting surprise, which is really a planning miss. Teams pick a base model for its benchmark scores, fine-tune it, then discover at deployment that serving it needs Provisioned Throughput and a monthly commitment they never budgeted. Decide how you will host before you choose what to customize. On a spiky or low-volume workload, a model with on-demand custom hosting is the difference between a few dollars and a few thousand. The fourth, quieter one: a custom model is frozen at a moment, and the base models keep improving, so revisit whether last quarter’s fine-tune still beats simply prompting this quarter’s base model.

Where a custom model earns its keep

Here is the posture I ship. Exhaust prompting and retrieval first, because they update instantly and cost nothing to change. When a high-volume, well-defined task has plateaued on those, fine-tune, and default to a Nova model so custom hosting stays on-demand and idle time is free. Start with supervised fine-tuning on a small, clean dataset, two or three epochs, and read the loss curve before you touch a hyperparameter. Keep the training system prompt with the model and send it unchanged. Reach for continued pre-training only when you have a large domain corpus and a strategic reason, and treat Provisioned Throughput as a deliberate purchase for proven steady volume, never a default.

Fine-tuning changes behavior you can demonstrate; it does not add facts you can look up, and it is not a fix for a weak prompt. Used in its lane it is cheap, fast, and genuinely useful. Used as a reflex it adds a pipeline and a hosting bill for a gain a better prompt would have handed you. The next part goes one step further along this path, into model distillation, where a large teacher model trains a small, cheap student to do its job. If you build on more than one cloud, the parallel move is fine-tuning Azure OpenAI, which trades Bedrock’s managed simplicity for a different set of controls. This week, take one task you are solving with a 4,000 token prompt, label 500 clean examples, and run a single Nova Micro fine-tune. The training bill will be smaller than your lunch, and the loss curve will tell you in an hour whether the task was worth it.

Related reading: the vendor-neutral look at why data, not model size, usually decides quality is the concept behind every good fine-tune.

References

• Amazon Bedrock, customize your model
• Supported models and Regions for customization
• Customize Amazon Nova models with Bedrock fine-tuning
• Amazon Bedrock pricing

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts