I get asked to fine-tune Gemini at least once a month, and most of the time the honest answer is do not. Tuning is the last lever you pull, not the first. Prompt engineering touches the input, retrieval touches the context, and tuning touches the weights, and in that order the cost and the risk climb at every step. Reach for the weights before you have wrung out the prompt and retrieval and you pay for training, you pay in iteration speed, and you inherit a model you now have to keep alive.

Supervised fine-tuning, or SFT, means you show a base model a few hundred to a few thousand labelled input-output pairs and it adjusts to match the style, format, or task in those examples. On Vertex AI, Google Cloud managed platform for building and running models, SFT on Gemini uses LoRA, low-rank adaptation, a method that trains a small set of extra weights instead of rewriting the whole model. That one choice is why a tuned Gemini model costs the same per token to run as the base model, and it is the fact that flips the economics in your favour. This part covers when tuning actually beats a better prompt, how to build a dataset the model can learn from, which knobs matter, what a real job looks like in code, what it costs, and where it goes wrong.

What supervised fine-tuning changes in the model

A base Gemini model is a generalist. It has read a great deal and it will do almost any task passably, but it does not know your ticket taxonomy, your house tone, or the exact JSON shape your downstream service expects. Tuning teaches it that. You are not adding facts to the model, you are adjusting how it behaves, so tuning is the right tool for form and style and narrow task skill, and the wrong tool for knowledge that changes. If the model needs a fact it never saw, that is a retrieval problem, and you solved it in Part 12, not a tuning problem.

LoRA is the reason this is cheap and low-risk. Instead of updating the billions of weights in Gemini, it freezes the base model and trains a thin layer of adapter weights on top. That layer is small, which keeps training fast and inexpensive, and it means the base model underneath is untouched, so nothing you do can degrade Gemini for anyone else. It also means the tuned model runs on the same serving stack as the base, which is why, on Vertex AI, tuned Gemini inference is billed at the base model rate with no surcharge. On most other platforms a tuned model costs more to serve. Here it does not.

One caution before you get excited. Tuning shows diminishing returns fast. The jump from a raw prompt to a hundred good examples is large. The jump from a thousand to five thousand is usually small, and past that you are mostly spending money to overfit. Quality of examples beats quantity every time. I would rather have three hundred hand-checked pairs than three thousand scraped ones.

flowchart TD
  A[Task underperforms] --> B{Better prompt fixes it}
  B -->|yes| P[Ship the prompt]
  B -->|no| C{Missing knowledge}
  C -->|yes| R[Add RAG retrieval]
  C -->|no| D{Wrong style or format}
  D -->|yes| S[Supervised fine-tuning]
  D -->|no| E[Try a stronger base model]
  R --> F{Still wrong}
  F -->|yes| S

When does tuning beat prompting and RAG?

The three techniques are not rivals, they stack. Prompting is free and instant and should always be tried first. Retrieval is the answer when the model lacks current or private facts. Tuning is the answer when you have shown the model what good looks like a dozen ways in the prompt and it still drifts, or when your prompt has grown so long with instructions and few-shot examples that it is slow and expensive on every single call. That last case is the quiet reason tuning pays: it lets you move a fat prompt into the weights and then send a short one. A shorter prompt on every request, forever, is a real saving.

Build a dataset the model can actually learn from

The dataset is the whole job. Everything else is a few API calls. Vertex AI wants your examples as a JSONL file, one JSON object per line, each object holding a conversation with a user turn and the model turn you want it to imitate. You put the file in a Cloud Storage bucket and point the tuning job at it. The three supported Gemini models for SFT today are gemini-2.5-pro, gemini-2.5-flash, and gemini-2.5-flash-lite, and you can tune on text, image, audio, and document inputs.

{"contents":[{"role":"user","parts":[{"text":"Extract the PO number from: Order ref PO-88421 shipped Tuesday."}]},{"role":"model","parts":[{"text":"88421"}]}]}
{"contents":[{"role":"user","parts":[{"text":"Extract the PO number from: No purchase order was supplied."}]},{"role":"model","parts":[{"text":"none"}]}]}

Expected shape: each line is one training pair. The first shows the model the happy path, the second teaches it the empty case so it returns none instead of inventing a number. Failure mode: a single malformed line, a trailing comma or a smart quote pasted from a doc, and the whole job fails validation before it starts. Lint the file before you upload it.

A few rules I hold to. Cover the edge cases on purpose, because the model learns the distribution you give it, and if every example is clean it will fall apart on the messy real input. Start around one hundred examples and grow only if the evaluation says you need to; Google recommends a floor near one hundred and quality over volume above that. Hold back ten to twenty percent as a validation set the model never trains on, so you can watch it generalise rather than memorise. And keep your examples in the same format your production prompt will use, or the model learns a shape you never actually send it.

Adapter size, epochs, and the learning rate multiplier

Vertex AI exposes three hyperparameters, and it fills in sensible defaults for all of them, so your first job should change nothing. Run the defaults, look at the result, then move one knob at a time. Here is what each one does. The adapter size sets the rank of the LoRA layer, which is roughly how much capacity the model has to learn your task; larger fits more complex behaviour but needs more data and more time and overfits more easily on a small set. Epochs is how many full passes the training makes over your data; too few and it underlearns, too many and it memorises. The learning rate multiplier scales how big each update step is; drop it when the model overfits, raise it when it barely moved.

Run a tuning job on Vertex AI

With the dataset in a bucket, the job itself is short. The Vertex AI SDK for Python starts it, polls until it finishes, and hands back the tuned model name and its endpoint. The training runs on Google-managed accelerators; you never see a GPU.

import time
import vertexai
from vertexai.tuning import sft

vertexai.init(project='my-proj', location='us-central1')

job = sft.train(
    source_model='gemini-2.5-flash',
    train_dataset='gs://my-bucket/tune/train.jsonl',
    validation_dataset='gs://my-bucket/tune/val.jsonl',
    epochs=4,
    adapter_size=4,
    learning_rate_multiplier=1.0,
    tuned_model_display_name='po-extractor-v1',
)

while not job.has_ended:
    time.sleep(60)
    job.refresh()
    print('state:', job.state)

print('tuned model:', job.tuned_model_name)
print('endpoint:', job.tuned_model_endpoint_name)

Expected output: a stream of state: lines while it runs, then a tuned model resource name and an endpoint you call exactly like a base Gemini model by passing that endpoint as the model. Failure mode: a permission error usually means the caller lacks the Vertex AI user role or cannot read the bucket; a quota error means you are over the concurrent tuning-job limit, which defaults low, so cancel a stale job or request more. Confirm the exact SDK argument names against the current docs before you run.

Once it finishes you evaluate the tuned endpoint on your held-out set, and if it wins you point production at it. Calling it is the same code you already wrote in Part 11, with the tuned endpoint in place of the base model id. Nothing else in your app changes.

What does tuning cost, and what stays the same?

Two costs, and only one of them recurs. Training is a one-time charge billed on the number of tokens processed, which is your dataset size times the tokens per example times the epochs, the same figure Figure 2 plots. Inference is the recurring cost, and here is the part that matters: on Vertex AI a tuned Gemini model is served at the base model token rate, with no tuning surcharge. You pay once to train and then you run the tuned model at Flash prices forever. Confirm the current training-token rate for your region before you forecast, since I could not re-verify the exact figure this run.

Overfitting, drift, and the model you now maintain

Tuning creates work that prompting never does, and the failure modes are quiet. Overfitting is the first: train too long or on too little and the model parrots your examples and cannot handle anything slightly different, which looks like great validation numbers and terrible real behaviour. Catastrophic forgetting is the second: push hard on one narrow task and the model can lose general ability it used to have, so test it on things outside your tuning set, not just on the task you tuned for. And then there is the maintenance you signed up for. A tuned model is frozen at the data you gave it. When your task shifts, when the format changes, when a new edge case shows up, the base model quietly improves under you but your tuned one does not, and you have to retrain to catch up.

There is a lighter cousin of tuning worth naming. If your goal is to make a small model behave like a large one, distillation can get you there by training the small model on the large model outputs rather than on hand-labelled pairs, and that is the next part in this series. Tuning and distillation solve overlapping problems, and knowing which to reach for saves a lot of labelling.

Where supervised tuning pays off

My recommendation is narrow on purpose. Do not tune to add knowledge, use retrieval for that. Do not tune because a prompt felt long, first try to make it a better prompt. Tune when you have a stable, well-defined task, a couple hundred high-quality examples, and either a quality gap that prompting cannot close or a fat prompt you want to bake into a cheaper model. In that last case the payoff is concrete: a tuned Flash or Flash-Lite that matches a premium model on your one task, billed at the small model rate, is the single best return I see from SFT on Vertex AI. Everywhere else, tuning is a cost you take on before you have earned it.

If you think you have a real tuning candidate, do not start with a training job. Start with the dataset. Write one hundred examples by hand, including the ugly edge cases, and run your current prompt against them first. Half the time that exercise alone shows you a prompt fix and you never need to tune at all. If the gap survives a hundred good examples, then you have earned the training run, and the next part, on distilling Gemini, shows the cheaper path when your labels can come from a bigger model instead of a human.

References

• About supervised fine-tuning for Gemini models, Vertex AI documentation
• Tuning API reference, Generative AI on Vertex AI
• To tune or not to tune, Google Cloud blog
• Prepare supervised fine-tuning data for Gemini models

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts