Ask a plain Gemini call who won a match last week and it will answer with total confidence, and sometimes it will be wrong. The model does not know it is out of date. Its weights froze at a training cutoff, so anything that happened after that cutoff is either guessed or hallucinated, and it hands you the guess in the same steady voice it uses for facts it actually learned. That is the failure grounding fixes.

Grounding means the model retrieves real documents at request time and writes its answer from them, then hands back the sources it used. On Vertex AI you can ground against two very different things: the live public web through Google Search, or your own private content through a Vertex AI Search data store. This part shows how to turn on each one, how to read the grounding metadata that comes back, how to render citations without corrupting them, and what the whole thing costs. Vertex AI is Google Cloud managed platform for building and running models; Gemini is the model family you call on it.

What grounding actually does

A normal generation call is a closed book exam. The model writes from what is baked into its weights. Grounding turns it into an open book exam: before answering, the model pulls in outside text and writes from that text, then tells you which passages it leaned on. You do not build a retrieval pipeline by hand for this. You attach a grounding tool to the call and the model runs the retrieve-then-generate loop itself.

Two grounding sources are worth knowing on Vertex AI. Grounding with Google Search connects the model to live, public web results, so it can answer about recent events and cite pages it found. Grounding with a Vertex AI Search data store points the model at a private index you built from your own documents, so it answers from your handbook, your product docs, your policies, and nothing else. A data store here is a managed search index over content you supplied, covered in Part 12.

The important behaviour is that the model, not your code, decides whether a given prompt needs retrieval at all. Ask it to rephrase a sentence and it will skip the search. Ask it about a price that changes weekly and it will search. You give it the capability. It picks the moment.

flowchart LR
  P[User prompt] --> D{Model needs sources}
  D -->|no| G[Answer from weights]
  D -->|yes, web| S[Google Search retrieves pages]
  D -->|yes, private| V[Vertex AI Search data store]
  S --> W[Model writes grounded answer]
  V --> W
  W --> M[Grounding metadata plus citations]

Google Search or your own data, which grounding source?

These two sources are not interchangeable, and picking wrong is the most common design mistake I see. Google Search is right when freshness and breadth matter and the answer lives on the open web: news, current prices, public specifications, anything that changed after the training cutoff. It is wrong when the answer is private, when you need a closed and auditable set of sources, or when you cannot let a question about internal data touch an external service. Your own data store is the mirror image: authoritative, private, bounded, and current only as far as your last ingestion run. It is wrong when the user asks about something you never indexed, because then it grounds on nothing and falls back to the same stale weights you were trying to escape.

Turn on Google Search grounding

On Vertex AI you use the google-genai SDK and attach a single tool. One naming trap to clear first: current Gemini models use the google_search tool, while older models used a tool called google_search_retrieval. If you copy a two-year-old snippet you will pass the wrong tool and get an argument error. Here is a current call on Gemini 2.5 Flash.

from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project='my-proj', location='us-central1')

resp = client.models.generate_content(
    model='gemini-2.5-flash',
    contents='What changed in the latest Vertex AI release notes this month?',
    config=types.GenerateContentConfig(
        tools=[types.Tool(google_search=types.GoogleSearch())],
    ),
)

print(resp.text)
meta = resp.candidates[0].grounding_metadata
if meta:
    print('searched for:', meta.web_search_queries)
    for chunk in meta.grounding_chunks or []:
        if chunk.web:
            print('source:', chunk.web.title, chunk.web.uri)

Expected output: a synthesized answer, then a line such as searched for: ['vertex ai release notes'], then one source: line per page the model cited. Failure mode: if grounding_metadata is None, the model chose not to search because it judged the prompt answerable from weights. Do not treat that as an error; treat it as an ungrounded answer and label it as such in your UI.

That is the whole activation. No index to provision, no embeddings to manage, no data prep. The cost of that convenience is that you are handing the query to an external search and you owe the display obligation covered below.

Read the grounding metadata and render citations

The answer text is the easy part. The value for a serious application is in the grounding metadata attached to each candidate, because that is what lets you show the user where a claim came from. Four fields carry the weight. The webSearchQueries list holds the queries the model actually ran. The searchEntryPoint holds a ready-made HTML widget of Search Suggestions you are required to display. The groundingChunks list holds the sources, each with a URI and title. And groundingSupports is the field that connects the answer to those sources: each support marks a segment of the answer text and lists which chunks back it, plus a confidence score per link.

Here is the trap that produces garbled citations in production. Each support segment carries a startIndex and endIndex, and those are offsets into the UTF-8 byte encoding of the answer, not character positions in a string. As long as your answer is plain ASCII the two happen to line up and everything looks fine in testing. The first time a response contains an accented name, an em character, an emoji, or any non-Latin script, slicing the Python string by those numbers cuts through the middle of a multibyte character and your quoted snippet turns to mojibake. Encode the text to bytes, slice the bytes, decode back.

Confidence scores deserve a real design decision rather than a blind display. Each grounding support carries a score between zero and one for how strongly the cited chunk supports that segment. A high score means the source clearly says what the model claims. A low score means the link is loose and the claim may be drifting from the source. I gate on this. Below a threshold I still show the answer but I flag the citation as weak, and for high-stakes surfaces I suppress the claim entirely. The chart below is a real distribution shape from a grounded support-bot answer.

Ground on your own data with a Vertex AI Search data store

Grounding on private content swaps the search tool for a retrieval tool that points at a data store you already built. The full path to the data store is the identifier you get back when you create it in AI Applications, and it must live in a location the grounding call can reach. The rest of the call looks almost identical, which is the nice part: same SDK, same response shape, different source.

from google import genai
from google.genai import types

client = genai.Client(vertexai=True, project='my-proj', location='us-central1')

ds = 'projects/my-proj/locations/global/collections/default_collection/dataStores/my-docs'

resp = client.models.generate_content(
    model='gemini-2.5-flash',
    contents='What is our refund window for enterprise plans?',
    config=types.GenerateContentConfig(
        tools=[types.Tool(
            retrieval=types.Retrieval(
                vertex_ai_search=types.VertexAISearch(datastore=ds)))],
    ),
)

print(resp.text)
for chunk in (resp.candidates[0].grounding_metadata.grounding_chunks or []):
    if chunk.retrieved_context:
        print('doc:', chunk.retrieved_context.title, chunk.retrieved_context.uri)

Expected output: an answer drawn from your indexed documents, then one doc: line per source chunk, this time from retrieved_context rather than web. Failure mode: a not-found or permission error almost always means the data store path is wrong or lives in a different location than the one in the client, or the caller lacks the Discovery Engine viewer role on it. The datastore path format should be confirmed against your own console value.

Notice the metadata reads from retrieved_context instead of web. That single difference is how your rendering code tells a private citation from a web citation, which matters because you probably link them differently and you certainly log them differently. If you built a full retrieval pipeline with the RAG Engine in Part 12, this grounding tool is the lighter path when you want the model to own the retrieve-and-cite loop instead of orchestrating it yourself.

What grounding costs

Grounding is a separate charge on top of the tokens the model generates, and the billing shape depends on which model generation you run. For Gemini 2.5 and older models, Grounding with Google Search bills per grounded prompt: one request that triggers a search is one billable unit no matter how many queries it fires. For Gemini 3 models the billing moved to per search query, so a single request that fires two queries counts as two billable uses. That difference flips the cost math depending on how many queries your prompts tend to spawn. The per-prompt versus per-query split is confirmed in the Google documentation; the exact dollar rates below I could not re-confirm this run, so treat them as directional.

Where grounding still gets it wrong

Grounding reduces hallucination. It does not end it. The model can still misread a retrieved passage, stitch two sources into a claim neither one makes, or ground on a page that is itself wrong, because Google Search grounds on what the web says, not on what is true. A data store has the opposite failure: it is only as current as your last ingestion, so a grounded answer can be confidently, cleanly cited, and out of date because nobody re-indexed the handbook after the policy changed. Neither source knows what it does not contain.

A few operational edges are worth stating plainly. Older models exposed a dynamic retrieval threshold that let you tune how eager the model was to search, trading recall against cost; on current models the model makes that call itself and the knob is gone. Grounding adds a network round trip and real latency, so a chatty UI that grounds every keystroke will feel slow. And privacy is a hard boundary: a question routed to Google Search leaves your project for an external search, which is fine for public questions and unacceptable for regulated internal ones. If your compliance posture forbids that, keep those intents on the data store and never let them reach the search tool.

Ground on your own data, add Search for what changes

Here is what I would build. Start with data store grounding, because for most business assistants the answers you care about live in your own content and you want them private, bounded, and auditable. Layer Google Search on top only for the intents that genuinely need the open web, route to it in your code rather than hoping the model guesses, and keep regulated questions off it entirely. Whichever source fires, read the grounding metadata, respect the byte offsets, show the Search Suggestions when the web tool was used, and gate weak citations on their confidence score. Log the query counts from day one so your cost forecast is grounded in something more than a guess.

Grounding gives the model the right facts to answer from. The next lever is teaching the model itself to behave the way your task needs, which is where tuning comes in: Part 16 on supervised tuning for Gemini. Before you move on, take one grounded call in your codebase and print the grounding_metadata. If it is None on a prompt you expected to search, you just learned the model is answering from memory when you thought it was checking the web.

References

• Grounding overview, Vertex AI documentation
• Grounding with Google Search, Gemini API documentation
• GroundingMetadata reference, Vertex AI
• Grounding with Vertex AI Search, Google Cloud documentation

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts