,

Vertex AI and Model Garden, from Catalog to Endpoint (Google Cloud Gen AI Series, Part 2)

Vertex AI is Google Cloud’s managed platform and Model Garden is its 200-plus model catalog. Here is how the three access paths, managed API, MaaS, and self-deploy, decide your cost, latency, and data isolation, and which one to start on.

Google Cloud Gen AI Series · Part 2 of 30

You open the Google Cloud console, type a prompt into a Gemini model, and it answers in about a second. Behind that single call sits a catalog of more than 200 models and three completely different ways to run any of them. Pick the wrong one and you end up paying for idle accelerators, or shipping data out of a network you meant to keep sealed. This part is about that catalog, Model Garden, and the platform it lives in, Vertex AI, so the choice stops being an accident.

Key takeaways

Vertex AI is Google Cloud’s managed platform for building, tuning, and serving models. Model Garden is the catalog inside it, holding Google first-party models, partner models, and open models in one place. Every model reaches you through one of three paths: a Google managed API, a serverless Model-as-a-Service endpoint, or a model you self-deploy onto your own dedicated endpoint. The path decides your billing, your latency floor, and whether traffic ever leaves your VPC. Get that decision right first, then argue about which model.

Who this is for: Engineers and architects who have read Part 1 and know what the Google Cloud GenAI stack contains at a high level. You need a Google Cloud project, the ability to enable the Vertex AI API, and rough familiarity with what a VPC is. No prior Vertex AI experience is assumed. If a term is new, it gets one plain definition on first use.

What Vertex AI actually is

Vertex AI is the single managed platform on Google Cloud for the whole model lifecycle: discover a model, test it, tune it, deploy it, and monitor it in production. Before Vertex, those steps lived in separate products and you stitched them together yourself. Now they share one API surface, one set of IAM permissions, one billing view. For generative AI specifically, Vertex gives you access to Gemini and the rest of the catalog, plus the plumbing around them, retrieval, grounding, evaluation, and tuning, that later parts cover one at a time.

One naming note, because it will confuse you otherwise. At Google Cloud Next in April 2026, Google began presenting much of this surface under the name Gemini Enterprise Agent Platform, with an agent-first framing on top of the same Vertex AI capabilities. The APIs, the SDK, and the resource names you will type are still Vertex AI. I use Vertex AI throughout this series because that is what the docs and the gcloud commands still say. If you see the newer name in the console, it is the same platform underneath.

Vertex AI, three ways to run any modelOne catalog on top, three access paths, one managed baseModel Garden catalog200+ models: Google first-party, partner, open weightGoogle managed APIGemini, Imagen, Veo.Serverless, per token.MaaS endpointClaude, Llama, Mistral.Serverless, per token.Self-deployedOpen or licensed model.Your VPC, per hour.Vertex AI managed infrastructureCloud TPUs, GPUs, regional endpoints, IAM, VPC Service Controls
Figure 1. The catalog is the same for everyone. The path underneath it is your decision.

How Model Garden groups its models

Model Garden is the catalog inside Vertex AI: a single searchable library of over 200 models with a card for each one. A model card tells you what the model does, how it is licensed, which access paths it supports, and lets you open it in a playground before you write a line of code. The models fall into three groups by who built them, and the group tends to decide how you will run it.

GroupExamplesUsual access path
Google first-partyGemini (Flash, Pro), Imagen, Veo, Chirp, LyriaGoogle managed API
PartnerAnthropic Claude, Mistral, GrokMaaS, or self-deploy with a license
Open weightGemma, Llama, DeepSeek, QwenMaaS, or self-deploy

Gemini is Google’s family of multimodal models, meaning they take text, images, audio, video, and PDFs as input. Within the family, Flash is the smaller, cheaper, faster tier and Pro is the larger, higher-reasoning tier. Gemma is Google’s open-weight family you can download and run anywhere. The partner models matter more than people expect: Claude and others are first-class citizens in the catalog, billed on your Google Cloud invoice, so you are not forced onto Gemini to stay inside one contract and one security boundary. If you came from AWS, this is the same idea as the Bedrock model catalog, with a wider open-model bench.

In practice

The catalog exposes far more models than any team should run in production. I keep a shortlist of three: one small Gemini tier for high-volume cheap calls, one strong reasoning model for the hard 10 percent of requests, and one open model I could self-host if a contract or a region ever forces me off the managed API. Everything else in the catalog is a candidate, not a dependency.

Reading a model card before you commit

Every model in the catalog has a card, and the card is where you settle the questions that decide whether the model even fits, long before you write code. Four fields carry most of the weight. The license tells you whether you can use the model commercially and whether self-hosting needs a paid agreement through Marketplace; skip this and you can ship something you have no right to ship. The supported regions tell you where the model can actually run, which matters the moment you have a data-residency rule, because a model that only serves from one region is not an option in a locked-down project. The context window, the amount of text the model can read at once, decides whether your documents fit in a single call or need chunking, a topic Part 12 handles in detail. And the input types tell you whether the model is text-only or takes images, audio, and video too.

The card also carries a playground on most models, so you can paste a real prompt and see the answer before committing a line of code or a cent of engineering time. I use it as a fast filter. Take three candidate models, run five of your actual hard prompts through each playground, and two of the three usually fall out on quality or refusal behaviour before any integration work starts. That saves days. What the card cannot tell you is how the model behaves on your traffic at your volume, so treat the playground as a screen, not a verdict, and confirm the real numbers with an evaluation run later in the series. Read the card first. It is the cheapest step in the whole process.

Which access path should you pick?

This is the decision that actually costs money, so slow down here. Model-as-a-Service, MaaS, means Google runs the model for you on shared, serverless infrastructure and bills you per token. You provision nothing. Self-deploy means you stand the model up on a dedicated Vertex AI endpoint, a set of machines with accelerators attached that you rent by the hour, running inside your own project and VPC network. The first path is elastic and cheap at low volume. The second gives you fixed cost, fixed latency, and traffic that never leaves your network.

PropertyManaged API / MaaSSelf-deployed endpoint
Infrastructure you manageNoneMachine type, accelerator, replica count
Billing modelPer input and output tokenPer node-hour, running or idle
Cold startNone, always warmMinutes to spin up an endpoint
Network isolationGoogle-managed, VPC-SC perimeter supportedRuns inside your VPC, full VPC-SC control
Best forSpiky or low volume, fast startSteady high volume, strict data rules

The break-even is about utilization, not preference. A serverless call bills only when it runs. A dedicated endpoint bills every hour it exists, whether or not a request arrives. So the honest question is: will the endpoint stay busy? If your traffic is bursty and your daily token count is low, MaaS wins and it is not close. Once you are pushing a model hard for most of the day, the per-hour rate on a self-deployed endpoint can undercut per-token pricing, and you get a latency floor you control.

Time from decision to first tokenApproximate minutes to a working call, first time on a fresh project0102030Minutes to first responseGoogle managed API, Gemini · about 1 minMaaS partner or open · about 3 minSelf-deploy endpoint · about 25 min
Figure 2. Serverless paths answer in about a minute. A self-deployed endpoint pays that setup time back only under sustained load.

Calling a Gemini model from the SDK

The managed path is the one you should reach for first, and it is a few lines. Install the Google Gen AI SDK, point the client at Vertex by passing vertexai=True with your project and region, then call generate_content. The same client shape works for partner MaaS models by swapping the model id, which is the point: one SDK, one auth path, many models.

# pip install google-genai
from google import genai

client = genai.Client(
    vertexai=True,
    project='my-gcp-project',
    location='us-central1',
)

resp = client.models.generate_content(
    model='gemini-2.5-flash',
    contents='Summarise VPC Service Controls in two sentences.',
)
print(resp.text)

Expected output is a two-sentence summary printed to your terminal, returned in roughly a second. The common failure on a fresh project is a 403 permission error, which almost always means the Vertex AI API is not enabled or your identity lacks the aiplatform.endpoints.predict permission. Enable the API, grant the Vertex AI User role, and rerun. A second frequent failure is a model id that does not exist in your chosen region; the error names the model, and the fix is to check the model card for regional availability.

Gotcha: The older vertexai.generative_models module in the legacy Vertex AI SDK was deprecated on 24 June 2025 and is scheduled for removal on 24 June 2026. Tutorials still show it. If your code imports from vertexai.generative_models import GenerativeModel, migrate to the Google Gen AI SDK shown above before that removal date, or your build breaks on an upgrade. Start new work on google-genai and skip the migration entirely.

Self-deploying an open model to your VPC

When a rule says the model must run inside your network, or when your load is heavy and steady enough to beat per-token pricing, you self-deploy. Model Garden has a one-click Deploy button on eligible cards, and the same action is available from the command line so you can put it in a pipeline. For open models the deploy is direct. For proprietary partner models you self-host, you first buy a commercial license through Google Cloud Marketplace, then deploy the same way. Either way the endpoint runs in your project, and it respects VPC Service Controls, so sensitive traffic stays inside the perimeter you already drew.

# Deploy an open model from Model Garden to a dedicated endpoint
gcloud ai model-garden models deploy 
    --model='google/gemma@gemma-3-27b-it' 
    --region='us-central1' 
    --project='my-gcp-project'

# Then call your own endpoint id, not a shared MaaS route.

Expected result is a new endpoint that reports as deploying, then ready after several minutes while machines and the accelerator come up. The failure mode that catches people is quota: a fresh project often has zero quota for the GPU or TPU type the model needs, and the deploy sits pending or fails outright. Check accelerator quota in the region before you deploy, and request an increase early because approval is not instant. The exact model reference format and supported machine types are on the model card, so read it rather than guessing the flags.

Worked example

Say a support assistant handles 2 million requests a month, each about 400 input tokens and 300 output tokens. That is roughly 1.4 billion tokens a month. On a serverless per-token rate, cost scales straight with that volume and every request answers with no cold start. Now put the same load on a self-deployed endpoint sized to hold peak traffic: two replicas of an accelerator node running every hour of the month, roughly 1,460 node-hours whether the queue is full or empty.

The serverless bill tracks actual usage. The endpoint bill is fixed. At this volume the endpoint is busy most of the day, so its per-hour cost can land below the per-token equivalent, and you gain a latency floor plus in-VPC traffic. Halve the traffic and the maths flips: the endpoint still bills 1,460 hours while serverless bills half as much. Utilization decides. Model the two curves with your own regional prices before you commit, because the crossover moves with the model and the accelerator.

Effort versus isolation, by access pathUp means more traffic stays in your VPC. Right means more you operate.highlowData isolationlowhighOperational effort you ownGoogle managed APIleast effort, Google-runMaaS partner or openserverless, still Google-runSelf-deployed in your VPCmost control, most to run
Figure 3. You buy isolation with operational effort. Most teams do not need the top-right corner.
Before you deploy in production: A self-deployed endpoint spends money the moment it exists and keeps spending while idle. Set up a budget alert, cap the replica count, and confirm autoscale-to-zero behaviour for your model type before you point production traffic at it. Test in a non-production project first. Nothing here should be a first change on a live workload.

Start on MaaS, self-deploy only when a rule forces it

Here is the recommendation, not a shrug. Begin every generative project on the managed API or MaaS. It costs you nothing when idle, it answers in about a second, and it lets you compare Gemini against Claude and an open model with one SDK and one bill. Move a workload to a self-deployed endpoint only when a concrete reason pushes you there: a data-residency or in-VPC rule the serverless path cannot meet, sustained volume where the per-hour rate clearly beats per-token, or a latency floor you must own. Those are real reasons and they do occur. What is not a reason is a vague wish for control, because control here is paid for in node-hours and on-call time.

One more caution. Do not confuse Vertex AI Model Garden with the consumer Gemini API in Google AI Studio; they overlap in models but differ in billing, quotas, and enterprise controls, and Part 5 of this series draws that line in full. For now, pick your path on this project deliberately, then open the model card for the one Gemini tier you will start with and send the SDK snippet above. If it prints a summary, you have the managed path working, and everything later in the series builds on top of that call.

Google Cloud Gen AI Series · Part 2 of 30
« Previous: Part 1  |  Guide  |  Next: Part 3 »

References

About The Author


Discover more from Journal of Intelligent Infrastructure – By Dr Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure - By Dr Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading