Pick the model on the license first and the benchmark second. That sounds backwards until the day your general counsel asks who pays if a model output infringes someone’s copyright. On watsonx.ai the answer depends entirely on which model you loaded, and that is the real reason Granite is the default choice. This part is about the family you get, the outside models sitting next to it, and the paperwork that separates them.

In Part 1 I mapped the three watsonx products at the platform level. In Part 2 we ran a prompt in Prompt Lab against a Granite model without stopping to ask what Granite actually is. Now we stop and ask.

What sits inside the Granite family

Granite is IBM’s own family of foundation models, trained by IBM Research and released as open weights under the Apache 2.0 license. A foundation model is a large model pre-trained on broad data that you then adapt to many tasks, rather than a narrow model built for one job. The word family matters here. Granite is not one model with a size dial. It is a set of models across several jobs, and knowing which is which saves you from loading a chat model when you needed a document reader.

The current generation is Granite 4.1, released on 29 April 2026. IBM calls it their most expansive release, and it spans five kinds of model. The language models are dense, decoder only text models in 3B, 8B, and 30B sizes, each shipped as a base and an instruct variant. Granite Vision 4.1 is a 4B vision language model tuned for document understanding, meaning tables, charts, and key value extraction from invoices and forms. Granite Speech 4.1 is a 2B transcription and translation model that posts a 5.33 percent word error rate, which puts it near the top of the open speech leaderboards. Granite Guardian 4.1 is a safety model that reads inputs and outputs and flags harm, and it replaces the older Guardian 3.3 8B. Granite Embedding Multilingual R2 turns text into vectors for search across more than 200 languages, with a small 97M variant for tight hardware.

Every one of those models ships under Apache 2.0. Hold that fact. It comes back when we get to why enterprises pick this family over a model that scores a point higher on some leaderboard. First, here is the collection as a tree, so you can see that a Granite deployment is usually several of these working together rather than one model doing everything.

flowchart TD
  G[Granite 4.1 collection] --> L[Language 3B 8B 30B]
  G --> V[Vision 4B]
  G --> S[Speech 2B]
  G --> R[Guardian safety]
  G --> E[Embedding R2]
  L --> LB[base and instruct]
  V --> VD[tables charts KVP]
  R --> RM[reads input and output]

Granite 4.1 versus Granite 4.0, and why 8B replaced 32B

You will still see Granite 4.0 models in the catalog, so it helps to know how the two generations differ. Granite 4.0, from October 2025, was the interesting architecture bet. Instead of a standard transformer, it used a hybrid design that interleaves a small share of attention layers with a majority of Mamba-2 state space layers, at roughly a nine to one ratio. Some of the 4.0 models also used a mixture of experts, where only a slice of the network fires for each token. Granite 4.0-H-Small carries 32B total parameters but activates about 9B per token. Granite 4.0-H-Tiny is 7B total and about 1B active. Granite 4.0-H-Micro is a plain 3B dense model. The payoff of that design is memory. IBM measured more than a 70 percent cut in serving RAM at long context, and roughly double the inference speed, against a comparable dense transformer.

Granite 4.1 changed direction. The language models went back to dense, decoder only transformers at 3B, 8B, and 30B, trained on about 15 trillion tokens with a staged recipe that pushes context length out to 512K. The headline result is the one worth remembering: the Granite 4.1 8B instruct model matches or beats the Granite 4.0 32B mixture of experts model on instruction following and tool calling, while being simpler to fine tune. A smaller, plainer model caught the older, larger one. That is why 8B is now the number I reach for by default, and why I no longer treat 30B as the safe choice just because it is bigger.

Reading a watsonx model id

When you call a model from code, you do not pass its friendly name. You pass a model id, and the id encodes what you are getting. Take the pattern ibm/granite-4-1-8b-instruct. The provider prefix ibm marks it as an IBM developed model. The stem granite is the family. The 4-1 is the generation, Granite 4.1. The 8b is the size. The instruct suffix means the tuned variant that follows instructions, as opposed to the base variant you would fine tune yourself. A third party id follows the same shape with a different provider, for example a mistralai or a meta-llama prefix, which is a fast way to see at a glance whether an id is an IBM model or an outside one.

One naming quirk trips people up. On Hugging Face the Granite 4.0 models use dots, like granite-4.0-h-small. In watsonx ids the same models are written with hyphens. Copy the exact id from the model card inside the studio rather than retyping it from a blog, because a single wrong character returns a model not supported error that looks like an outage but is a typo. Always confirm the live id under View all foundation models for your region before you hardcode it.

You do not have to guess which ids exist. The SDK will list them. This snippet authenticates, pulls the model specs, and prints every Granite id available to your project, which is the clean way to find the exact string to paste into the parameters panel from Part 2.

from ibm_watsonx_ai import APIClient, Credentials

client = APIClient(
    credentials=Credentials(
        url='https://us-south.ml.cloud.ibm.com',
        api_key='YOUR_IBM_CLOUD_API_KEY',
    ),
    project_id='YOUR_PROJECT_ID',
)

# get_model_specs returns the catalog for your region and plan
specs = client.foundation_models.get_model_specs()

for m in specs['resources']:
    if 'granite' in m['model_id']:
        print(m['model_id'])

Expected output: a list of Granite ids such as ibm/granite-4-1-8b-instruct and ibm/granite-4-h-small, one per line. Failure modes: a 401 means the api_key or project_id is wrong; an empty list usually means a region with a different catalog, so switch the url to your region; an AttributeError on get_model_specs means an older ibm-watsonx-ai package, so upgrade it. The method name and return shape can shift between SDK releases, so confirm against your installed version.

Which Granite size should you load?

Three sizes, one honest rule. The 3B model is for the edge and for high volume, simple tasks like short classification, where latency and throughput matter more than nuance. The 8B model is the workhorse. It is where I start for instruction following, tool calling, and answering over retrieved documents, and it is the size that now matches the old 32B. The 30B model is for when you need the most quality Granite can give on a task and can pay for the extra compute and latency. Here is the part people miss on cost: because a Resource Unit meters tokens, and a workload sends the same tokens regardless of model size, the raw token bill is identical across 3B, 8B, and 30B for a given prompt and output. What changes is the price per Resource Unit tier, the latency, and the hardware footprint, not the token count.

Third-party models in the catalog

Granite is the default, not the only option. The watsonx.ai catalog also carries third party and open models, with Meta Llama and Mistral the two names you will meet first. IBM curates that list through technical, licensing, and performance review, so the catalog is a shortlist rather than every model on the internet. These models run through the same Prompt Lab and the same API you learned in Part 2, so switching one in is a matter of changing the model id, not rewriting your code.

The reason to reach for one is usually a specific capability, or a model your team already validated elsewhere and wants to keep using. The reason to pause is everything in the next two sections. A third party model can be the right engineering call and still carry legal terms that change how you are allowed to deploy it and who carries the risk if an output goes wrong. That is not a reason to avoid them. It is a reason to read the license before you commit, the same way you would for any dependency you pull into production.

Why the Apache 2.0 license matters for Granite

Apache 2.0 is a permissive open source license, and in plain terms it lets you use the model commercially, modify it, redistribute it, and build products on top, while asking little in return beyond keeping the copyright notices. It includes an explicit patent grant, and it is not copyleft, so a derivative you build does not have to be released under the same license. For a model, that combination is the freedom to self host, fine tune, and ship a Granite based system, including on premises or fully air gapped, without a separate model license to negotiate with a vendor.

Third party models are not automatically the same. Some Mistral models are open weight under Apache 2.0, while others are commercial. The Llama models come under Meta’s community license, which carries an acceptable use policy and a scale clause for very large deployments. None of that makes them bad choices. It means the license, not the benchmark, decides whether you can even run the model the way your deployment requires. An air gapped bank and a public chatbot face different constraints from the same license text. Check it before the architecture depends on it.

Indemnification, and where the liability lands

Indemnification is the part your legal team cares about most, and it is simpler than it sounds. If a third party sues you claiming that a model’s output infringes their intellectual property, an indemnity is the vendor agreeing to defend you and cover the resulting costs. For its own developed models, including Granite, IBM provides standard contractual IP indemnification, the same kind it offers for its hardware and software. Two details make it strong. IBM does not require you to indemnify it back for your use of those models, and it does not cap its indemnification liability for IBM developed models. Uncapped, with no reverse indemnity, is about as clean as this gets.

Third party models generally do not come with that protection. The notable exception is Mistral Large, which IBM covers through a capped IP indemnity, and it was the first third party foundation model to get IBM indemnification at all. Cap versus no cap is the whole point of the table below. When a regulated customer asks me why Granite is the default even though a competing model edged it on one benchmark, this is the answer. The benchmark gap is a point or two. The indemnity gap is the difference between capped and uncapped exposure on a claim you cannot predict.

Indemnity terms are set in your IBM agreement and can change, so confirm the current terms with IBM for your contract before relying on them. For how a different cloud frames third party model terms, see the Google Cloud take in Vertex AI Model Garden third party models.

My default: granite-4.1-8b, and when I break that rule

For new work on watsonx.ai I start on granite-4.1-8b-instruct and change my mind only with evidence. It carries the Apache 2.0 license and IBM’s uncapped indemnity, it matches the older 32B on the metrics enterprise work leans on, and it fine tunes cleanly for the tuning we cover later in the series. That is a strong default before you have run a single eval.

I break that rule in three cases. When latency or throughput at the edge is the pain, I drop to 3B and confirm quality holds. When long context serving is memory bound on the hardware I have, I look at Granite 4.0-H-Small, whose hybrid design fits where a dense 32B will not. When a third party model has a capability Granite genuinely lacks and my own eval proves it, I use it, and only after I have read the license and checked the indemnity with legal. Notice that two of those three exceptions are engineering calls and the third is a legal one. That is the whole shape of choosing a model here.

Do one thing now. Open View all foundation models in your project, find the exact granite-4-1-8b id for your region, and pin it in your config so the code in Part 2 has a real id to call. In the next part we leave model choice behind and look at where watsonx actually runs, from SaaS on IBM Cloud to Cloud Pak for Data and on premises OpenShift, because the deployment option you pick constrains which of these models you can even use.

References

IBM Research: Introducing the Granite 4.1 family
IBM: Granite 4.0 hybrid models announcement
IBM newsroom: client protections for watsonx models
IBM docs: third party foundation models
watsonx.ai foundation models

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts