A team I worked with spent three weeks standing up Red Hat OpenShift and a GPU pool before anyone asked whether they needed to run watsonx.ai themselves at all. They did not. Their data was allowed to live in IBM Cloud, the SaaS plan would have had them building against Granite that same afternoon, and the cluster they hand rolled turned into a maintenance bill nobody had budgeted for. The deployment choice came before the model choice, and they got it backwards.
The short version
watsonx.ai comes two ways. As a Service, IBM runs it for you on IBM Cloud, and now also on AWS, and you pay by usage. As software, you run it yourself on Red Hat OpenShift through IBM Software Hub, the control plane that used to be called Cloud Pak for Data.
SaaS gets you building today with no cluster to own. Software exists for one reason: a requirement SaaS cannot meet, usually data residency, an air gap, or an OpenShift and GPU estate you already run.
Pick SaaS first. Move to software only when a named rule forces it, because self managing brings GPU sizing, operators, and version matching that IBM otherwise handles for you.
In Part 1 I mapped the three watsonx products at the platform level. In Part 2 we ran a prompt in Prompt Lab, and in Part 3 we picked a model and read its license. Every one of those steps assumed watsonx.ai was already running somewhere. This part is about that somewhere, because the place you deploy sets which models, which regions, and which controls you can even use.
Two ways to get watsonx.ai, SaaS or software
watsonx.ai is delivered in exactly two shapes, and everything else is a variation on one of them. The first is Software as a Service, where IBM operates the whole platform in its own cloud and you reach it over the network with an API key and a browser. You never see the servers. The second is software you install and run yourself on Red Hat OpenShift, IBM’s enterprise Kubernetes distribution, through a control plane called IBM Software Hub. If you have read older IBM material you will know that control plane as Cloud Pak for Data. IBM renamed it to IBM Software Hub with the 5.x releases, and watsonx.ai installs as a service on top of it.
The split matters because it decides who carries the operational weight. On SaaS, IBM patches the platform, sizes the GPUs, and keeps the models current. Your job stops at your data, your prompts, and your application. On software, all of that becomes yours: the cluster, the GPU drivers, the upgrade cadence, the capacity planning. That is not a small handover. It is the gap between a project that ships this week and one that hires a platform team first. So the honest default is SaaS, and the rest of this part is really about the specific reasons you would give it up.
flowchart TD Q[Where must the data live] --> A[IBM Cloud or AWS is allowed] Q --> B[Must stay in our datacenter] A --> SAAS[watsonx.ai as a Service] B --> C[We already run OpenShift and GPUs] C --> SW[watsonx.ai software on Software Hub] B --> D[No cluster yet] D --> HCI[IBM Fusion HCI appliance]
watsonx.ai as a Service
The fastest way onto watsonx.ai is the as a Service option. You create a watsonx.ai Runtime instance in the IBM Cloud catalog, pick a region, and you are calling Granite within minutes. IBM Cloud offers the service in several regions, Dallas, Frankfurt, London, Tokyo, and Sydney among them, and the region you choose fixes where your data is processed and which models appear in your catalog, the same region trap Part 3 flagged when a model id failed in the wrong place. Availability by region shifts over time, so confirm the current list in the catalog. As of the recent releases the same SaaS is also offered on AWS, so a shop standardized on AWS can consume watsonx.ai without pulling IBM Cloud into the account.
SaaS pricing runs through two plans, and you sign up for one of them the moment you provision. Essentials is pay as you go with no monthly instance fee, which is the right start for a pilot where you cannot yet predict volume. Standard is the enterprise plan: a fixed monthly instance fee of 1,110 US dollars that includes a block of 2,500 Capacity Unit Hours, with anything above that metered at the Standard rate. Two billing units show up on that invoice, and it helps to separate them now. A Capacity Unit Hour, or CUH, meters compute for watsonx.ai Runtime work such as tuning and pipelines, everything except foundation model inference. Inference itself is billed in Resource Units, where one Resource Unit is 1,000 tokens of input plus output, priced from a base of 0.0001 US dollars times the model rate. Part 5 takes those two meters apart properly. Here they matter only because they are the cost you agree to when you pick SaaS.
| Plan | Monthly instance fee | Included CUH | Best for |
|---|---|---|---|
| Essentials | None, pay as you go | None | Pilots, spiky or unknown volume |
| Standard | 1,110 US dollars | 2,500 CUH included | Steady production with predictable compute |
Plan terms and rates change, so confirm current pricing on the IBM watsonx.ai pricing page before you commit a budget. Foundation model inference is billed separately in Resource Units on both plans.
When does self-managed software make sense?
If SaaS is the default, the software option needs a reason, and there are really only three good ones. The first is data residency or an air gap: a regulator, a contract, or an internal rule says model inference and the data around it must never leave your own datacenter. SaaS cannot satisfy that; software can. The second is an estate you already own. If you run Red Hat OpenShift with GPU nodes today, installing watsonx.ai onto it can be cheaper and more controllable than paying SaaS rates on top of hardware you already bought. The third is deep customization of the platform itself, the kind that needs operator level access. Convenience is not on that list. If your reason is that you would rather run it yourself, you are buying a platform team you did not need.
The software path also brings a versioning discipline that SaaS hides from you. watsonx.ai software installs on IBM Software Hub, and the two run at matching versions. If the Software Hub control plane is at 5.2.2, you install watsonx.ai at 5.2.2, not a mismatched pair. That sounds obvious until an upgrade window, when moving the control plane forward without moving the service, or the reverse, is exactly how a cluster ends up in a state IBM support asks you to unwind. On SaaS none of this is your problem. On software it is a standing item on your run book, and the reason software deployments need a change process before they need a model.
Worked example
A bank runs a document assistant over customer records that a regulator forbids from leaving the bank datacenter. SaaS is off the table on that rule alone, so this is a software deployment on their existing OpenShift. Sizing the inference load still uses the same token math from Part 3: the assistant answers about 40,000 queries a month, roughly 900 input and 100 output tokens each, so about 1,000 tokens, or 1 Resource Unit, per query. That is near 40,000 Resource Units a month of inference the bank must serve on its own GPUs, the exact throughput IBM would have handled on SaaS.
They did not size the full cluster first. A 5,000 query pilot on the same design is about 5,000 Resource Units, shown next, which they ran on two GPUs to confirm quality and latency before buying for the 40,000 query load. The math is identical to the SaaS meter; only the party running the GPUs changed.
Running the software on Red Hat OpenShift
Running the software yourself starts below watsonx.ai, at the cluster. You need a supported Red Hat OpenShift Container Platform. Version 4.14 is the working floor, and IBM recommends 4.15 or higher for supportability, so target the higher one on anything new. On that cluster you install the IBM Software Hub control plane first, then add watsonx.ai as a service on top, at the matching version. The order is not optional. The service assumes the control plane is already there, and it will not install into empty space.
Because watsonx.ai serves and tunes models, the cluster needs GPUs, and OpenShift does not expose GPUs to workloads on its own. Two operators make them visible. The Node Feature Discovery operator labels nodes with the hardware they carry, and the NVIDIA GPU Operator installs the drivers and the device plugin that let a pod actually request a GPU. Install both before the watsonx.ai service, not after, because the install looks for schedulable GPU capacity and fails cleanly if it finds none. This is the step teams new to OpenShift most often skip, then spend a day debugging an install that was never going to find a card.
Before you start the watsonx.ai install, verify the cluster is actually ready to schedule GPU work. These four checks catch the usual reasons an install stalls.
# 1. Confirm the cluster is on 4.14 or, better, 4.15+
oc version
# 2. Node Feature Discovery should be labelling GPU nodes
oc get nodes -l feature.node.kubernetes.io/pci-10de.present=true
# 3. The NVIDIA GPU Operator should be installed and Succeeded
oc get csv -n nvidia-gpu-operator | grep gpu-operator
# 4. A GPU node should advertise schedulable capacity
oc get node <gpu-node> -o jsonpath='{.status.allocatable.nvidia.com/gpu}'Expected output: oc version prints a Server Version of 4.15 or higher; the NFD query lists your GPU worker nodes; the GPU Operator CSV shows Succeeded; and the last command prints a number such as 1 or 8, the GPUs that node can schedule. Failure modes: an empty node list means NFD is not labelling nodes, and the label key can differ by version, so check with oc get nodes --show-labels; a CSV in Installing or Failed means the GPU Operator is not ready, so the watsonx.ai install will not find GPUs; a blank capacity means the device plugin is not running on that node. Confirm the exact label key against your own cluster.
GPU requirements and the single node rule
The GPUs themselves are not generic. To deploy custom foundation models on watsonx.ai software, IBM calls for NVIDIA A100 80GB or H100 80GB cards, and the 80GB of memory is the part that matters, because it sets how large a model, and how long a context, a single card can hold. Smaller stock models have lighter needs, but size the cluster around the largest model you intend to serve, not the smallest, or you will be back ordering hardware a month after go live.
The specific GPU count per model comes from IBM’s published requirements tables, which move as models change, so read the current table for the model you plan to run rather than trusting a number from a blog. What does not move is the shape of the sizing problem: memory first, then how those cards have to be grouped, which is the trap in the next callout.
On-premises and IBM Fusion HCI
The deepest end of the software option is fully on premises, in your own datacenter with no public cloud in the path. That is OpenShift on your own hardware, and it raises a question SaaS never does: where does the cluster come from. Two common answers. If you already run OpenShift, you add GPU nodes and install onto it. If you do not, IBM Fusion HCI gives you a bare metal appliance built for this, with worker nodes, GPU nodes, and the high speed switches that training and inference traffic need, so you are not assembling a platform from parts. IBM Power systems are supported for shops standardized on that architecture, and IBM has published patterns for running watsonx on premises on VMware infrastructure as well.
There is a middle ground worth naming, because on premises does not have to mean your own steel. The same software can run in your own virtual private cloud on AWS or Azure, which keeps the platform inside your cloud account and network boundary while a hyperscaler still supplies the hardware. That is the option I reach for when the rule is our cloud account, our VPC, rather than our building. It gives you most of the control of on premises without owning a datacenter. For how the same which platform does this project belong in question plays out on another cloud, see the Azure take in Azure OpenAI vs Foundry vs Azure Machine Learning.
Can you move between them later?
A fair question before you commit: if you pick one shape now, how stuck are you. Less than you might fear on the code, more than you might hope on the data. The application layer ports well, because the ibm-watsonx-ai SDK and the REST API you met in Part 2 look the same whether you call SaaS or software. Point the client at a different base URL, swap the credentials, and the same code runs. That is deliberate, and it is the strongest reason to start on SaaS without worrying you have painted yourself into a corner.
What does not move by itself is everything with weight. Your grounding data, your vector indexes, and any tuned model artifacts live where you put them, so a move from SaaS to software means re hosting that data on your cluster and re running the tuning jobs there. There is no one click export that lifts a whole watsonx.ai project from IBM Cloud onto your own OpenShift. Plan a migration as a project, not a setting you flip.
The practical read is this. Prototype on SaaS, keep your prompts and app code portable by not hardcoding a base URL, and treat the data layer as the thing that pins you. If a residency rule is likely to land later, design the data path for it now, even while you build on SaaS, so the eventual move is re hosting data you already control rather than untangling a design that assumed IBM held everything. I have watched both versions of that migration. The one that planned the data path early took a sprint; the one that did not took a quarter.
Start on SaaS, move to software only when a rule forces it
Here is the call I make on every new watsonx.ai project. Start on SaaS, on the Essentials plan, in a region that satisfies your data rules. It gets you building against Granite the same day, and it hands IBM the GPU sizing, the operators, and the version matching that otherwise become your standing costs. Move to the software option only when you can name the rule that forces it: data that cannot leave your datacenter, an OpenShift and GPU estate you already pay for, or a customization SaaS will not allow. If you cannot name that rule, you do not have a reason yet.
The mistake I see most is the team that reaches for the cluster first, out of habit or a feeling that real deployments run on premises. Real deployments run wherever the requirement points, and most requirements point at SaaS. Own the platform only when owning it buys you something you actually need, and remember that every layer you take on, the cluster, the drivers, the upgrades, is a layer you now have to keep alive at two in the morning.
Before Part 5, write one sentence and keep it: where must this project’s data live. That single answer decides your deployment shape, and Part 5 takes the pricing of each shape apart, the CUH and Resource Unit meters and the tiers, so you can put a real number against the choice you just made.
References
IBM docs: Installing IBM watsonx.ai on Software Hub
IBM docs: watsonx.ai Runtime service plans
IBM watsonx.ai pricing
IBM docs: GPU requirements for models
IBM: watsonx on premises with VMware
About The Author
DrJha