Do we need OpenShift for this? A platform lead asked me that two days after we picked the model in Part 2, and it is the right question asked slightly wrong. We had one Granite model, one grounded assistant, and one box in mind. Reaching for a Kubernetes cluster on day one would have added three machines and a month of plumbing to serve a workload that fits on a single server. Reaching for nothing but a bare operating system would have meant rebuilding, by hand, the exact stack Red Hat already packages.

Three product names sit close enough to blur together: Red Hat Enterprise Linux, RHEL AI, and OpenShift AI. They are not three tiers of the same thing and they are not interchangeable. One is an operating system, one runs a single model on one server, and one runs many models across a cluster. This part draws the lines between them and settles which one our support assistant belongs on right now, and which one it grows into later.

Three products with three different jobs

Start with what each product is in one sentence, because the names invite confusion. Red Hat Enterprise Linux is a general purpose enterprise operating system, the substrate that everything else runs on. RHEL AI is a specialised build of that same operating system, delivered as a bootable image, with a foundation model platform bolted in so one server can develop, tune and serve a Granite model. OpenShift AI is a full machine learning and generative AI platform that runs on an OpenShift cluster, coordinating many models and many teams with pipelines, a registry and cluster wide serving.

Under the hood, RHEL AI is literally RHEL. Red Hat builds it using image mode, an approach that packages the operating system as a bootable container image, or bootc image in one clause, so it deploys and updates like a container rather than a hand configured server. OpenShift AI sits a layer higher again: it assumes you already run OpenShift, which is Red Hat Kubernetes, and it adds the AI specific machinery on top. So the relationship is not three rungs of one ladder. It is an operating system, a single node product built from that operating system, and a cluster product built on Kubernetes. Part 1 sketched how these fit; here we choose between them.

Where plain RHEL stops being enough

Plain Red Hat Enterprise Linux gives you a great deal, just none of it specific to running a model. A subscription buys a hardened operating system, a 10 year life cycle, live kernel patching, 24×7 support and legal cover through the Open Source Assurance program. What it does not buy is a model, a tuning tool, or an inference server. On a bare RHEL box you install the GPU drivers, the CUDA or ROCm stack, a Python environment, vLLM, and a model you sourced yourself, and you wire them together and keep them patched. That is a legitimate path if you want total control over every layer, and plenty of teams run inference exactly this way.

Cost of that control is that you rebuild, and then own, the precise stack RHEL AI already assembles and supports. When a driver update breaks your serving engine at 2am, plain RHEL support will help with the operating system, not with why vLLM stopped loading your model. For a team whose job is to ship an assistant, not to maintain an inference platform, bare RHEL is usually a false economy. It is the right base when you have strong opinions about the stack and the staff to hold them, and the wrong one when you just want a model answering questions.

RHEL AI on a single server

RHEL AI takes that assembly problem off your plate for one server. It ships as a bootable image that already contains the Granite models under Apache 2.0, a supported and life cycled build of InstructLab for tuning, PyTorch, and accelerators tuned for NVIDIA, Intel and AMD. You boot the image, activate a subscription, pull a model, and serve it, without hand building the stack underneath. Everything the assistant needs for Phase 2, tuning Granite on the company docs and serving it, lives on that one box, which is exactly why Part 2 sized a single card rather than a rack.

Its ceiling is the single node. RHEL AI has no cluster scheduler, so it cannot spread one model across many machines or pack many models onto shared hardware. It has no multi tenancy, so two teams on one box share everything and isolate nothing. It has no pipeline engine, model registry or built in monitoring, so the machine learning operations you would want for a fleet are simply absent. None of that is a defect. RHEL AI is deliberately a single server product, and for one model in front of one team it is the shortest honest path from nothing to a served answer.

OpenShift AI across a cluster

OpenShift AI answers a different question: how a whole organisation runs many models across shared infrastructure. It is a platform on top of OpenShift, Red Hat Kubernetes distribution, and it adds machine learning and large language model operations, data science pipelines, a model registry and versioning, distributed training, and cluster wide serving. Serving runs on KServe, an upstream Kubernetes project that in 2025 became a CNCF incubating project, which turns a model into an InferenceService, a Kubernetes object that describes a deployed model, in one clause, and handles autoscaling, storage and networking around it. GPU scheduling, time slicing and multi tenancy through projects and resource quota let many teams share the same accelerators without trampling each other.

All of that power arrives with a cluster underneath it. OpenShift AI needs an OpenShift cluster, which at the barest minimum is a three node compact cluster and is more typically five nodes or more, plus the operators for Service Mesh and Serverless that KServe leans on in its advanced serving mode. This is the same pipeline and lifecycle discipline the Data Science series covers in CI and CD for machine learning pipelines and in serving machine learning models, now delivered as a supported product rather than assembled by hand. That is real value once you have many models and many teams, and real overhead when you have one of each.

Matching a platform to project shape

Last part the assistant got a model, Granite H-Tiny at FP8 on one card. This part gives it a home. The choice is not about which product is most capable, because OpenShift AI wins that on paper every time. It is about the shape of the project: how many models, how many teams, and how much platform you are willing to operate to serve them. The numbers below are the ones I use to keep that honest, because a platform decision made on capability alone always over buys.

Where the OpenShift AI first instinct goes wrong

Two beliefs push teams onto a cluster too early. First, the slideware positions OpenShift AI as the enterprise AI platform, which reads as an instruction to start there, and for a single grounded assistant that instruction is wrong. On a fresh cluster the cluster becomes the project for the first month, and the model waits. Second, Red Hat own phrasing that RHEL AI can be deployed and managed with OpenShift AI gets misread as RHEL AI needs OpenShift AI. It does not. RHEL AI boots and serves on one machine with zero Kubernetes, and the OpenShift AI integration is an option for when you scale out, not a prerequisite for starting.

I have watched this cost real time. A fintech team I advised put its first assistant, a single 7B model, straight onto a fresh four node OpenShift cluster because a slide said OpenShift AI was the enterprise path. Three weeks went into GPU operator versions, Service Mesh, Serverless and an InferenceService that would not scale from zero, and not one token had reached a user. We moved the same model onto a single RHEL AI box on a Thursday afternoon, and it answered its first grounded question before we went home. The cluster was the right destination and the wrong starting line, and the reversal cost them the better part of a month they never got back.

Choosing RHEL AI for the support assistant

For the assistant this series builds, the platform is RHEL AI, on the single server from Part 1, running the Granite size from Part 2. One model, one team, one box: every number in the decision table reads one, so the single node product is not a stepping stone we are settling for, it is the right tool for the job in front of us. Plain RHEL loses because we would rebuild the stack Red Hat already supports, and OpenShift AI loses today because it would trade an afternoon for a month and hand us a cluster to operate before we have a single answer to show. When Phase 3 brings a second team, a second model or a scale the box cannot hold, we move to OpenShift AI on purpose, carrying a tuned model and a proven workload with us. That upgrade is the same cost and scale reasoning the Data Science series works through in GPU cost, scale and sizing for machine learning.

Your action for Monday is to write down three numbers for your own project, how many models, how many teams and how many boxes you will run in the next six months, and let those numbers pick the platform instead of the marketing. If they are one, one and one, commit to RHEL AI and stop shopping. Part 4 takes the next question head on: where Red Hat AI actually runs across the hybrid cloud, from a datacentre box to a cloud instance to the edge, now that you know which product you are deploying.

References

• Red Hat blog, RHEL vs RHEL AI, what RHEL AI adds and the bootable image model
• Red Hat Customer Portal, the difference between RHEL AI and OpenShift AI
• Red Hat OpenShift AI product page, the cluster platform, MLOps and serving
• Red Hat blog, KServe joins CNCF as an incubating project, the serving layer under OpenShift AI

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts