A blunt claim to end on: for most teams the cheapest way to run generative AI is still a managed API, and the reason to run Red Hat AI anyway is almost never the monthly bill. That sentence would have annoyed me at Part 1, when the entire point of standing up our own inference was to stop paying per token. Twenty nine parts later the assistant runs on our own hardware, tuned on our own docs, guarded, grounded and disconnected, and I can finally say what that bought us and what it cost. This part is the verdict and the honest ledger, not a victory lap.

Where the assistant landed

Last part put the assistant behind an air gap, where nothing can phone home. Walk back the whole build for a second, because the verdict only makes sense against it. We picked a Granite size and an accelerator, installed RHEL AI on one server, tuned Granite on the company docs with InstructLab, then moved the workload onto OpenShift AI for scale, served it through the Red Hat AI Inference Server and llm-d, and finally guarded it, grounded it with RAG and put a cost ceiling on the GPU in Part 28. That is a real platform, and it took real weeks.

Versions this verdict is written against: Red Hat AI 3, RHEL AI 1.5, OpenShift AI 3.x self managed, and the Granite family up to Granite 4.0 in the model catalog, with the Inference Server being a hardened build of upstream vLLM. Names and numbers move, so treat the specifics as a snapshot and the shape as durable. One honest question hangs over all of it: a managed cloud would have given a team an endpoint in an afternoon. Was the platform worth building, and for whom.

Managed cloud stacks Red Hat AI competes with

Four managed platforms cover most of the market a Red Hat AI decision runs against. AWS Bedrock fronts many model providers behind one API on AWS. Google Vertex AI does the same on Google Cloud with the Gemini family at the center. Azure AI Foundry, formerly Azure AI Studio, sits on Azure with deep access to the GPT models. IBM watsonx is the closest neighbour, since it also ships Granite and can run on IBM Cloud or on premises, and IBM now offers Red Hat AI Inference on IBM Cloud too. Each managed option hands you an endpoint and takes the hardware, the scaling and the model updates off your plate, in exchange for your tokens crossing their boundary and your workload living inside their walls. We covered the two biggest in their own right, so lean on the AWS generative AI guide and the Azure generative AI guide rather than repeat them here.

Read that table as one line: the managed options trade your control for their convenience, and Red Hat AI trades your convenience for your control. Everything below is a way of pricing that trade.

Cost crossover between self hosted and managed inference

Here is the number that reframes the whole debate. A managed API bills a variable cost that rises with every token; a self hosted deployment pays a mostly fixed cost whether the GPU is busy or idle. So the two cost lines cross, and the crossover point, not a vibe about sovereignty, is what should decide the small cases. Put a rough floor under self hosting first: one H100 class GPU runs about 1,000 to 1,500 dollars a month reserved, and the engineering time to keep a self hosted stack patched and monitored adds roughly 750 to 3,000 dollars a month, so call the floor near 2,600 dollars a month before a single token is served. That floor barely moves with volume until you saturate the card.

Now the part that contradicts most self hosting pitches, including some of the sovereign AI marketing: self hosting to save money is usually wrong, because the fixed floor only pays off when a single GPU is nearly saturated, and most internal workloads leave it idle. A card at 6 percent utilization is the most expensive way to serve tokens ever devised. The token accounting behind this is the same discipline as the generative AI cost breakdown from the GenAI Series, just applied to hardware you own instead of an invoice you receive.

Choosing between Red Hat AI and a managed cloud

Cost sets the default, and two other forces override it: data that cannot leave your control, and a need to run the same workload across clouds without rewriting it. When either applies, the crossover math stops being the deciding factor and Red Hat AI earns its keep even below break even. The flow below is the decision I actually run in a room, and the table after it is the same logic as a lookup.

flowchart TD
  A[New GenAI workload] --> B{Data must stay in house}
  B -- Yes --> C[Red Hat AI self hosted]
  B -- No --> D{High steady volume and busy GPU}
  D -- Yes --> C
  D -- No --> E[Managed API]
  C --> F{Need frontier quality answers}
  F -- Yes --> G[Route hard prompts to a managed frontier model]
  F -- No --> H[Serve Granite locally]

Weak spots in the Red Hat AI stack

Fair is fair, so here is where the stack I just spent a series praising falls down. Granite tops out as a strong mid tier model; on the hardest reasoning and coding prompts it trails the frontier models a managed cloud hands you by default, and pretending otherwise sets a team up to be disappointed. Owning the platform means owning the pager: every OpenShift upgrade, GPU driver bump and vLLM change is now your change window, which is exactly the ops load a managed API removes. OpenShift AI is also heavy for a single small workload; standing up the full platform to serve one low volume assistant is using a freight elevator to carry a letter, and the earlier crossover math is why.

One more honest note that cuts against the tidy product diagrams: the console and tooling experience still lags the managed clouds. A polished managed studio gives a data scientist a working endpoint and a playground in minutes, while the self hosted path asks for real Kubernetes fluency before the first token. That gap is closing, and Models as a Service in OpenShift AI 3 narrows it, but do not scope a project as though the developer experience already matches a hosted console. Red Hat wraps and hardens strong upstream projects, vLLM, KServe and Kubernetes among them, and adds support and integration; it did not remove the operational weight that comes with running them yourself.

Skills worth learning after this series

If this series pointed you at self hosting, the next investment is not another model, it is the operational muscle around it. Kubernetes and OpenShift fluency is the price of entry; without it the platform runs you. Deepen vLLM tuning and quantization, since throughput per GPU is where the crossover math is won or lost, and revisit quantization from Part 22 until it is second nature. Get serious about evaluation, because a self hosted model with no eval harness is a silent downgrade waiting to ship, and the same monitoring discipline the Data Science Series uses for drift and decay in production models applies directly to a served LLM. Finally, keep your application layer provider portable, the way the AI Engineering Series argues in choosing and switching providers, so that today Red Hat AI verdict is never a one way door.

Those four skills, cluster operations, serving efficiency, evaluation and portability, are what separate a team that self hosts on purpose from one that self hosts and regrets it. None of them is Red Hat specific, which is the point: they transfer whether you land on Red Hat AI, a managed cloud, or the hybrid split most mature teams end up running.

What I would run, and what I would rent

Verdict, plainly. I would run Red Hat AI for exactly the workload this series was built around: regulated data that cannot leave the building, or a genuine multi cloud mandate where portability is worth paying for, both at volume high enough to keep a GPU busy. I would rent a managed API for everything else, and I would avoid the trap I fell into, self hosting a low volume workload for the feeling of control while a card idles at 6 percent. The model to avoid outright is the all in bet either way; the mature answer is usually hybrid, Granite on Red Hat AI for the routine and sovereign path, a managed frontier model for the hardest prompts and the long tail. Choose Red Hat AI for control and portability, choose a managed cloud for cost and convenience, and let the two tables in this part, not a vendor pitch, tell you which one this workload actually is.

References

• Red Hat AI, product overview
• Red Hat AI 3 delivers speed, accelerated delivery and scale
• Red Hat OpenShift AI, product page
• Red Hat Enterprise Linux AI, Red Hat Developer

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts