INFO Install complete! landed in my terminal 38 minutes after I pressed enter, and the platform lead sitting beside me asked whether it had actually built anything. His scepticism was earned. Standing up a Tanzu Kubernetes Grid Integrated (TKGI) cluster in this estate meant a BOSH director, a Tanzu Operations Manager tile, an NSX-T principal identity and a change window measured in half days. OpenShift Container Platform 4 on vSphere replaces that machinery with one binary, one YAML file and a vCenter account that can create virtual machines.

Part 11 handed us a design: two clusters instead of three, 27 virtual machines, failure domains declared on day zero. This Part installs the first of them and settles the argument every TKGI team has in week one, which is whether to let openshift-install own the virtual machines or build them yourself.

Preflight, What to Prove Before openshift-install Runs

Every failed vSphere install I have sat through failed on something a five minute check would have caught. Run these from a jump host on the machine network the cluster will live on, not from a laptop across a VPN, because name resolution and reachability differ and that difference is exactly what breaks bootstrap.

Two of those DNS records are obvious and one is not. api-int is the internal endpoint that control plane members and the bootstrap node use to reach the API server, and it resolves to the same VIP as api. Network reviewers delete it because it looks like a redundant alias of a record they already approved. It is not redundant, and the install dies at bootstrap without it. Note also that the apps entry must be a wildcard, so *.apps.ocp-prod.example.internal, rather than one record per route hostname.

IPI and UPI Side by Side

Both methods produce an identical cluster. What differs is who owns the virtual machine lifecycle afterwards, and that question lands harder on a TKGI team than on anyone else, because BOSH trained them to own every VM in the estate. On OpenShift that instinct costs you the capability you migrated for.

flowchart TD
  A[Can openshift install reach the vCenter API] -->|No| U[Choose UPI]
  A -->|Yes| B[Does the design need custom NICs or non standard VM hardware]
  B -->|Yes| U
  B -->|No| C[Will you scale or replace nodes after go live]
  C -->|Yes| I[Choose IPI]
  C -->|No| D[Choose IPI anyway and revisit at first node failure]

Alongside the trade offs sit some fixed numbers that neither method negotiates. Print this second table and pin it above the change request, because three of these rows have started arguments in every design review I have run.

Six of those UPI hours went on rediscovering disk.EnableUUID, a per virtual machine advanced setting that the vSphere CSI driver needs in order to match a Kubernetes volume to a VMDK. Set it to FALSE on a hand built template and everything installs perfectly, then persistent volume claims sit in Pending three weeks later when somebody finally migrates a database. Part 9 covered the storage side of that in detail, so this Part only flags where it bites during installation.

Installing with IPI, Step by Step

Step 1. Render install-config.yaml from a template

Credentials belong in the environment, not in a file that ends up in a ticket attachment. Keep a template under version control with placeholders, render it at run time, and remember that openshift-install consumes install-config.yaml when it creates manifests, so back it up first. Failure domains come straight from the Part 11 design, and declaring one now costs nothing while retrofitting zones after the control plane is placed costs a rebuild.

Step 2. Create manifests, then run the install

Splitting manifest generation from the install gives you one chance to inject anything the installer cannot express, for example a custom certificate authority bundle or a chrony MachineConfig for a time source your security team mandates. Skip that step and you are editing a running cluster instead.

Step 3. Take the kubeadmin password out of the install directory

ocp-prod/auth holds a kubeconfig with cluster admin rights and a plaintext kubeadmin password. Move both into your secrets manager the same hour you finish, and plan to delete the kubeadmin user once an identity provider is wired, which is Part 15. Losing the rest of the install directory is worse than losing the password though, because metadata.json in there is what makes a clean teardown possible.

I prefer oc over kubectl throughout this series for one reason: oc understands OpenShift objects such as Route, Project and SecurityContextConstraints, and it carries subcommands like oc adm and oc debug that have no kubectl equivalent. Anything kubectl can do, oc does identically.

Verification and Rollback

Install complete is not the same as healthy. Four checks separate a cluster you can hand over from one that will page somebody at 03:00. Green looks like every cluster operator Available and neither Progressing nor Degraded, every node Ready, zero problems from the vSphere Problem Detector Operator, and a MachineSet whose ready count matches its desired count.

That last object is the whole argument in four lines of output. A MachineSet exists, it knows the vSphere folder, datastore, port group and template, and oc scale machineset ocp-prod-9k4mt-worker-0 –replicas=6 adds two nodes without anyone opening vCenter. On a UPI build that object is absent, and adding a node is a clone, a boot and a round of CSR approvals.

Rollback during installation means destroy and retry, and IPI makes that cheap enough to be worth doing rather than debugging in place. openshift-install destroy cluster reads metadata.json from the install directory and removes every virtual machine, folder and tag it created, including the bootstrap node. Guard that directory. Without it you are deleting objects by hand in vCenter, matching them on the random infrastructure ID embedded in each VM name.

Install Failures and Remediation

Seven failures account for nearly every stalled vSphere install I have been handed. Keep this table beside the runbook, because each row maps a literal error string to a cause you can act on in minutes rather than an afternoon of guessing.

Two of those rows exist only on UPI, which is a cost worth naming out loud. Here are the commands behind them, neither of which an IPI operator ever types.

Field Note from a Bootstrap That Never Completed

First attempt on this estate died at 30 minutes with the timeout above. Everyone in the room blamed the installer, because in TKGI a stalled deployment usually meant a BOSH task you could tail and reason about. I gathered the bootstrap bundle, grepped for api-int, and found a DNS lookup failure on a record I had personally requested three weeks earlier. Network review had removed it, with a comment that a second name resolving to the same address was noise.

Cost of that lesson: two failed installs, one destroy that took 2 minutes 14 seconds, roughly three hours of my afternoon, and a stalled change window. Adding the record took the DNS team four minutes. Second attempt completed in 38 minutes and has not been reinstalled since.

Second thing I got wrong, and this one is more embarrassing. We had originally planned a UPI build, on the reasoning that a regulated estate should never grant an installer write access to vCenter. Security agreed with us for eleven days while the privilege request sat in a queue. Then somebody asked the obvious question, which was what happens when a worker node dies at 02:00 in month four. On UPI the answer is a human clones a template. On IPI the answer is a MachineHealthCheck deletes the Machine and the MachineSet builds a replacement. We reversed the decision, and the same privileges got approved in two days once the request said node self healing instead of installer automation.

Pick IPI for This Estate, and Reserve UPI for One Case

Verdict: install both clusters from the Part 11 design with IPI. Common advice inside VMware shops points the other way, because owning every virtual machine feels like operational maturity and because a BOSH background makes an opinionated installer feel like a loss of control. On OpenShift that reasoning inverts. Node lifecycle is a platform capability delivered through the Machine API, and choosing UPI switches it off at install time to buy flexibility that a standard three tier estate will never use. Reserve UPI for exactly one case: the installer genuinely cannot reach the vCenter API, or the design needs VM hardware the installer does not model, such as a second NIC on a storage VLAN.

A clean result looks like this. Every cluster operator Available, Progressing False, Degraded False. Seven nodes Ready on v1.31.7. Zero problems from the vSphere Problem Detector Operator. One MachineSet with ready matching desired. Kubeadmin credentials moved out of the install directory, and the install directory itself archived somewhere you can find it in a year. Cluster left running and non degraded for 24 hours so the first certificate rotation completes before anyone declares victory.

Monday action, and it takes ten minutes on your own estate: run dig against api, api-int and the apps wildcard for the cluster name you intend to build, from a host on the target machine network. If api-int does not resolve, you have just found the failure that would have cost you an afternoon, and you found it before booking a change window. Part 13 takes the cluster this Part installed and turns to cluster networking, OVN-Kubernetes, Routes and load balancing, which is where the NSX-T habits from Part 8 finally meet running configuration.

References

Red Hat, Installer-provisioned infrastructure, Installing on VMware vSphere, OpenShift Container Platform 4.18

Red Hat, User-provisioned infrastructure, Installing on VMware vSphere, OpenShift Container Platform 4.18

Red Hat, Using the vSphere Problem Detector Operator, OpenShift Container Platform 4.18

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts