, ,

TKGI to VKS Is a Migration, Not an Upgrade (TKGI to VKS Series, Part 3)

TKGI and VKS differ at the control plane, machine lifecycle, network and identity, so there is no in place upgrade path. Here is why this move is a migration, and what that word changes about budget, risk and rollback.

TKGI to VKS Series · Part 3 of 26

A platform director asked me last quarter why the jump to VKS could not be an in place upgrade, the way his team moved from vSphere 7 to vSphere 8 over a single weekend. It is the most reasonable question anyone raises about this move, and the honest answer is the reason this whole series exists. TKGI and VKS are not two versions of one platform. They are two platforms, and you do not upgrade across that gap, you migrate. This part is the argument behind that sentence, laid out so you can defend it to a change board without hand waving.

Who this is for: A platform engineer, SRE, Kubernetes operator or VMware admin who already accepts that TKGI is winding down and now needs the reasoning behind the word migration before defending a plan to a change board or a budget owner. Part 1 introduced the move and Part 2 set the two architectures side by side, so this part is the argument you will actually have to make. Terms on first use: TKGI is Tanzu Kubernetes Grid Integrated, formerly Enterprise PKS; VKS is the vSphere Kubernetes Service in VMware Cloud Foundation 9; Supervisor is the Kubernetes control plane built into vSphere; BOSH is the release and virtual machine lifecycle engine TKGI inherited from Cloud Foundry; Ops Manager is the tile based console that installs BOSH managed products; Velero is the open source backup and restore tool for Kubernetes.
Key takeaways: No code path turns a BOSH managed TKGI cluster into a Supervisor managed VKS cluster, because the two differ at the control plane, the machine lifecycle, the network model and the identity store, and each difference alone rules out an in place conversion. Workloads are portable, platforms are not, so you copy the workloads with Velero and rebuild the platform underneath them. Two platforms run at once for weeks or months, and that overlap is the design, not a failure. It also buys the one thing an upgrade cannot, a per wave rollback with the old platform still live.

Four layers that rule out an in place upgrade

An upgrade rewrites one platform in place and hands the same workloads back on a newer version. A migration lifts workloads off one platform and lands them on another that never shared a spine with the first. TKGI and VKS diverge at four layers, and any one of them on its own would be enough to break a conversion. Stacked together they make the idea of an upgrade path a category error, not a missing feature.

Start with the control plane. TKGI provisions clusters through BOSH, driven by Ops Manager tiles, so the source of truth for a cluster lives in BOSH deployments and manifests that sit outside vSphere. VKS provisions clusters through Supervisor, a Kubernetes control plane Broadcom builds directly into vSphere and runs on the ESX hosts themselves. One reconciles cluster state from BOSH, the other from vCenter and Supervisor. No setting flips a cluster from being owned by the first to being owned by the second, because nothing in vSphere was ever watching the BOSH managed cluster to begin with.

Machine lifecycle is the second layer. Under TKGI, BOSH owns virtual machine creation, health checks and repair through stemcells and a release model that predates Kubernetes. Under VKS, node virtual machines come from Cluster API and the vSphere VM Service, expressed as Kubernetes objects a cluster class shapes. These are different engines building and healing the nodes, not two dialects of one engine. A node that BOSH resurrects on failure and a node that Cluster API reconciles from a machine template have nothing in common except that both run kubelet.

Networking is the third layer, and it surprises people most. TKGI wires pods through NSX-T with the NSX Container Plugin, and leans on the NSX-T load balancer for services. VKS clusters run Antrea as the container network plugin, over an NSX VPC or a vSphere Distributed Switch, with load balancing from the Foundation Load Balancer or NSX Avi. Antrea is the default container network plugin on VKS; the NSX Container Plugin, or NCP, is the integration that bound TKGI pods to NSX-T. Pod networking is rebuilt here, not repointed, and the load balancer in front of your services changes product. If networking is where your anxiety sits, the NSX Series owns that ground in depth.

Identity and cluster state close the list. TKGI authenticates through UAA backed by LDAP, while VKS uses vCenter SSO with Pinniped, which means your role bindings do not copy across untouched. And the cluster state itself, the etcd contents behind a running TKGI cluster, is not a portable artifact you can pour into a Supervisor cluster. Read the table below as the whole argument on one screen, because every row is an independent reason the upgrade button was never coming.

LayerTKGI mechanismVKS on VCF 9Why in place conversion breaks
Control planeBOSH plus Ops Manager tilesSupervisor built into vSpheredifferent reconcilers, neither can adopt the other
Machine lifecycleBOSH stemcells and releasesCluster API and VM Servicenodes are built and healed by unrelated engines
Pod networkingNSX-T with NCPAntrea over NSX VPCthe network plane is rebuilt, not repointed
Load balancingNSX-T load balancerFoundation Load Balancer or NSX Aviservice front end changes product and config
Identity and RBACUAA with LDAPvCenter SSO and Pinnipedrole bindings do not copy across untouched
Cluster stateetcd behind a BOSH clusteretcd behind a Supervisor clusterstate is not a portable artifact between the two

One clause on a platform people confuse with this: if you also run TAS, the Cloud Foundry application platform once sold as TAS for VMs, it is out of scope here and its successor is Tanzu Platform for Cloud Foundry, a separate path you should not fold into a VKS plan. This series moves Kubernetes clusters from TKGI to VKS, nothing else.

Where a conversion tool would plug in, and why none does

People reach for a converter because most hard migrations eventually grow one, so the absence feels like a gap Broadcom forgot to fill. It is not a gap. A converter would have to read the desired state of a cluster from BOSH and reconcile it into Supervisor while preserving running pods, attached volumes and network identity, and those desired states do not describe the same kind of object. There is no common intermediate form to translate through, so there is nothing for a tool to sit in the middle of.

Draw the line that actually matters: your workloads are portable, your platform is not. A Deployment, a Service, a ConfigMap, a PersistentVolumeClaim are Kubernetes objects, and Kubernetes objects move between conformant clusters. That portability is exactly what Velero uses, backing objects up from the source cluster and restoring them onto the target, with volume data carried alongside. What does not move is the substrate underneath, the control plane, the node engine, the network plane and the identity store. So the plan writes itself once you accept the split. Copy the portable half, rebuild the fixed half, and never try to drag the substrate across. Part 17 lives inside Velero for exactly this reason, and the VKS Series covers the target platform you are rebuilding onto.

flowchart TB
  subgraph No conversion path exists
    T1[TKGI cluster on BOSH] -.->|no converter| V1[VKS cluster on Supervisor]
  end
  subgraph Migration path that works
    T2[TKGI cluster] --> K[Velero backup] --> R[Velero restore] --> V2[VKS cluster]
  end
The dotted arrow is the path that does not exist. The solid path copies portable objects with Velero while the platform underneath is rebuilt separately.

Coexistence is the design, not a workaround

Because you rebuild the platform and copy the workloads across, both platforms are live at the same time for the length of the project, which for a real estate is weeks to months rather than a maintenance window. Teams new to this read the overlap as a smell, a sign the plan is inefficient. It is the opposite. Coexistence is the mechanism that makes the move safe, because it is what lets you validate each workload on VKS while the same workload still serves users from TKGI.

Plan for the coexistence window as a real line item, not an afterthought. You need enough VCF 9 capacity to stand VKS up while TKGI still carries production, which for a mid size estate means running with roughly 20 to 30 percent capacity overlap during the busiest migration waves. That overlap ends when the last wave is proven and you reclaim the TKGI hardware, and not a day before. A plan that assumes you can free the old capacity on day one has quietly turned a migration back into a big bang cutover, and it will fail the first time a restored workload misbehaves and there is nowhere to fall back to.

Moving a production estate, expectation versus realityCalendar time an upgrade mindset assumes, against how long a wave migration runs012 weeks24 weeksa weekend4 to 6 monthsUpgrade mindsetWave migrationWaves and validation set the schedule, not a single install window
Planning figures for a mid size estate. The exact length depends on workload count and how many stateful apps you carry, but the shape holds, months of waves rather than one window.

How the migration word rewrites budget and risk

Here is where budget owners trip, and where the obvious read is wrong. On paper a migration looks more expensive than an upgrade. You pay for overlap capacity, for the Velero tooling and object storage behind it, and for a parallel run where two platforms draw power and licences at once. An in place upgrade appears to cost a weekend of labour and nothing else. So the instinct is that the upgrade is the cheaper, safer default, and the migration is a costly detour. For this move the instinct is backwards on both counts.

Risk tells the same story from the other side. An in place upgrade, if one existed, would be a big bang, every cluster and workload crossing at once on a single night with one rollback plan covering all of it. A migration spreads risk across waves, each one small, each one validated before the next begins, and each one reversible on its own. Five workloads migrated across five weekends carry a fraction of the blast radius of fifty workloads crossing in a single window. When budget people see only the capacity line, they miss that the migration bought a lower risk profile for the money, not a higher one. That is the number worth putting in front of them, blast radius per event, not total capacity consumed.

Contrarian take: The overlap capacity that makes a migration look pricier than an upgrade is not waste, it is the rollback insurance an upgrade never gives you. You are not paying twice for one platform, you are paying once for the new one and once for a live fallback you can retreat to at any wave boundary. Cut that overlap to save money and you have bought a big bang cutover by accident, which is the single most expensive way to move a Kubernetes estate.

Rollback as a first class plan

In an upgrade, rollback is the scariest slide in the deck. You are restoring a control plane from a snapshot taken minutes before, hoping state has not drifted, and praying the workloads come back the way they went in. In a migration, rollback is almost boring, which is the whole point. Traffic still points at TKGI until you deliberately cut it over, and you keep the source cluster intact until the target has passed validation. If a restored workload misbehaves on VKS, you have not lost anything, because the live copy never stopped serving from the old platform. Backing out is a decision, not an emergency.

That property is worth designing around from the first wave. Cut traffic over only after the target proves itself, keep the source warm through a soak period, and reclaim old capacity last. Do that and rollback stays a first class option through the entire project rather than a panic move reserved for the worst night.

War story: A change advisory board booked a four hour maintenance window for what they had written up as a VKS upgrade, expecting the platform to return on the far side the way a vSphere jump does. The first stateful application alone, a web tier with a PostgreSQL volume, took the better part of two days to back up, restore, reattach and validate on the target. We stopped, rebooked the work as a migration wave with the app still serving from TKGI throughout, and lost nothing the second time because traffic never left the old platform until the new one was proven. Four hours planned, two days of real work, zero downtime once we stopped pretending it was an upgrade.

Reflexes to unlearn before you draw a timeline

Twenty years of vSphere upgrades install a set of reflexes, and every one of them misfires on this move. The table below is the reference artifact of this part, a short list of the assumptions that quietly wreck a plan and what each one should become. Pin it above your desk and check any migration draft against it, because most bad plans I review fail on exactly these six lines.

Upgrade reflexWhy it misleads hereDo this instead
Book one maintenance windowthere is no single install to runschedule waves, one workload group each
Roll back with a snapshotno control plane snapshot spans platformskeep the source cluster live until validated
Platform returns on the new versionthe old platform is not upgraded, it is retiredstand VKS up beside TKGI, then drain it
Workloads never noticepods are recreated on a new clustertest each app on VKS before cutting traffic
Budget a weekend of laboura real estate takes months of wavesfund overlap capacity and parallel run
Freeze, then cut over oncea single cutover maximises blast radiusshift traffic per workload, reversibly

Plan two platforms side by side, then drain one

My recommendation is plain and it governs everything that follows in this series. Treat this as a migration project from the first planning meeting, draw two platforms rather than one, and budget the coexistence overlap as rollback insurance instead of waste. The teams that struggle are the ones who carried an upgrade mindset into a migration, waited for a converter that was never coming, and met their hardest workloads under deadline pressure with no fallback left. You avoid all of that by accepting the framing now, while both platforms can still run together, and by starting the low risk waves early to build the muscle before the difficult stateful apps arrive.

Do this on Monday: Draw your estate as two platforms on one page, TKGI on the left with its clusters and NSX-T, VKS on the right on VCF 9, and mark the single lowest risk workload as wave one. Verdict: commit to the migration framing now and budget the coexistence overlap as rollback insurance, not waste. Avoid the in place upgrade plan entirely, there is no converter to wait for, and every hour spent hunting one is stolen from the wave one you could scope today.

Next part turns from why to when, with the business case, the lifecycle drivers and a phased timeline you can put in front of a budget owner. Bring the two platform drawing, because the timeline hangs off it.

TKGI to VKS Series · Part 3 of 26
« Previous: Part 2  |  Guide  |  Next: Part 4 »

References

About The Author


Discover more from Journal of Intelligent Infrastructure

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure

Subscribe now to keep reading and get access to the full archive.

Continue reading