Twelve ESXi hosts were still pinned under the old platform three weeks after the last workload moved to VKS, each one carrying licence cost, patch exposure and a BOSH director that nobody logged into anymore. Migration was finished. Teardown was not, and that gap is where an estate quietly bleeds money and risk while everyone celebrates the cutover.
Delete clusters one by one with tkgi delete-cluster after clearing PodDisruptionBudgets, PVCs and retained PVs, do not lean on the tile delete all clusters errand.
TKGI leaves protected NSX-T objects behind, so clear the tier-1 routers and load balancers through the NSX API before you retire the network.
Export the Ops Manager installation and hold it at least 30 days before deleting the VM, because deletion takes the only copy of your old configuration with it.
Headline command: tkgi delete-cluster prod-vks –non-interactive –wait
Where the migration stands and what is left to retire
Every earlier part in this series moved something onto VKS. Part 22 piloted a non-production cluster, Part 23 shifted production traffic and validated it, and Part 25 built the backup and observability net that now watches the new platform. Our running estate has been the same since Part 5: three TKGI clusters named dev, staging and prod, a payments application with a web tier and a PostgreSQL database on persistent volumes, and a couple of stateless services beside it. All of that runs on VKS now. TKGI 1.18 on NSX-T sits idle.
Idle is not gone. What remains is a full platform in miniature: three empty TKGI clusters, the NSX-T tier-1 routers and load balancers they provisioned, a BOSH director tracking their VMs, an Ops Manager appliance holding the configuration, and twelve ESXi hosts committed to all of it. None of that serves a request. Every piece of it still consumes licences, still needs patching, and still widens the attack surface of an estate that has already moved on.
Here is where the common instinct goes wrong. Teams leave the old platform running for months as a safety blanket, telling themselves they might need to fail back. In practice you almost never fail back after production has run clean on the new platform for a week, and an idle TKGI does not stay a viable fallback anyway, its releases drift out of support, its certificates expire, and its snapshot of your workloads goes stale the moment you deploy anything new on VKS. A held Velero backup and an exported Ops Manager configuration are a far better insurance policy than a live platform rotting in place. Retire it deliberately instead.
Preflight and the point of no return
Teardown is destructive by design, so prove four things before you delete anything: no client traffic still reaches a TKGI cluster, a restorable backup of every migrated workload is held off-platform, the Ops Manager installation has been exported, and you have written down the hold window after which reversal is no longer possible. Skip the export and a single click on a trash can takes your entire historical configuration with it, which is a mistake you make exactly once.
# authenticate to the TKGI control plane, password read from the environment, never typed inline
export TKGI_PASSWORD="$(vault kv get -field=password secret/tkgi/admin)"
tkgi login -a api.tkgi.corp.local -u admin -p "$TKGI_PASSWORD" -k
# what is still standing
tkgi clusters
# NAME PLAN ACTION LAST ACTION K8S VERSION STATUS
# dev-vks small UPDATE succeeded 1.27.5 running
# staging-vks small UPDATE succeeded 1.27.5 running
# prod-vks large UPDATE succeeded 1.27.5 running
# export the Ops Manager installation BEFORE anything is deleted, this is your only rollback artifact
export OM_PASSWORD="$(vault kv get -field=password secret/opsman/admin)"
om –target https://opsman.corp.local –username admin –password "$OM_PASSWORD"
export-installation –output-file opsman-final-20260801.zip
# exported installation to opsman-final-20260801.zip (412 MB)
Keep the table below in front of you for the whole teardown, it is the reference artifact for this part. It sets the order of operations and, for each step, whether it can still be reversed and how long the safety copy should live. Read it as a one way ratchet: everything above a row is recoverable while that row is pending, and once you cross the Ops Manager deletion line, the old platform is gone for good.
| Order | Action | Reversible while pending | Safety copy to hold |
|---|---|---|---|
| 1 | Confirm traffic cut, backups held | Fully, nothing deleted yet | Velero backups, DNS records |
| 2 | Delete TKGI clusters | No, but workloads already live on VKS | Velero restore into a fresh cluster |
| 3 | Clear NSX-T objects, BOSH deployments | Recreatable from tile if not yet removed | NSX config export |
| 4 | Remove TKGI tile, apply changes | Reinstallable from the exported settings | Exported installation zip |
| 5 | Power off Ops Manager and BOSH director | Yes, power back on during hold | Powered-off VM plus export |
| 6 | Delete VMs, reclaim ESXi hosts | No, point of no return | Exported zip retained off-host |
Deleting the TKGI clusters cleanly
Last section proved the platform is idle and the settings are exported. This section removes the workload clusters, and the order inside a cluster matters as much as the order between them. A cluster that still holds a PodDisruptionBudget or a persistent volume set to reclaim policy Retain can leave the delete half finished, with orphaned disks and NSX objects that no longer map to anything. Clear those first.
Step 1, prepare each cluster for deletion. Log into the workload cluster, remove any PodDisruptionBudget so evictions are not blocked, delete the persistent volume claims, and re-point any PV with a Retain policy to Delete so the storage actually releases. Do the lowest risk cluster first, dev, then staging, then prod, so any surprise shows up where it costs nothing.
tkgi get-credentials dev-vks
kubectl config use-context dev-vks
# remove PodDisruptionBudgets that would block node drain
kubectl delete pdb –all –all-namespaces
# poddisruptionbudget.policy "web-pdb" deleted
# release any Retain volumes so they do not orphan
kubectl get pv -o custom-columns=NAME:.metadata.name,POLICY:.spec.persistentVolumeReclaimPolicy | grep Retain
# pvc-8b1e… Retain <- will orphan on delete
kubectl patch pv pvc-8b1e-... -p '{"spec":{"persistentVolumeReclaimPolicy":"Delete"}}'
# persistentvolume/pvc-8b1e-... patched
Step 2, delete the cluster and watch it to completion. Run tkgi delete-cluster with the non-interactive and wait flags so the command blocks until BOSH has actually torn the VMs down, rather than returning the moment the request is accepted. On the first prod run this is exactly where a large NSX-T estate bites.
tkgi delete-cluster dev-vks –non-interactive –wait
tkgi delete-cluster staging-vks –non-interactive –wait
tkgi delete-cluster prod-vks –non-interactive –wait
# the failure I hit on prod-vks, the delete stalled and then errored
# Error: timed out waiting for cluster deletion after 120s
# nsx-t: still deleting logical router ports for tier-1 pks-prod-vks-t1
# cause, the default TKGI Operation Timeout of 120 seconds is too short for a large NSX-T estate
# fix, raise it on the TKGI tile, Networking pane, TKGI Operation Timeout 120 -> 600, apply changes, then re-run
tkgi delete-cluster prod-vks –non-interactive –wait
# waiting for cluster deletion…
# delete-cluster succeeded for prod-vks (elapsed 7m42s)
Verify the control plane shows no clusters before moving on. An empty list is what green looks like here, and it is the signal that BOSH has released the cluster VMs back to vSphere.
# Name Plan Name UUID Status Action
# (no clusters found) <- all three released
Clearing NSX-T objects and BOSH deployments
Deleting a cluster does not always delete everything it created. TKGI provisions a set of NSX-T objects per cluster, a tier-1 router and logical switch for each node subnet, one per Kubernetes namespace, and one for each load balancer, and many of these are marked as protected objects owned by the TKGI principal identity. A protected object cannot be removed by an ordinary admin, which is the single most common reason an NSX-T estate stays cluttered long after the clusters are gone.
Step 3, find and clear the leftover objects through the NSX policy API. Search for anything the cluster prefix owns, then delete it with the override header that lets you remove principal-identity protected objects. Read the object list before you delete, so you remove only what belongs to the retired clusters.
export NSX_PASSWORD="$(vault kv get -field=password secret/nsx/admin)"
curl -k -u admin:"$NSX_PASSWORD"
‘https://nsx-mgr.corp.local/policy/api/v1/search?query=resource_type:Tier1%20AND%20display_name:pks-prod*’
# results: pks-prod-vks-t1, pks-prod-vks-lb-t1 <- orphaned, clusters already gone # a plain admin delete is refused curl -k -u admin:"$NSX_PASSWORD" -X DELETE 'https://nsx-mgr.corp.local/policy/api/v1/infra/tier-1s/pks-prod-vks-t1' # 403 Forbidden: The object is protected and owned by principal identity pks-nsx-t-superuser # fix, add the override header to delete a protected object curl -k -u admin:"$NSX_PASSWORD" -X DELETE -H 'X-Allow-Overwrite: true' 'https://nsx-mgr.corp.local/policy/api/v1/infra/tier-1s/pks-prod-vks-t1' # 200 OK <- object removed
Step 4, confirm the BOSH director holds no service deployments for the retired clusters. Each TKGI cluster ran as a BOSH deployment named service-instance underscore and a UUID, and a clean delete removes them, but a half finished delete can leave one behind. List them, and delete any straggler by name.
bosh -e tkgi deployments
# Name Release(s) Stemcell(s)
# pivotal-container-service-abc123 … … <- the TKGI control plane, keep for now # (no service-instance deployments) <- clusters fully released # if a straggler remained, it would be removed by name # bosh -e tkgi -d service-instance_9f2c... delete-deployment --non-interactive
Removing the tile and retiring Ops Manager
With the clusters and their network objects gone, the TKGI tile itself comes out. Step 5, remove the tile from the Ops Manager Installation Dashboard by clicking the trash can icon on the TKGI tile, then apply changes. There is one decision here that the tutorials get wrong. The tile carries a Delete all clusters pre-delete errand, and the default guidance is to let it run and clean up for you. On a large NSX-T estate that errand serializes through the same protected-object deletion that already timed out on you once, so it can hang for the better part of an hour and leave the tile in a failed delete state. Since you already deleted the clusters by hand, disable that errand under Pre-Delete Errands and let the tile removal be the fast, boring operation it should be.
Step 6, retire Ops Manager and the BOSH director, but do it in two stages separated by a hold window. Power the appliances off first and leave them powered off for at least 30 days, so any late request for a historical configuration, an audit of the old RBAC, a forgotten manifest, can still be answered by turning them back on. Only after the hold window clears do you delete the VMs and hand the datastore space back. The exported installation zip from preflight lives longer still, kept off-host as the permanent record.
Reclaiming capacity into VCF 9
The payoff for all of this is hardware. Our TKGI estate held twelve ESXi hosts, ten under the workload clusters and two under the management plane that ran Ops Manager, the BOSH director and the NSX edges. Deleting the three clusters released the first ten back to vSphere, and retiring the management plane freed the last two. Those hosts do not sit idle, they move into the VCF 9 fleet and become capacity for the Supervisor and future VKS clusters, so the migration ends by feeding its own target.
Failure lookup for the teardown
Below is the lookup I keep open during a decommission. Every row is a symptom I have hit tearing down this estate, its cause, and the fix, so a stalled delete is a page turn rather than a scramble.
| Symptom | Cause | Fix |
|---|---|---|
| delete-cluster times out at 120s | TKGI Operation Timeout too low for NSX-T scale | Raise the timeout to 600s on the tile, re-run |
| Cluster delete leaves orphaned disks | PVs set to Retain never released | Patch reclaim policy to Delete before deleting |
| NSX tier-1 delete returns 403 | Protected object owned by principal identity | Delete with the X-Allow-Overwrite header |
| Tile delete hangs in failed state | Delete all clusters errand stuck on NSX cleanup | Disable the errand, clusters were already deleted |
| No copy of old config after removal | Ops Manager deleted without an export | Always export-installation and hold it 30 days |
Decommission verdict and what comes next
My worst teardown lesson cost a day and a finding. The afternoon the last cluster came down, I deleted the Ops Manager VM in the same maintenance window, feeling efficient. Two weeks later an auditor asked for the historical UAA to LDAP group mapping and the old BOSH deployment manifests, and I had thrown away the only copy. Reconstructing what I could from memory and screenshots took the better part of a day and still left a gap in the record. I reversed that habit the same week: export the installation first, power off rather than delete, and hold for 30 days minimum before anything is destroyed. It costs a little datastore space for a month and buys back every question a decommission tends to attract after the fact.
Two verdicts for anyone at this stage. On cluster removal, delete the clusters yourself with the Delete all clusters errand disabled, and avoid the tutorial default of letting the tile clean up for you, because that errand serializes through protected-object deletion and hangs on any estate large enough to matter. On Ops Manager, export and hold, and avoid the instinct to delete the VM the moment the last workload is off, because the configuration inside it is the only record of how the old platform was wired and you will be asked for it.
What comes next is the good part. The estate is fully on VKS now, so the skills to build are the ones that platform rewards: Cluster API and the declarative cluster lifecycle that replaced BOSH, GitOps for the namespace and workload definitions you used to click through in Ops Manager, and the Day-2 upgrade and backup routines from Parts 24 and 25 as standing practice. If you also run Cloud Foundry applications, their path is Tanzu Platform for Cloud Foundry on its own timeline, a separate journey from this one. The VKS Complete Guide and the VCF 9 Complete Guide carry the ongoing operations, the NSX Complete Guide owns the network layer underneath, and the TKGI to VKS guide holds the full map of these 26 parts.
Two questions close out every decommission review. Can we keep TKGI running as a fallback for a while? Not usefully, an idle platform still consumes licences and patch cycles, drifts out of support, and holds a snapshot of your workloads that goes stale the moment you deploy on VKS, so a held backup is the better insurance. Do we delete Ops Manager if it also runs Cloud Foundry apps? No, TAS keeps Ops Manager and BOSH, so you remove only the TKGI tile and leave the director for the Cloud Foundry estate.
NSX-T shows no orphaned tier-1 routers or load balancers under the retired cluster prefixes.
The TKGI tile is removed and the Delete all clusters errand was disabled during removal.
Ops Manager and the BOSH director are powered off, exported, and inside the 30 day hold.
The freed ESXi hosts are reclaimed into the VCF 9 fleet as Supervisor capacity.
On Monday, open the reversibility table and mark exactly where your estate sits on it, then pick the next reversible step and take it. If your clusters are already gone, run the NSX search for orphaned tier-1 routers and see what is still lingering. That single command usually turns up more leftover objects than anyone expects, and clearing them is the difference between a migration that finished and one that merely stopped. With that, the estate is fully on VKS and this series is complete.
References
Broadcom TechDocs, Deleting Clusters in TKGI 1.18
VMware Docs, Deleting Tanzu Kubernetes Grid Integrated Edition
Broadcom Knowledge, TKGI End of Life and End of Support Behavior


DrJha