Run tkgi clusters against an estate you think you know and the returned count is often the least interesting thing on the screen. On the running estate it came back with three clusters, dev, staging and prod, exactly matching the internal wiki, and that tidy agreement is what almost sank the migration eight weeks later. A cluster list tells you how many Kubernetes control planes exist. It says nothing about the 400 GB persistent volume sitting in staging that no runbook mentioned. Part 4 turned the business case into a phased timeline, and this part fills the first box of Phase 2 with real data pulled from your own clusters.

Prerequisites and preflight checks

Discovery is read only, but read only against the wrong context still burns a day. Prove four things before a single inventory command runs. You can reach the TKGI API and hold a valid token. Your kubectl points at a scratch kubeconfig, not your daily driver. The BOSH CLI is reachable from the Ops Manager VM, because BOSH diagnostic commands only run there. And your account carries cluster admin scope in UAA, or the workload listings come back half empty with no error to explain why.

Point KUBECONFIG at a throwaway file for the whole exercise, so nothing here can overwrite the context you use for daily work. That single habit is the fallback for a read only task. If a command targets the wrong cluster you delete the scratch file and start again, and your real kubeconfig never moves. Confirm your tool versions first, because command output and flags drift between releases and this series pins to specific ones.

Cluster inventory with the TKGI CLI

Start where the platform makes it easy, then distrust the result. tkgi login authenticates against UAA or LDAP and stores a token, tkgi clusters lists what exists, and tkgi cluster NAME –details opens up the plan, the Kubernetes version and the NSX addressing for one cluster. This is the pass everyone runs and the pass most teams stop at. Read the API address from an environment variable rather than pasting it, so no credential or endpoint ends up in your shell history.

Three clusters, three plans, all healthy. If instead of a table you get a certificate error on the token endpoint, the TKGI API certificate has aged out, and that is a finding, not a nuisance. A platform whose own API cert is expiring is a platform where cluster certs are probably close behind, so note it and move on to the certificate check below.

Prod runs three control plane nodes and six workers on the production plan at Kubernetes 1.27.6. Record the Kubernetes version against upstream support dates, because a cluster two minor versions behind changes how you sequence its wave. tkgi certificates with -d 90 lists anything expiring inside 90 days, and on this estate one cluster cert sat 12 days out, which reorders your whole schedule if you find it late.

Control plane inventory from BOSH and Ops Manager

BOSH sees the VMs the TKGI CLI abstracts away. Each cluster is a BOSH deployment named service-instance_ followed by its UUID, so the prod cluster UUID from the last pass maps straight to a deployment name. From the Ops Manager VM, bosh deployments lists them and bosh -d NAME vms shows the master and worker VMs with their process state, availability zone and IPs. This is where a cluster the TKGI CLI reports as succeeded reveals a worker stuck in a failing state, or a stemcell that drifted behind the rest. NSX-T addressing detail that shows up here is covered in depth in the NSX Complete Guide, so record the profile name and move on rather than re deriving it.

A worker in failing state on prod is exactly the kind of thing the cluster pass hides behind a succeeded status. Fix it or note it before you plan that cluster into a wave, because a node that cannot drain cleanly turns a routine migration into an incident. Cross check the deployment count against the cluster count from the first pass, they should match, and a mismatch means a cluster exists in one view but not the other.

Workload, storage and identity inventory with kubectl

Now the pass that matters. tkgi get-credentials writes a kubeconfig context, then you enumerate namespaces, workloads and above all persistent volumes. A stateless pod migrates in an afternoon. A bound 250 GB PostgreSQL volume can own a week of Part 19, so the storage listing is the row set you will keep returning to. Always pass –all-namespaces, because a single namespace view is the fastest way to convince yourself an estate is smaller than it is.

Sort persistent volumes by capacity and the picture reorders itself around the biggest ones. On this estate the largest volume was not attached to any running workload at all, a 400 GB PV in Released state whose namespace had been deleted months earlier, kept alive by a Retain reclaim policy and still pinned to a real datastore. Identity is lighter here, a quick kubectl get clusterrolebindings tells you which UAA or LDAP groups map to cluster roles, and the deeper RBAC assessment belongs to Part 9. Chart the storage before you trust the cluster list, because storage is where the estate hides its weight.

Verification and the estate inventory sheet

Verification for a discovery run is not a green pod, it is a completed sheet with the three passes reconciled against each other. Cluster count from tkgi clusters should equal deployment count from bosh deployments should equal the number of contexts in your kubeconfig. When those three disagree, you have found something worth chasing. What green looks like here is plain: every cluster row has a Kubernetes version and a plan, every persistent volume has a size, a StorageClass and a bound namespace or a flag if it has none, and every cluster cert clears 90 days. The table below is the reference artifact of this part, the estate inventory sheet, and it is the thing you carry into Part 6 to plan waves.

flowchart TB
  A[Preflight, verify CLI versions and a valid token] --> B[Cluster pass, tkgi clusters and cluster details]
  B --> C[Control plane pass, bosh deployments and vms]
  C --> D[Workload pass, kubectl nodes pods and namespaces]
  D --> E[Storage pass, kubectl get pv and pvc all namespaces]
  E --> F[Record every row in the estate inventory sheet]

Common failures during discovery

Every one of these bit someone on a real inventory pass, and none of them throws a helpful message on its own. Keep this lookup next to the sheet, because half the time the failure is not a broken cluster, it is a token, a context or a cert quietly getting in the way of a true reading.

What a clean inventory looks like

Before you close Phase 2, the sheet should pass a short end state check. Every cluster has a plan, a Kubernetes version and a worker count. Every persistent volume has a size, a StorageClass, a reclaim policy and either a bound namespace or a flag saying it has none. Every cluster certificate clears 90 days, and any that does not is already on a remediation ticket. Registry inventory names each Harbor project, its image count and any replication rules, because images move in their own wave and a project nobody claims is a project nobody migrates. Node sizing is recorded per cluster, worker count, instance type and kernel, so capacity planning for the VKS target starts from measured numbers rather than a guess. When all six of those hold, the sheet is done, and not a moment before.

• Cluster count reconciles across tkgi clusters, bosh deployments and kubeconfig contexts
• Every PV row carries size, StorageClass, reclaim policy and bound namespace
• Released or orphaned volumes are flagged and traced back to a datastore
• Every cluster certificate clears 90 days, with exceptions ticketed
• Harbor projects, image counts and replication rules captured
• Node sizing recorded per cluster for VKS capacity planning

Do I inventory every namespace, even system ones? Yes, but tag them. System namespaces like kube-system come across as part of standing up the VKS cluster, not as workload waves, so record them and mark them platform owned. What you are hunting for is the application namespace nobody remembers, the one running a single pod against a large volume. Those are the rows that move your schedule.

Can I skip the BOSH pass if the TKGI CLI shows everything healthy? No. A succeeded status from tkgi clusters reports the last control plane operation, not the live health of every VM. That BOSH pass is the only place a worker stuck in a failing state, or a stemcell that never finished updating, shows itself. On a migration you want that surprise during inventory, not halfway through a wave.

How current does the sheet need to be? Inventory drifts, so treat it as a dated snapshot and re run the storage pass right before each wave. A volume that read 120 GB at assessment can be 180 GB by the time its wave arrives, and namespaces appear or vanish in the weeks between. These commands are cheap to re run, and a stale sheet is worse than none because it reads as authoritative while quietly lying to you.

Start the inventory from storage, not from cluster counts

Tutorial order, and the order the TKGI CLI nudges you toward, is clusters first and then drill down. Invert it. Cluster counts are the cheapest fact in the estate and the least likely to hurt you, since a control plane either exists or it does not. Persistent volumes are the expensive fact, the one that sets the length of your hardest wave and the one most likely to be undocumented, orphaned or mis sized. Run all four passes, but treat the storage listing as the deliverable and the cluster list as the cover page. Teams that inventory storage first walk into wave planning knowing where the weight sits. Teams that sign off on a reconciled cluster count meet that weight later, usually at the worst possible moment.

References

• Broadcom TechDocs, TKGI CLI reference
• Broadcom TechDocs, using the BOSH CLI from the Ops Manager VM
• Broadcom TechDocs, retrieving cluster credentials and configuration

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts