, ,

TKGI vs VKS Architecture Compared, Component by Component (TKGI to VKS Series, Part 2)

TKGI and VKS set side by side, control plane, cluster lifecycle, networking, storage and identity, so the architecture gap behind the migration is concrete before you design the VCF 9 target.

TKGI to VKS Series · Part 2 of 26
Key takeaways: VKS runs its Kubernetes control plane, the vSphere Supervisor, inside vSphere on the ESX hosts. TKGI runs its control plane as a set of BOSH managed virtual machines that Ops Manager installs beside vSphere. A cluster create request on TKGI passes through five control components before vSphere provisions a node; on VKS it passes through three. Networking moves from NSX-T with the NSX Container Plugin to Antrea over an NSX VPC, and identity moves from UAA to vCenter SSO with Pinniped. Nothing copies across untouched, which is why this is Part 2 of a migration, not a diff of two config files. Versions referenced: VCF 9.0 and 9.1, TKGI 1.18 through 1.20.
Who this is for: A platform engineer, SRE or VMware admin who has read Part 1 and wants the architecture laid out box for box before committing to a design. No VCF 9 build experience assumed. Terms on first use: Supervisor is the Kubernetes control plane built into vSphere; BOSH is the release and machine lifecycle engine TKGI inherited from Cloud Foundry; NCP is the NSX Container Plugin that wires TKGI pods into NSX-T; Antrea is the default container network plugin on VKS; VM Service is the Supervisor component that builds cluster virtual machines; CSI is the Container Storage Interface; NSX VPC is the virtual private cloud construct NSX exposes to a Supervisor.

Draw both platforms on a whiteboard and count the boxes that have to be running before a single Kubernetes cluster exists. On TKGI you count at least four dedicated management virtual machines, Ops Manager, the BOSH Director, the TKGI API and the TKGI database, before you even reach the NSX-T managers and edges underneath them. On VKS you count the control plane that already ships inside vSphere, and then you stop counting. That gap in box count is not trivia. It is the reason the two platforms cannot be diffed and reconciled, only stood up separately and drained one into the other. This part walks the architecture component by component, so the differences behind the migration stop being slogans and start being design decisions you can defend.

Two control planes, two philosophies

Start with the piece that defines everything else, the control plane, because that is where TKGI and VKS stop resembling each other. On TKGI the control plane is a set of virtual machines that live outside vSphere and are built by BOSH, the release and machine lifecycle engine TKGI inherited from Cloud Foundry. Ops Manager, the tile based console, installs BOSH first, then the TKGI tile stands up a TKGI API virtual machine and a TKGI database virtual machine, and an On Demand Service Broker waits to translate cluster requests into BOSH manifests. Every one of those pieces is a machine you run, monitor and patch, and none of them is part of vSphere.

VKS inverts that arrangement. Its control plane is the vSphere Supervisor, a Kubernetes control plane that Broadcom builds directly into vSphere and runs on the ESX hosts themselves. You enable it on a workload domain and vSphere begins exposing a declarative Kubernetes API from inside the hypervisor. Supervisor runs with either one control plane virtual machine for a lab or three for high availability, and those three can spread across up to three vSphere Zones so the loss of one vSphere cluster does not take the control plane down with it. There is no separate BOSH, no Ops Manager, no tile to order. Provisioning logic that TKGI packaged as external brokers now lives as controllers inside the Supervisor. If you want the target platform on its own terms, the VKS Series covers the Supervisor in depth, and this comparison leans on it rather than repeating it.

That difference is philosophical before it is technical. TKGI treats Kubernetes as a product you install onto vSphere. VKS treats Kubernetes as a capability vSphere already has. Read the rest of this comparison through that lens, because most of the component level changes below are downstream of this one decision about where the control plane lives.

Cluster lifecycle, tiles versus Supervisor

Ask what happens when someone requests a new cluster and the two platforms diverge again. On TKGI a request runs a long relay. An operator runs the tkgi command line, which calls the TKGI API, which hands the On Demand Service Broker a plan, which generates a BOSH manifest, which the BOSH Director turns into virtual machines, while a separate NSX Proxy Broker calls NSX Manager to carve out the networking those machines need. Five distinct control components touch the request before vSphere provisions a single node. Each hop is a place the request can stall, and anyone who has watched a cluster sit at a BOSH task for twenty minutes knows the relay is real.

On VKS the same request is a Kubernetes object. You define a cluster in a vSphere Namespace and apply it with kubectl against the Supervisor. The Supervisor API accepts it, the VM Service and the cluster lifecycle controllers reconcile it, and vSphere builds the nodes from a cluster class, a reusable template that fixes the shape of the cluster. Three control components instead of five, all inside the platform, and the request is declarative state rather than an imperative job you sit and watch scroll past.

flowchart TB
  subgraph s1 [TKGI cluster create path]
    T1[tkgi create cluster] --> T2[TKGI API]
    T2 --> T3[On Demand Broker]
    T3 --> T4[BOSH Director]
    T4 --> T5[NSX Proxy Broker]
    T5 --> T6[NSX Manager]
    T6 --> T7[vSphere builds node VMs]
  end
  subgraph s2 [VKS cluster create path]
    V1[kubectl apply cluster] --> V2[Supervisor API]
    V2 --> V3[VM Service]
    V3 --> V4[vSphere builds node VMs]
  end
Same intent, two path lengths. The TKGI relay crosses five control components before vSphere acts; the VKS request crosses three, all inside the platform.

A shorter path is not only faster to reason about, it is easier to debug, because the request is a Kubernetes resource you can describe and inspect rather than a BOSH task buried in a director log. This is also the first place the migration bites. Your TKGI plans, the small, medium and large shapes your teams selected from, do not exist on VKS. Their nearest equivalent is a cluster class, and mapping one onto the other is a design task, not a copy.

Networking, NCP and NSX-T versus Antrea

Networking is where the architecture gap is widest and where the most migration time disappears. TKGI wires pods into NSX-T using NCP, the NSX Container Plugin, which watches the Kubernetes API and programs NSX-T objects to match. Standing that up means Tier 1 routers, IP blocks for node and pod networks, and NAT rules, all provisioned in NSX-T and all assumed by the TKGI install. Load balancing for services comes from the NSX-T load balancer. It works, and once it is built it is stable, but it is a large amount of NSX-T specific plumbing that a team learned to operate by hand.

VKS uses Antrea as its default container network plugin, with Calico as the supported alternative, so there are two CNI choices rather than a single NSX-T coupled path. Antrea runs an overlay inside the cluster and, on VCF 9.1, integrates upward through an Antrea to NSX adapter so NSX can still see and secure pod traffic. The Supervisor itself draws its networking from an NSX VPC rather than from the hand built Tier 1 topology TKGI expected. Service load balancing shifts to either the Foundation Load Balancer or NSX Avi, a choice Part 13 weighs in detail.

Here is where the obvious assumption fails. Because both platforms sit on NSX, teams expect their NSX-T configuration to carry across. It does not. NCP programmed NSX-T objects on TKGI terms, and VKS provisions its own NSX VPC and Antrea overlay on different terms, so the load balancer definitions, the IP block layout and the network profiles do not transfer. You design the target networking fresh. If NSX is where your anxiety sits, the NSX Series owns that ground and is the right place to send a networking colleague while you keep planning.

Storage, registry and identity mapped

Three more components change shape, and each one hides a migration trap under an apparent similarity. Storage looks like a straight swap, because both platforms present persistent volumes through the vSphere Container Storage Interface, the standard plug between Kubernetes and vSphere storage. The trap is that a persistent volume claim carries a StorageClass name, and the StorageClass you defined on TKGI is not the one VKS creates. A volume restored under the wrong class binds to nothing, so storage migration is a remap, not a lift. Part 8 and Part 19 live inside that detail.

Registry moves from Harbor delivered as an Ops Manager tile to Harbor delivered as a Supervisor service, the same project with a different install and a different lifecycle. Identity changes the most. TKGI authenticated through UAA, the User Account and Authentication server, usually chained to LDAP. VKS authenticates through vCenter Single Sign On and Pinniped, the token exchange that lets a Kubernetes client trust vCenter identity. Because the identity provider changes, your role bindings do not copy across untouched, and Part 9 treats that remap as its own assessment.

Gotcha: Three components, storage, registry and identity, look like swaps because the names survive, CSI, Harbor, RBAC. Every one of them changes underneath the name. Plan each as a remap with its own validation, never as a copy, or you find out at restore time when a volume stays Pending and a role binding points at a provider that no longer exists.

Component scorecard, TKGI against VKS

Keep the two tables below. The first is a trade off matrix, how each dimension changes and what the change costs you in migration effort. The second is a facts table, the countable differences you can put in front of a change board. Between them they are the reference artifact of this part, the thing to return to when someone claims the move is a simple like for like swap.

DimensionTKGI approachVKS on VCF 9Migration effort
Control plane locationBOSH managed VMs beside vSphereSupervisor inside vSphereNew platform, stand up fresh
Cluster provisioningTKGI plans via the brokerCluster classes in a NamespaceMap each plan to a class
Pod networkingNSX-T via NCPAntrea over an NSX VPCRedesign, no config transfer
Load balancingNSX-T load balancerFoundation LB or NSX AviChoose and rebuild, see Part 13
IdentityUAA with LDAPvCenter SSO with PinnipedRebind RBAC, see Part 9
Storage class modelTKGI defined StorageClassVKS defined StorageClass on CSIRemap class names on restore
MetricTKGIVKS on VCF 9
Dedicated management VMs before first cluster4 (Ops Manager, BOSH, TKGI API, TKGI DB)0, control rides inside vSphere
Control components in a cluster create path53
Supported CNI options1 (NSX-T via NCP)2 (Antrea default, Calico)
Control plane HA modelBOSH managed, no zone construct3 VMs across up to 3 vSphere Zones
Registry deliveryHarbor as an Ops Manager tileHarbor as a Supervisor service
Identity providerUAA plus LDAPvCenter SSO plus Pinniped

Operational blast radius compared

Numbers make the gap concrete, so here is the one that matters most day to day: how many control components a cluster create request has to pass through before vSphere provisions a node. On TKGI it is five. On VKS it is three. That is not a benchmark, it is a count taken straight off the architecture, and it maps directly to how many places an operation can fail and how many products you keep patched to keep the path healthy.

Control components a cluster create request crossesBefore vSphere provisions the first node03553TKGIVKS on VCF 9Each component is a failure domain and a patch surface on the path
Count taken off the architecture, not a benchmark. Fewer components on the path means fewer independent failure domains between a request and a running node.
War story: On a TKGI estate I once spent close to five hours chasing a cluster create that hung. The TKGI API was healthy, BOSH was healthy, and the stall turned out to be the NSX Proxy Broker waiting on an NSX Manager that had run out of IP block space. Five control components meant five things to rule out before the real one surfaced. On VKS the same failure shows up as a single Kubernetes event on the cluster object, visible in one kubectl describe. Path length is not academic when you are the one holding the pager.

Fewer components in the path is the quiet argument for VKS that rarely makes the slide deck. It is not about raw speed, it is about how many independent failure domains sit between a request and a running node, and how many of them you carry a support contract and a patch schedule for. That is the operational cost the architecture diagram hides, and it is real money and real on call load once the estate is large.

Read the scorecard before you size hardware

My recommendation out of this comparison is narrow and practical: do not size or design the VKS target by analogy to your TKGI layout. Control planes moved into vSphere, networking changed from NCP on NSX-T to Antrea on an NSX VPC, and identity and storage changed providers, so a design that copies TKGI node counts and network profiles will be wrong in ways you only discover mid migration. Take the scorecard above into your architecture review and force a per component decision for each row, because every row is a place the old assumption breaks.

Avoid the trap of treating this as a diff between two similar systems. It is not a diff, it is two systems that happen to share a hypervisor, and that shared hypervisor is exactly what tricks people into underestimating the gap. Read the scorecard as a list of decisions to make, not a list of settings to copy.

Do this on Monday: Take the facts table, walk your own TKGI estate, and fill the TKGI column with your real numbers, your management VM inventory, your CNI, your registry and your identity setup. That filled table is the input for Part 3, which explains why none of this converts in place, and for the estate inventory that starts in Part 5. Verdict: design the VKS target from the scorecard, decision by decision, and avoid the copy the old layout reflex that turns a clean migration into a debugging tour.

Next part makes the full case for why these architecture differences add up to a migration rather than an upgrade, and puts it in language a change board will accept. Bring the filled scorecard with you.

TKGI to VKS Series · Part 2 of 26
« Previous: Part 1  |  Guide  |  Next: Part 3 »

References

About The Author


Discover more from Journal of Intelligent Infrastructure

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure

Subscribe now to keep reading and get access to the full archive.

Continue reading