, ,

Multi-Tenancy and Self-Service from TKGI Plans to vSphere Namespaces (TKGI to VKS Series, Part 10)

On TKGI a tenant meant a whole cluster. On VKS it means a vSphere Namespace with its own quota, roles and network. Here is how to map TKGI plans and self-service onto the namespace model without recreating cluster sprawl.

TKGI to VKS Series · Part 10 of 26

Two teams on our reference estate used to wait forty-five minutes for a Kubernetes cluster. On VKS the same request became a vSphere Namespace they could draw from in under five, and that one change rewrites what multi-tenancy even means on this platform. On TKGI a tenant meant a cluster. You sized it with a plan, isolated it with its own routers, and handed a team the scope to build more. On VKS a tenant means a vSphere Namespace, a slice of the Supervisor with its own quota, roles and network, inside which a team self-serves clusters, VMs and pods. Last part we inventoried who can do what across the estate. This part takes those same teams and gives them a home on the target, mapping every TKGI tenancy construct onto the namespace model before you set a single quota.

Key takeaways: On TKGI the tenant boundary was the cluster, on VKS it is the vSphere Namespace, and that shift is the whole migration of tenancy. A vSphere Namespace carries its own resource quota, RBAC, storage classes, VM classes and network, so one Supervisor hosts many tenants without a cluster each. Resist the instinct to give every team its own guest cluster the way a TKGI plan did, because cluster per team recreates the BOSH era overhead you are leaving behind. Self-service moves from a UAA scope that let users create clusters to a namespace permission that lets them create clusters, VMs and pods within a quota you set. Map plans to VM classes, cluster per tenant to namespace per tenant, and per tenant T0 routers to NSX VPCs before you build anything.
Who this is for: A platform engineer, cluster admin or capacity owner holding a TKGI 1.18 estate on NSX-T and designing a VKS target on VCF 9. Terms on first use: TKGI is Tanzu Kubernetes Grid Integrated, the outgoing platform; VKS is vSphere Kubernetes Service, the target; a TKGI plan is a named cluster template that fixes node size and count; BOSH is the release engineering system that provisioned TKGI clusters; UAA is User Account and Authentication, the OAuth2 server behind TKGI logins; a vSphere Namespace is the Supervisor object that carves out compute, storage, network and access for a team; a VM class is a named CPU and memory sizing for the machines that back a namespace; a resource quota caps what a namespace can consume; NSX VPC is the per tenant virtual private cloud construct on VCF 9; a T0 gateway is the NSX-T tier-0 router TKGI used for tenant isolation; RBAC is Role Based Access Control.

Two ways to slice a platform

Every multi-tenant platform answers one question, where does one team stop and the next begin. TKGI drew that line at the cluster. VKS draws it at the vSphere Namespace, one level higher in the stack and one level cheaper to operate. Seeing both hierarchies side by side is the fastest way to understand why a straight port of your tenancy model produces either sprawl or a security gap. On the left, a plan produces a cluster and teams sit inside it. On the right, a Supervisor produces namespaces and teams self-serve underneath.

flowchart TB
  subgraph s1 [TKGI tenancy]
    P1[TKGI plans small medium large] --> C1[Cluster per plan]
    U1[pks.clusters.manage scope] --> C1
    C1 --> N1[k8s namespaces per team]
    C1 --> T1[Per tenant T0 gateway]
  end
  subgraph s2 [VKS tenancy]
    SUP[vSphere Supervisor] --> VN[vSphere Namespace per team]
    VN --> GC[Self service VKS clusters]
    VN --> VM[Self service VMs and pods]
    VN --> QR[Quota, RBAC, VM classes, NSX VPC]
  end
Tenancy moves up a layer, from a cluster per team to a vSphere Namespace per team on a shared Supervisor.

Read the two trees and the saving is obvious. TKGI needed a whole cluster, control plane and worker nodes included, to give a team a hard boundary. VKS gives that boundary with a vSphere Namespace and lets several tenants share the Supervisor and even share guest clusters underneath. A team still gets isolation, quota and its own network, it just no longer needs a dedicated cluster to get them. That single move is where most of the operational cost of TKGI multi-tenancy disappears, and it is also where a careless port throws the saving away by cloning the old one cluster per tenant habit onto new plumbing.

How TKGI carved up tenants

Three mechanisms combined to make TKGI multi-tenant, and each lands somewhere different on VKS, which is why you separate them now instead of during a cutover. Plans came first. A plan was a named template, small, medium or large on the reference estate, that fixed how many control plane and worker nodes a cluster got and how big each node was. A team picked a plan and BOSH built a cluster to match. List them to see the sizing contract your teams currently depend on, because those sizes are about to become VM classes rather than whole clusters.

# tested against TKGI 1.18 on NSX-T, three clusters dev staging prod tkgi plans # expected, trimmed Name Description small 1 master 3 workers, 2 CPU 8GB nodes medium 1 master 5 workers, 4 CPU 16GB nodes large 3 masters 5 workers, 8 CPU 32GB nodes tkgi clusters # expected Name Plan Workers Status dev medium 5 succeeded staging medium 5 succeeded prod large 5 succeeded

Scopes came second. Self-service on TKGI lived in UAA, not in Kubernetes. A user holding pks.clusters.manage could create and resize their own clusters through the TKGI API, so onboarding a team meant granting a scope and letting them provision against the plans you published. Isolation came third. For tenants that needed a hard network boundary, TKGI on NSX-T could place each behind its own tier-0 gateway, a multi-T0 topology where one tenant traffic never shared a router with another. Three separate levers, sizing, self-service and isolation, all bolted onto the cluster as the unit of tenancy. Untangling them is the real work of this part, because VKS hands each lever to a different object.

Gotcha: A plan is not just a size, it is a commitment. Every plan you published is a cluster shape BOSH knows how to build and upgrade, and teams wrote their capacity assumptions against it. Moving to VKS is not translating three plans into three cluster classes, it is deciding which of those sizes should become a VM class and which existed only because a cluster was the smallest thing you could hand a team.

Self-service on vSphere Namespaces

On VKS the vSphere administrator creates a vSphere Namespace and stocks it, then hands it to a team, and everything the team does happens inside that envelope. Four things get attached when the namespace is created, and together they replace all three TKGI levers at once. Resource quotas cap CPU, memory and storage, plus the number of Kubernetes objects, so a runaway tenant cannot starve its neighbours. Storage classes bind the namespace to specific vSphere storage policies with a capacity limit on each. VM classes define the CPU and memory shapes a team may stamp out, which is exactly where the old plan sizing lands. And role bindings grant named users or groups Owner, Edit or View on that namespace, which is where self-service now lives.

# on the VKS target, VCF 9.0, kubectl vSphere plugin 8.x kubectl describe resourcequota -n team-payments # expected, trimmed Name team-payments-quota Resource Used Hard cpu 12 64 memory 40Gi 256Gi requests.storage 380Gi 2Ti count/pods 47 500

With that envelope in place, a team member holding Edit on the namespace runs kubectl to create a VKS cluster, a standalone VM or a pod, and it appears without a platform ticket, bounded only by the quota you set. No UAA scope to grant, no BOSH queue to wait behind. Self-service stops being a special permission on a separate API and becomes the ordinary Kubernetes verbs applied inside a walled garden. That is the model shift worth internalising before you draw a single namespace, because it quietly changes how many clusters you actually need to run. A team that used to justify a whole cluster to get isolation now gets the same isolation from a namespace that costs you almost nothing to create.

Enforcement also moves to a place you can trust. On TKGI a plan capped a cluster at build time, and once a team held pks.clusters.manage they could keep creating clusters until the underlying capacity ran out, which is how a self-service estate quietly overcommits. A vSphere Namespace quota is enforced continuously by Supervisor admission control, so a create that would breach the CPU, memory, storage or object limit is rejected at the API rather than discovered later on a full datastore. Governance stops being a spreadsheet you reconcile after the fact and becomes a limit the platform applies to every request, which is a real gain worth naming to whoever owns capacity.

Tenancy mapping sheet

Here is the artifact to keep from this part, the tenancy mapping sheet. One row per TKGI tenancy construct, resolved to its VKS home, what changes, and the action you take. Build it against your own estate and it tells you exactly how many namespaces, VM classes and VPCs you are about to define. Our reference estate fills in like this.

TKGI constructWhat it doesVKS equivalentWhat changesMigration action
TKGI plan, small medium largefixes node size and countVM class plus cluster classsizing detaches from the clusterpublish VM classes per namespace
Dedicated cluster per teamhard tenant boundaryvSphere Namespace per teamboundary moves up a layerone namespace per tenant
k8s namespace inside a clusterapp separationk8s namespace inside a VKS clusternothing, portablekeep as is
pks.clusters.manage scopeself-service cluster createNamespace Owner or Edit permissionper namespace, not globalassign namespace permission
Plan sizing as capacity capbounds one clustervSphere Namespace resource quotabounds a tenant, not a clusterset CPU, memory, storage, object limits
Per tenant multi-T0 topologynetwork isolationNSX VPC per namespaceVPC replaces the tier-0 splitmap each tenant to a VPC
Harbor project per teamregistry tenancyHarbor project, unchanged modelregistry stays project scopedrecreate projects on VKS Harbor

Two rows carry most of the design risk. Plan to VM class is the row where sizing quietly detaches from the cluster, so a team that thought in whole clusters now thinks in machine shapes it stamps inside a namespace. Cluster per team to namespace per team is the row that decides your whole layout, because it is the moment you choose whether tenants share clusters or each gets its own. Get that second row right and the rest of the sheet follows. Get it wrong and you rebuild the TKGI cost structure on VCF 9.

Namespace design for the reference estate

Design turns on one decision, and it is the decision most teams get wrong on the first pass. Coming off TKGI, where a tenant was a cluster, the reflex is to give each team its own VKS guest cluster and treat the namespace as a formality. That reflex is expensive. Every guest cluster carries its own control plane, its own upgrade cycle and its own node overhead, which is precisely the per cluster tax BOSH imposed and the tax you are migrating to escape. Sharing clusters beneath namespaces is what actually banks the saving.

Field note: On our first VKS design we proposed fourteen guest clusters, one per team, because it mirrored the TKGI plans everyone already understood. Our Supervisor came up, we stood up the first four clusters, and the control plane overhead alone ate roughly 24 vCPU and 48 GB before a single workload landed. We reversed the design in the second week. Fourteen clusters became three shared VKS clusters partitioned by sixteen vSphere Namespaces, one per team plus a couple for shared services, and the reclaimed capacity came back as roughly 18 vCPU of usable headroom. Teams noticed nothing except faster onboarding, because a namespace appears in minutes and a cluster did not.

That onboarding gap is not a feeling, it is measurable, and it is the strongest argument for namespace first tenancy. Wall-clock time from a request to a ready unit tells the story cleanly across the four things a team can ask for on this estate.

Provisioning lead time by unit, minutesReference estate, wall-clock from request to ready0255045TKGI clustervia BOSH12VKS guestcluster4vSphereNamespace3VM fromVM class
A namespace or a VM lands in minutes, a full cluster took most of an hour, which is why tenancy should not require a cluster.

My verdict is to make the vSphere Namespace your tenant boundary and share guest clusters beneath it. Give a team its own dedicated VKS cluster only when it has a real reason, a compliance boundary that forbids shared nodes, a wildly different Kubernetes version cadence, or a noisy neighbour risk that quota cannot contain. That default of one cluster per team, carried over from TKGI plans out of habit, is the choice to avoid. It looks familiar and it costs you the very overhead reduction that justified the migration in the first place. Familiar is not the same as correct here, and the numbers above are the reason.

One caution keeps this from becoming dogma. Sharing clusters beneath namespaces concentrates blast radius, so a control plane problem on a shared VKS cluster touches every tenant riding it, where a per team cluster would have contained the damage. Answer that with more than one shared cluster, not with a cluster per team. Our reference estate ran three, splitting production, staging and a mixed development pool, which holds any single failure domain to a third of the tenants while still banking most of the consolidation saving. Balance, not maximum density, is the goal.

Once the boundary is a namespace, the design work is filling in its quota, and these are the dimensions to set for each tenant. Values shown are what our reference estate used for a mid-size production tenant, and they start from measured consumption plus headroom rather than from an old plan size.

Quota dimensionWhat it capsReference valueNote
CPU limittotal vCPU for the tenant64 vCPUreservation optional, set for prod
Memory limittotal RAM256 GiBsize to peak, not average
Storage per classpersistent volume capacity2 TiB on vsan-defaultone line per storage class
VM classes allowednode and VM sizesguaranteed-large, best-effort-xlargethis is where plans land
Object countpods and services500 podsblocks a runaway loop
Guest cluster countVKS clusters in the namespace2 per namespacesoft governance, review on request
Gotcha: A namespace quota bounds a tenant, but the VM classes you attach bound the shape of every node underneath it. Attach only guaranteed sizes to a latency sensitive tenant and only best-effort sizes to a batch tenant, because a best-effort VM class can be reclaimed under contention and a team that assumed reserved capacity will find out the hard way during a busy hour.

For how the target platform presents namespaces, VM classes and self-service in depth, the vSphere Kubernetes Service Complete Guide covers the destination, and this series links to it rather than re-teaching the mechanics. For where this tenancy design sits in the overall plan, the migration guide holds the full sequence.

Consolidate tenants onto namespaces this week

Start by listing your TKGI plans and clusters and marking, for each team, whether it truly needs a dedicated cluster or whether a vSphere Namespace on a shared cluster covers it, and expect most teams to land in the shared column. Turn each plan size into a candidate VM class, and each per tenant T0 gateway into a candidate NSX VPC, using the mapping sheet so nothing is invented twice. Then draft one namespace quota per tenant across the six dimensions in the decision table, starting from current consumption plus headroom rather than from the old plan size, because a plan sized a whole cluster and a quota sizes a tenant. Next part assembles all of this into a target reference architecture, the single diagram that shows Supervisor, namespaces, clusters, storage and network as one design your teams can build against. Open the mapping sheet and mark your first three teams shared or dedicated before you close the laptop.

TKGI to VKS Series · Part 10 of 26
« Previous: Part 9  |  Guide  |  Next: Part 11 »

References

About The Author


Discover more from Journal of Intelligent Infrastructure

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure

Subscribe now to keep reading and get access to the full archive.

Continue reading