, ,

vSphere Namespaces and Zones, Configuration and Day Two Management (VCAP-VKS Exam Series, Part 17)

Objective 4.2 asks you to configure and manage vSphere Namespaces and Zones. Here is the runbook I use on a three zone VCF 9.0 Supervisor, the fields that lock at creation, and the CPU limit that quietly wedged an autoscaler for 41 minutes.

VCAP-VKS Exam Series · Part 17 of 34

Three. That is the ceiling on how many vSphere Zones a single vSphere Namespace can draw compute from, and in every VCF 9.0 build I have tested it is a number you commit to at creation and cannot revise afterwards. A team that carves a namespace against two zones because the third cluster was still being patched has quietly decided that no zonal VKS cluster will ever live there, and they will find out weeks later when a cluster create is rejected and nothing in the message says namespace.

Last Part we activated Supervisors across five legal network and load balancer pairings. This Part carves that estate up. A vSphere Namespace is the unit a vSphere administrator hands to a platform team: a slice of CPU, memory and storage taken from one to three zones, with a specific set of storage policies, VM classes and content libraries stapled to it. Everything a VKS cluster can later do is decided here, which is exactly why objective 4.2 carries more weight in troubleshooting items than its position in the blueprint suggests.

Key takeaways: Objective 4.2, published wording Configure and manage vSphere Namespaces and Zones. Zone assignment, namespace name and network binding lock at creation; storage policies, VM classes, content libraries, permissions and limits stay editable for the life of the namespace. A namespace spanning three zones gets one resource pool per mapped vSphere cluster, and a Supervisor refuses to attach a storage policy that is not topology aware to it. Storage policies project into the namespace as StorageClasses, so a missing assignment shows up as storageclass not found on a PVC and never as a namespace error. Do not put CPU and memory limits on a namespace that hosts VKS clusters built from guaranteed VM classes; the reservations count against the limit and scaling stops. Headline check is kubectl get availabilityzones on the Supervisor context.
Who this is for: A vSphere admin or platform engineer with a Supervisor already activated who now has to hand out tenancy and keep it working. Terms defined on first use here: a vSphere Zone is a failure domain object that maps to exactly one vSphere cluster and gives a Supervisor its compute; a vSphere Namespace is the resource and policy boundary you assign to a team; a VM Class is a named CPU and memory shape that VM Service and VKS nodes are sized from; a Content Library holds the images that Kubernetes Releases are pulled from; a Kubernetes Release is the versioned node image and add on bundle a VKS cluster is built against; VKS is vSphere Kubernetes Service, the product previously called TKG Service or TKGS, and the rename is not finished, so you will still see the tanzu string in API groups and the tkr short name throughout this Part.

Zone assignment as a one way door

When you create a namespace against three zones, vCenter does not create one container. It creates a namespace resource pool on each of the three vSphere clusters those zones map to, and every quota you later set is enforced across that set rather than inside a single pool. That detail explains most of the confusing behaviour operators report. A namespace showing plenty of headroom in aggregate can still refuse to place a large guaranteed node, because the reservation has to fit inside one zone, not across three.

Storage inherits the same shape. Broadcom documentation is explicit that a three zone namespace will not let you assign a storage policy that is not topology aware, and the reason is placement: a persistent volume has to be creatable in whichever zone the consuming pod lands in. A policy scoped to one cluster local datastore cannot satisfy that, so the Supervisor removes it from the list rather than letting you discover the problem at PVC time. On a single zone namespace the same policy attaches happily, which is why a manifest that works in a lab namespace fails in the zonal one.

flowchart TD
  Z1[vSphere Zone alpha] --> S[Supervisor]
  Z2[vSphere Zone bravo] --> S
  Z3[vSphere Zone charlie] --> S
  S --> NS[vSphere Namespace ns platform a]
  NS --> RP1[Resource pool on cluster alpha]
  NS --> RP2[Resource pool on cluster bravo]
  NS --> RP3[Resource pool on cluster charlie]
  NS --> SP[Storage policies project as StorageClasses]
  NS --> VC[VM classes project as VirtualMachineClass]
  NS --> CL[Content library supplies Kubernetes Releases]
  SP --> W[VKS cluster and PVC requests succeed]
  VC --> W
  CL --> W
One namespace, three resource pools, and three separate assignment lists that every downstream workload depends on.

Keep the table below. It is the artifact from this Part, and it answers the only question that matters when someone asks you to change a namespace: can I edit this, or am I rebuilding.

Namespace attributeEditable after createSymptom when it is wrong
Zone assignment, one or three zonesNo, rebuildZonal VKS cluster create rejected, no namespace error shown
Namespace nameNo, rebuildEvery kubeconfig context and pipeline reference breaks at once
Network binding, VPC or segmentNo, rebuildIngress and egress plane fixed for life, covered in Part 12
Storage policies assignedYes, add and removePVC Pending with storageclass not found
Storage limit per policyYesPVC create forbidden by an exceeded quota message
VM classes assignedYesCluster or VM Service create rejected naming the class
Content libraries assignedYesNo Kubernetes Release listed, cluster has nothing to pull
Permissions from the identity sourceYesLogin succeeds, namespace absent from kubectl get ns
CPU and memory limitsYesGuaranteed class nodes fail to place, autoscaling stops
Kubernetes object limitsYesObject creation forbidden once the count is reached

Namespace attribute mutability and the first symptom each one produces on a VCF 9.0 Supervisor.

Namespace creation and configuration procedure

Everything below was run against the reference estate: one vCenter, one workload domain, three vSphere Zones mapped to three clusters, VCF Networking with VPC, Avi in the load balancing path, vSAN backing two storage policies. Versions matter for the exam and for reproducing this, so here they are first.

$ kubectl version –short && kubectl vsphere version Client Version: v1.33.4 Server Version: v1.33.4 kubectl-vsphere: version 9.0.0, build 24322831 # Estate under test # VCF 9.0, vCenter 9.0.0.0, Supervisor 9.0, VKS 3.3.1 # VCF CLI 9.0.0, govc 0.49.0

Step 1, preflight the zones before you touch a namespace. A zone that is not healthy will still appear in the create wizard, so prove them from the Supervisor context first. Log in with the vSphere plugin rather than a raw kubeconfig, because the plugin is what mints the token your identity source issued, and it is the only login path the exam treats as correct for a Supervisor.

$ kubectl vsphere login –server=$SUPERVISOR_VIP –vsphere-username $VSPHERE_USER –insecure-skip-tls-verify Password: (read from the VSPHERE_PASSWORD environment variable, never inline) $ kubectl config use-context $SUPERVISOR_VIP $ kubectl get availabilityzones NAME AGE zone-alpha 61d zone-bravo 61d zone-charlie 61d $ kubectl get availabilityzones -o custom-columns=NAME:.metadata.name,CLUSTERS:.spec.clusterComputeResourceMoIDs NAME CLUSTERS zone-alpha [domain-c1011] zone-bravo [domain-c1042] zone-charlie [domain-c1073]

Note the API group behind that resource. It is topology.tanzu.vmware.com, not anything with vks in the name. Broadcom renamed the product and left the API surface alone, so any objective that touches zones, releases or cluster objects will show you tanzu strings, and a candidate who has only read the marketing pages hesitates on exactly those screens. Three zones, three cluster managed object ids, one per zone. If a zone maps to more than one cluster or a cluster appears twice, stop and fix the topology before carving tenancy on top of it.

Step 2, create the namespace against the zone set you actually want. In the vSphere Client this is Supervisor Management, then Namespaces, then Create Namespace. You pick the Supervisor, the name, the network, and the zones in one screen. For a namespace that will host zonal VKS clusters, select all three zones. There is no undo on that screen worth the name; deleting and recreating is the supported correction and it took six minutes on my estate including the resource pool teardown.

Step 3, assign storage policies and set a per policy limit. Configure, then Storage. Add the topology aware vSAN policy as the default and, if the team needs it, the high performance policy as a second class. Set a limit on each rather than leaving them unbounded, because an unbounded storage class on a shared vSAN cluster is how one team fills a datastore for everyone. Each assigned policy appears inside the namespace as a StorageClass with the policy name normalised to lower case and hyphens.

Step 4, assign VM classes and the content library. Configure, then VM Service. Add the classes the team is allowed to size nodes from, and add the subscribed content library that carries Kubernetes Releases. Skipping the library is the single most common cause of a cluster create that sits doing nothing, because the cluster controller has no image to resolve and reports the shortage as a class or release mismatch rather than a library problem.

Step 5, grant permissions from the identity source. Permissions, then Add. Bind the LDAP group to the edit role for a team that provisions its own clusters, and view for auditors. A user with a valid token and no binding logs in without error and then sees an empty namespace list, which every service desk on earth misreads as a login failure.

Step 6, cap object counts instead of reaching for CPU limits. Configure, then Object Limits. Capping the number of pods, services, PVCs and config maps constrains a runaway pipeline without touching the reservation arithmetic that VKS nodes depend on. This is the part of the procedure most guides skip, and it is the part that keeps a shared Supervisor stable.

Verification, limits and failure signatures

Three commands prove a namespace is actually usable, and they prove it faster than reading the Configure tabs back. Run them in the namespace context, not the Supervisor root.

$ kubectl config use-context ns-platform-a $ kubectl get storageclass NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE AGE vsan-default-storage-policy csi.vsphere.vmware.com Delete WaitForFirstConsumer 9m vsan-high-perf csi.vsphere.vmware.com Delete WaitForFirstConsumer 9m $ kubectl get virtualmachineclass NAME CPU MEMORY AGE best-effort-large 4 16Gi 9m guaranteed-large 4 16Gi 9m guaranteed-2xlarge 8 32Gi 9m $ kubectl get tkr NAME VERSION READY v1.33.4—vmi.v3.3.1 v1.33.4+vmware.1-vks.1 True

That is what green looks like: at least one StorageClass, at least one VirtualMachineClass, and at least one Kubernetes Release reporting Ready True. Note the short name tkr and the release string carrying vmi and vks together. Broadcom kept the tanzukubernetesrelease resource and grafted the new product version onto the label, which is a small thing until you are reading a hot area screen under time pressure and looking for a resource called something sensible.

Now the failure. My PostgreSQL StatefulSet went to Pending against a namespace where the high performance policy had been removed during a cleanup and nobody had noticed, because a StatefulSet with WaitForFirstConsumer does not complain until a pod is scheduled.

$ kubectl describe pvc data-pg-0 Name: data-pg-0 Status: Pending StorageClass: vsan-high-perf Events: Type Reason Age From Message Warning ProvisioningFailed 14s persistentvolume-controller storageclass.storage.k8s.io "vsan-high-perf" not found # and later, once the policy was back but the limit was not raised $ kubectl apply -f pg-statefulset.yaml Error from server (Forbidden): error when creating "pg-statefulset.yaml": persistentvolumeclaims "data-pg-2" is forbidden: exceeded quota: ns-platform-a-storagequota, requested: requests.storage=20Gi, used: requests.storage=95Gi, limited: requests.storage=100Gi

Neither line mentions a vSphere Namespace. Both are pure Kubernetes messages produced by a vSphere side configuration change, and learning to read backwards from them is most of what objective 5.1 will later ask of you. Storage policy assignment produces the first message. Storage limit per policy produces the second.

Namespace create to first PVC Bound Median of five runs, three zone VCF 9.0 estate, vSAN backed policies Policy and class at create 38 s Class added later 44 s Library added later 52 s Policy added after the claim 191 s Policy not topology aware never bound, assignment refused on three zones Assigning storage before the first claim is worth roughly two and a half minutes per namespace.
Front loading the assignment lists costs nothing at create time and saves minutes per namespace afterwards.
Where the popular advice is wrong: Nearly every namespace walkthrough tells you to set CPU and memory limits as basic hygiene. On a namespace that hosts VKS clusters built from guaranteed VM classes, that advice is actively harmful. A guaranteed class creates a full CPU and memory reservation on the node virtual machine, and those reservations are charged against the namespace limit whether the workload inside is busy or idle. Cap storage per policy and cap object counts instead. Reach for a CPU limit only on namespaces running Supervisor Pods and VM Service machines from best effort classes, and size it above the sum of every reservation you expect.

Rollback. Every assignment in steps 3 to 6 is reversible in place: remove a storage policy, remove a VM class, unbind a permission, clear a limit. Removing a storage policy that is currently backing bound volumes does not delete the volumes, but it does stop new claims immediately, so treat removal as a change with a blast radius rather than a tidy up. The only true rollback for a wrong zone set or a wrong name is deletion and recreation, and deleting a namespace deletes everything in it including VKS clusters, so drain and back up first. Part 28 covers the Velero side of that.

Error or symptomSupervisor object that caused itRemediation
storageclass.storage.k8s.io not found on a PVCStorage policy not assigned to the namespaceConfigure, Storage, add the policy, then delete and recreate the claim
exceeded quota, requests.storage limitedPer policy storage limit reachedRaise the limit on that policy or reclaim released volumes
Cluster create rejected naming a vmClassVM class not assigned to the namespaceConfigure, VM Service, add the class, reapply the cluster manifest
kubectl get tkr returns nothingContent library unassigned or not syncedAssign the library, force a sync, recheck after the sync completes
Login succeeds, kubectl get ns is emptyNo role binding for that identity on the namespacePermissions, Add, bind the group to edit or view
Node virtual machine will not power on after a limit changeGuaranteed class reservation exceeds the namespace CPU or memory limitRaise or clear the limit, then let the machine controller retry
Topology aware policy missing from the assign listNamespace spans three zones, policy is cluster localExtend the policy rule set to all three clusters or use a single zone namespace

Failure to cause lookup for vSphere Namespace and zone misconfiguration on VCF 9.0.

Exam focus for objective 4.2

Objective 4.2, Configure and manage vSphere Namespaces and Zones: What this objective expects you to be able to do is create a namespace against the correct zone set, attach the four assignment lists, set limits that do not break workloads, and read a Kubernetes error back to the vSphere object that produced it. Expect it in point and click and hot area items on the Supervisor Management screens, where you have to know which tab a setting lives under rather than describe it; in matching items pairing a namespace setting to a symptom; and in multiple selection items about what is fixed at creation. Trap that catches experienced admins: treating a vSphere Namespace as a Kubernetes namespace with a resource pool attached, and answering quota questions with Kubernetes ResourceQuota reasoning. Zone to resource pool mapping, storage policy projection into StorageClasses, VM class assignment and content library binding have no Kubernetes equivalent, and those four are where the distractors live.

One more habit worth building before the exam. Whenever a VKS cluster misbehaves, name the Supervisor object that could have caused it before you touch the cluster. Nine times in ten for this objective it is one of five things: the zone set, a storage policy, a VM class, a content library, or a permission. Working that list takes about ninety seconds and it is faster than any log you could open.

Objective checkpoint

1. A vSphere Namespace spans three vSphere Zones. Which category of storage policy will the Supervisor refuse to let you assign to it, and why?
Answer: Any policy that is not topology aware. Reasoning: a persistent volume must be creatable in whichever zone the consuming pod is scheduled into, so a policy scoped to datastores on a single cluster cannot satisfy the placement contract and is filtered out of the assignment list rather than failing later.
2. A platform team asks you to extend an existing single zone namespace so it can host a zonal VKS cluster. What do you tell them?
Answer: Zone assignment is fixed at namespace creation, so the namespace has to be recreated against all three zones and the workloads migrated. Reasoning: a namespace resource pool is created on each mapped vSphere cluster at creation time, and there is no supported in place operation that adds a pool afterwards.
3. After a CPU limit is applied to a namespace, an existing VKS cluster stops adding nodes although the namespace shows spare capacity. What is the most likely cause?
Answer: Nodes are sized from a guaranteed VM class, so each one carries a full CPU reservation that is charged against the namespace limit regardless of actual usage. Reasoning: guaranteed classes reserve, best effort classes do not, and the limit is enforced against reservations rather than observed consumption.

Namespace and zone call for this estate

Here is the war story, because it is the reason I changed how I hand out namespaces. On a Tuesday afternoon I put a 96 GHz CPU limit on ns-platform-a to stop a noisy build farm from starving a database team on the same vSAN cluster. Nothing broke that day. On Friday the cluster autoscaler on a VKS cluster in that namespace stopped adding workers under load, and the cluster reported nothing more useful than a machine stuck in Provisioning. Nodes were sized from guaranteed-2xlarge, 8 vCPU and 32 GiB each, fully reserved. Five existing workers had already consumed most of the ceiling, and the sixth could not reserve. Forty one minutes went into reading machine controller logs before I looked at the namespace tab I had edited myself three days earlier. Clearing the CPU limit fixed it in under a minute.

My recommendation for this estate, and the one I give candidates: create every namespace that will host VKS clusters against all three zones even if only one cluster is needed today, assign both storage policies and every VM class the team could plausibly need at creation time, attach the content library in the same sitting, and govern with storage limits and object limits only. Leave CPU and memory limits alone unless the namespace is running Supervisor Pods from best effort classes. Three zones cost you nothing if unused; two zones cost you a rebuild.

What a clean result looks like: kubectl get availabilityzones lists three zones, each mapped to exactly one cluster. Inside the namespace context, kubectl get storageclass returns at least one topology aware class, kubectl get virtualmachineclass returns the classes the team is entitled to, and kubectl get tkr shows a release with Ready True. Configure, Storage shows a finite limit against each policy. Configure, Object Limits shows caps on pods, services and claims. Permissions lists a group bound to edit. No CPU or memory limit is set on any namespace hosting VKS clusters.

Tonight, in your own lab, log into the Supervisor and run those four verification commands against your busiest namespace. If any one of them comes back empty, you have found a cluster failure that has not happened yet. For the underlying product mechanics this Part deliberately does not re teach, the VKS Series is the walkthrough that sits beneath this exam path, and the NSX Series covers the segment and VPC objects a namespace binds to. Next Part moves from carving tenancy to filling it, with Supervisor Pods and VM Service workloads under objective 4.3.

VCAP-VKS Exam Series · Part 17 of 34
« Previous: Part 16  |  Guide  |  Next: Part 18 »

References

About The Author


Discover more from Journal of Intelligent Infrastructure

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure

Subscribe now to keep reading and get access to the full archive.

Continue reading