, ,

VKS Load Balancer Sizing and Its Effect on Supervisor Functionality (VCAP-VKS Exam Series, Part 11)

Objective 3.1 is not about throughput. Load balancer sizing on VCF 9.0 decides whether a Supervisor enables at all, and this Part maps every shortfall to the exact symptom it produces.

VCAP-VKS Exam Series · Part 11 of 34

Eight vCPUs and 32 GB of RAM. That is the smallest NSX Edge node Broadcom permits underneath a Supervisor, and getting it wrong does not give you a slow application, it gives you a Supervisor that never finishes enabling.

Key takeaways: Objective 3.1, published wording Evaluate the implication for Load Balancer size on Supervisor functionality. Sizing here means object count, not throughput; a Supervisor carrying 40 namespaces at almost zero requests per second will exhaust a small load balancer long before it saturates a link. Broadcom design requirement VCF-SUP-NLB-REQD-CFG-003 puts a hard floor under the NSX model: Edge nodes must be deployed at large form factor or bigger. Avi Controllers have no CPU or memory hot add, so controller size is a redeploy decision, not a Day 2 slider. Headline check before you touch anything: GET /policy/api/v1/infra/lb-node-usage?node_path=<node-path>.
Who this is for: You settled the container network interface and the trusted CA bundle in Part 10, so the lab now builds clusters that pull images and trust the registry. You have sized an NSX Edge cluster before, probably by copying whatever the last deployment used. Objective 3.1 sits in Section 3, Plan and Design, and it is one of the few objectives where a wrong answer at build time cannot be corrected with a kubectl command.

Three sizing decisions hiding inside one objective

Broadcom writes this objective as a single phrase about load balancer size, and candidates read it as a single question. It is three. First, which load balancer model your workload network permits, which Part 6 settled and this Part will not restate. Second, how large the thing running that load balancer is, meaning an NSX Edge form factor, an Avi Controller size and Service Engine group, or the count of Foundation Load Balancer virtual machines. Third, how many virtual IP addresses the pool you handed it can actually issue.

Only the second one is what most people mean by sizing, and it is the one Broadcom is asking about. Miss the third and you get a stranded Supervisor with plenty of spare capacity and no addresses to hand out, which is a different failure with an almost identical symptom.

Every part of this decision is a Supervisor decision, not a cluster decision. You pick the load balancer model and its size before Supervisor activation, and after that every vSphere Namespace, every VKS cluster control plane endpoint and every Service of type LoadBalancer inside those clusters draws from the same pool. A VKS cluster that will not finish provisioning is almost never a cluster problem. It is a Supervisor that cannot get a virtual server allocated on your behalf. One naming note while we are in this territory: VKS is the product formerly called TKG Service or TKGS, and when you go looking at the load balancer objects backing a workload cluster you will still see tkg in the generated names. Broadcom renamed the product, not the object naming.

Versions this Part was checked against vCenter and vSphere Supervisor 9.0 vSphere Kubernetes Service 3.3.1 NSX 4.2.x Avi Load Balancer 22.1.4 Enterprise Avi Controller sizing reference 30.1.1 allocations kubectl client v1.32.4 kubectl-vsphere plugin 9.0.0 VCF CLI 1.5.x

Sizing knobs per load balancer model

Each of the three models exposes a completely different knob, and that asymmetry is what scenario items are built on. Print this one. It is the first of two reference artifacts in this Part and it answers the only question that matters when a scenario tells you the estate is short on load balancer capacity: what, specifically, do you enlarge.

ModelWhat you actually sizeBroadcom floorShared with non Supervisor workloadsResize in place
Foundation Load BalancerAppliance count and arm topology: one arm with one NIC, one arm with two NICs, or two arm with three NICsTwo or three routable distributed port groups, prepared manually before activationNoAvailability only, one VM to two VMs active and passive
NSX Load BalancerEdge node form factor, plus load balancer service size small, medium, large or extra largeEdge nodes at minimum large form factor, 8 vCPU and 32 GBNoScale Edges up or out, which means redeploying nodes
Avi Load BalancerController size, controller disk, and Service Engine group scale out per virtual serviceThree node controller cluster recommended; single node only at small size and only via a KB procedureYes, Avi also fronts traditional VM workloadsService Engines yes, Controller no, there is no CPU or memory hot add

Two rows on that table deserve a second look. Foundation Load Balancer and NSX Load Balancer both refuse to serve anything other than Supervisor and the services Supervisor manages, so capacity you buy for them is capacity you cannot amortise across the rest of the estate. Avi is the only model where an oversized deployment earns its keep elsewhere, and that changes the cost conversation more than any feature comparison does.

Avi Controller sizing is where hard numbers exist, so learn these rather than a vague sense that bigger is better. Allocations below come from the 30.1.1 reference and are the ones a design question will lean on.

Controller sizeCPU and memoryBase processesLog analyticsService Engine scale
Essentials4 CPU, 24 GB15 GB9 GB0 to 10, capped at 100 virtual services
Small6 CPU, 32 GB19 GB13 GB0 to 100
Medium10 CPU, 32 GB19 GB13 GB100 to 200
Large16 CPU, 48 GB24 GB24 GB200 to 400
Service Engine ceiling by Avi Controller sizeUpper bound of the documented SE scale range, Avi Load Balancer 30.1.1 allocations400200010Essentials4 CPU / 24 GB100Small6 CPU / 32 GB200Medium10 CPU / 32 GB400Large16 CPU / 48 GB
Controller size sets a hard ceiling on Service Engine count, and no amount of Service Engine group tuning moves it.

Notice that Small and Medium carry identical memory and identical analytics allocations. Four extra cores are the entire difference, and they double the Service Engine ceiling from 100 to 200. That is unusually good value in a sizing table, and it is the answer to the most common real world question about Avi in a VKS estate: if you are anywhere near 100 Service Engines, go to Medium and stop thinking about it.

Disk is a separate axis and it is sized by virtual service count rather than by controller size. Broadcom’s guidance is 80 GB total for 100 virtual services without full logs, 216 GB for 1,000, and 736 GB for 5,000, with metrics alone consuming roughly 32 MB per virtual service. Default controller OVA disk must be raised to 128 GB before deployment. I have watched more Avi Controllers fall over from a full metrics database than from CPU pressure, and the symptom is not obvious: analytics stop updating, then the user interface starts timing out, and the load balancing data plane keeps working perfectly the whole time, which is exactly why nobody suspects the controller.

Supervisor functions that break when the load balancer runs short

Here is the artifact worth returning to, and the reason this Part exists. Objective 3.1 asks about implication, which means Broadcom wants you to reason from a shortfall to a symptom. Every row below is a Supervisor function that consumes load balancer capacity, and every symptom is one I have watched a platform team misdiagnose at least once.

Supervisor functionWhat it consumesSymptom when capacity runs outWhere to look first
Supervisor activationOne virtual server for the control plane endpointEnablement stalls at Configuring and never reaches RunningvCenter Workload Management status, then Edge node usage
vSphere Namespace creationNetwork objects and, on segment networking, a tier 1 with load balancer attachmentNamespace shows created in vCenter but never becomes usableNSX Manager alarms on the Edge cluster
VKS cluster provisioningOne virtual server per cluster control plane endpointCluster sits in Provisioning with control plane nodes up and no reachable API serverkubectl get cluster and the virtual server list
Service of type LoadBalancerOne virtual server and one virtual IP eachEXTERNAL-IP stays pending and a SyncLoadBalancerFailed event repeatskubectl describe svc on the affected Service
Supervisor add on servicesAt least one virtual server each, Harbor includedService installs but its endpoint never publishes, so image pulls fail estate wideSupervisor Services pane, then the service namespace
Cluster and Supervisor upgradesTransient extra capacity while old and new endpoints coexistRolling update wedges partway with no obvious node level errorRemaining credit on the Edge node during the window
Sizing rule I use, and why it is deliberately generous: budget one virtual server for the Supervisor, one per VKS cluster control plane, one per LoadBalancer Service you expect, then add 25 percent headroom purely for upgrades. Last row of that table is the reason. During a rolling upgrade the old and new endpoints briefly coexist, so an estate sized to exactly 100 percent of steady state will pass every acceptance test and then jam on its first upgrade window, at 2am, with a change ticket expiring. Headroom is not waste here, it is the only thing standing between you and a rollback.

Usage discovery and a sizing walkthrough for the reference estate

Reference estate this series has carried since Part 3 is one vCenter, one workload domain, three vSphere Zones across three clusters, NSX with VPC networking, Avi in the load balancing path and vSAN behind two storage policies. Last Part we pinned the container network interface and the trusted CA bundle. Now we count what the load balancer is being asked to carry, because a sizing answer without a current usage figure is a guess wearing a suit.

Start on the Kubernetes side, because that is where the shortfall shows up first and where you have credentials already. Read the password from an environment variable rather than typing it, on the exam and in production both.

export KUBECTL_VSPHERE_PASSWORD="$VSPHERE_ADMIN_PW" kubectl vsphere login –server=$SUPERVISOR_VIP –vsphere-username $VSPHERE_ADMIN_USER –insecure-skip-tls-verify # every Service currently holding or waiting on a virtual IP kubectl get svc –all-namespaces –field-selector spec.type=LoadBalancer NAMESPACE NAME TYPE EXTERNAL-IP PORT(S) AGE ns-web web-lb LoadBalancer 10.50.12.41 80:31220/TCP 21d ns-db postgres-ro LoadBalancer 10.50.12.42 5432:30119/TCP 19d harbor harbor-ingress LoadBalancer 10.50.12.43 443:31981/TCP 26d ns-web web-canary LoadBalancer <pending> 80:31544/TCP 14m

Fourth row is the failure, and it is the one worth practising because it looks like an application problem and is not.

kubectl describe svc web-canary –namespace ns-web Events: Type Reason Age From Message —- —— —- —- ——- Normal EnsuringLoadBalancer 14m service-controller Ensuring load balancer Warning SyncLoadBalancerFailed 2m (x9 over 14m) service-controller Error syncing load balancer: virtual server allocation failed

Nothing in that output names a size. You have to go and ask the Edge node how full it is, and this next call is the single most useful command in the whole objective. Field names in the response move between NSX releases, so read the shape rather than memorising keys.

export NSX_PW="$NSX_ADMIN_PW" export NODE=/infra/sites/default/enforcement-points/default/edge-clusters/$EDGE_CLUSTER/edge-nodes/0 curl -sk -u admin:$NSX_PW "https://$NSX_MANAGER/policy/api/v1/infra/lb-node-usage?node_path=$NODE" { "node_path": "/infra/sites/default/…/edge-nodes/0", "current_virtual_server_count": 38, "current_pool_count": 38, "current_pool_member_count": 214, "usage_percentage": 79.1 }

Seventy nine percent on a large Edge node, with three zones and a documented plan to add four more VKS clusters this quarter. Each of those clusters costs one virtual server for its control plane before a single application is deployed, and the applications the platform team has queued add eleven more. That is 53 virtual servers against a ceiling this node will not reach, and the fix is not a bigger load balancer service size, it is a second pair of Edge nodes. Scaling out beats scaling up here because Edge form factor cannot be changed under a running Supervisor without a redeploy, and a redeploy of the node hosting your Supervisor endpoint is an outage.

flowchart TD A[Workload network model] –> B[Permitted load balancer models] B –> C[Foundation LB on VLAN networking] B –> D[NSX LB on segment or VPC] B –> E[Avi LB on any model] C –> F[Size by appliance count and arm topology] D –> G[Size by Edge form factor, large minimum] E –> H[Size by controller size and SE group] F –> I[Virtual server and VIP capacity] G –> I H –> I I –> J[Supervisor activation] I –> K[Namespace and VKS cluster provisioning] I –> L[LoadBalancer services and add on services]
Sizing flows one way. Network model constrains the load balancer model, the model dictates which knob exists, and the knob sets the capacity every Supervisor function draws from.

War story, and it cost a customer a weekend. A platform team I was helping in March had a management cluster under memory pressure, so when they built the workload domain they deployed three NSX Edge nodes at medium form factor, 4 vCPU and 8 GB each, reasoning that their traffic volumes were tiny and a medium Edge would be generous. Traffic volumes were tiny. That was never the constraint. Supervisor enablement reached the point of publishing its control plane endpoint and sat there, and vCenter reported Configuring for two hours with no error worth reading. We spent another three hours chasing certificates and DNS before someone opened the Broadcom design requirement and read VCF-SUP-NLB-REQD-CFG-003, which states plainly that Edge nodes must be at least large form factor because NSX load balancers take fixed resource allocations on the Edge. Redeploying three Edge nodes at large, re-establishing the tier 0 peering and reactivating the Supervisor took nine hours end to end. Decision we reversed: we stopped sizing Edge nodes from expected throughput and started sizing them from the Supervisor requirement first, throughput second. Nobody on that team has built a medium Edge since.

One honest inconsistency to carry into the exam room rather than be surprised by. Avi product documentation names four controller sizes, Essentials, Small, Medium and Large. VCF 9.1 design library, describing the controller deployed and lifecycled by VCF Operations, lists Small, Large and X-Large instead. Both are current, they describe different deployment paths, and neither is a typo. If an item gives you controller size options, take your cue from whether the scenario deploys Avi through VCF Operations or standalone.

Exam focus for objective 3.1

Objective 3.1, Evaluate the implication for Load Balancer size on Supervisor functionality: what this objective expects you to be able to do is read a described estate, spot which load balancer knob is undersized, and name the Supervisor function that fails as a result. Item types this shows up in are scenario multiple choice, multiple selection where several Supervisor functions are affected by one shortfall, and matching, where a column of symptoms is paired against a column of sizing causes. Trap that catches experienced admins: sizing from bandwidth. Years of load balancer work teach you to size for requests per second and concurrent connections, and on this objective that instinct is actively wrong. Supervisor consumes load balancer objects, so a quiet estate with many namespaces exhausts capacity while the data plane idles. Second trap, smaller but sharper: assuming any of these knobs is adjustable later. Avi Controllers have no CPU or memory hot add, and an Edge form factor change means redeploying the node your Supervisor endpoint lives on.

Objective checkpoint

Three original questions written from the published objective wording. No real exam items are reproduced here.

1. A workload domain runs NSX segment networking on three Edge nodes at medium form factor. Supervisor activation stalls indefinitely at Configuring. What is the most likely cause?
Answer: Edge nodes are below the minimum large form factor required for the NSX Load Balancer. Reasoning: NSX load balancers take fixed resource allocations on the Edge, and Broadcom states large as the floor for Supervisor, so no amount of traffic tuning helps.

2. An estate must front 140 Service Engines and the current Avi Controller is Small. Which change is required, and what is the operational cost?
Answer: Move to a Medium Controller, which means a redeploy. Reasoning: Small tops out at 100 Service Engines and Medium covers 100 to 200, but there is no CPU or memory hot add for the Controller, so size is not adjustable in place.

3. A team wants one load balancer to serve both Supervisor workloads and a set of traditional virtual machines, and the workload network is NSX VPC. Which model satisfies this?
Answer: Avi Load Balancer. Reasoning: NSX Load Balancer and Foundation Load Balancer are both documented as not shared with non Supervisor workloads, and Foundation Load Balancer is restricted to VLAN networking regardless.

Sizing call for this estate

For the reference estate, which is NSX VPC networking with Avi in the path, my call is a three node Avi Controller cluster at Medium with 256 GB of disk per node, and Service Engine groups scaled out rather than up. Medium costs four cores more than Small and buys a Service Engine ceiling of 200 instead of 100, which is the difference between a sizing decision you make once and a redeploy you schedule in eighteen months. Option to avoid is Essentials. It looks attractive for a lab, it caps at 100 virtual services and 10 Service Engines, and every estate I have seen start there has outgrown it inside a year with no in place upgrade path.

If you run NSX Load Balancer instead, the call is simpler and less negotiable: large Edge nodes, always, and scale out to more Edge pairs rather than trying to scale a node up. Foundation Load Balancer stays where it belongs, on VLAN networking, in a two virtual machine active and passive pair, for estates that will never grow past a handful of namespaces.

Do this tonight before you read the next Part: run the lb-node-usage call against every Edge node carrying a Supervisor, write down the virtual server count and the usage percentage, then add up the clusters and Services your roadmap says you will add in the next two quarters. If that total lands above 75 percent of current capacity, you have a design action, not a monitoring alert. Underlying component mechanics live in the VKS Series and the NSX Series if you need to go a level deeper on either.

VCAP-VKS Exam Series · Part 11 of 34
« Previous: Part 10  |  Guide  |  Next: Part 12 »

References

About The Author


Discover more from Journal of Intelligent Infrastructure

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure

Subscribe now to keep reading and get access to the full archive.

Continue reading