Dr. Pranay Jha

Your cluster is up and your pods are running, but nothing outside can reach them until you solve load balancing and ingress. In VKS this is a platform decision as much as a Kubernetes one, because the external IPs your services receive come from the Supervisor’s load balancer, not from something inside the cluster. This part untangles the Layer 4 options from Avi’s Layer 7, and explains when the extra component earns its place. It pairs with my VCF 9 comparison of Avi versus NSX native load balancing.

How a LoadBalancer service actually works in VKS

When a developer creates a Service of type LoadBalancer, Kubernetes asks the platform for an external IP, and that request is satisfied by whatever load balancer the Supervisor is configured with. On VDS networking the Foundation Load Balancer is the default and provides Layer 4 VIPs. With NSX, the NSX native load balancer can do the same. If Avi is configured, the Avi controller provisions the virtual service instead. The developer’s experience is identical, a standard Kubernetes object, the difference is which engine answers and what it can do. This is also why the load balancer must exist and be healthy before services work: a cluster whose Supervisor load balancer is misconfigured shows services stuck in <pending> forever, waiting for an IP that never arrives. It is one of the first things to check when external access fails (Part 15).

Layer 4 defaults versus Avi’s Layer 7

The Foundation and NSX native load balancers are the path of least resistance, no extra product to license or operate, and for many internal platforms a Layer 4 VIP per service is entirely sufficient. Avi is the step up. Through the Avi Kubernetes Operator (AKO), it provides L4 Service type LoadBalancer plus L7 ingress and Gateway API support, a dedicated scalable data plane, per-VIP analytics, and a path to WAF. AKO runs as an operator tied into the VKS environment, and in the NSX VPC model the load-balancer services are created in the Avi tenant that corresponds to the NSX project. The trade-off is another component to license and run in exchange for materially richer capability and visibility.

Ingress: turning one IP into many routed apps

A LoadBalancer service gives you an IP per service, which gets expensive and unwieldy once you have many HTTP apps. For routing by host and path you want L7 ingress. With Avi, AKO handles ingress and Gateway API objects directly, mapping them to Avi virtual services. Without Avi, you deploy a standard ingress controller, Contour and NGINX are both common on VKS, behind a single LoadBalancer service and route internally from there. Either way, the ingress controller or Gateway implementation is the thing that turns one external IP into many routed applications.

How AKO maps Kubernetes objects to Avi

If you choose Avi, it is worth understanding what the Avi Kubernetes Operator actually does, because the mental model prevents most of the confusion. AKO runs inside the cluster and watches for Kubernetes objects, Services of type LoadBalancer, Ingress resources, and Gateway API objects. When it sees one, it translates it into the corresponding Avi construct: a LoadBalancer Service becomes an L4 virtual service, an Ingress becomes an L7 virtual service with pools and routing rules, and the Avi controller programs its data plane (the Service Engines) to serve the traffic. In the NSX VPC model the load balancer services land in the Avi tenant that corresponds to the NSX project, which keeps each project’s load balancing isolated.

The consequence for you is that the developer still writes plain, portable Kubernetes, an Ingress is an Ingress, and AKO does the Avi-specific work behind the scenes. That is exactly the property you want: no Avi-specific annotations sprinkled through every manifest, no lock-in at the application layer. When something does not appear in Avi, the question is almost always whether AKO is healthy and watching the right namespaces, not whether the developer’s manifest is wrong. Check AKO first.

TLS termination and certificate management

Where you terminate TLS is a decision the Layer 4 path quietly forces and the Layer 7 path lets you make. With a plain L4 LoadBalancer, the load balancer passes encrypted traffic straight through and your pods terminate TLS themselves, which means every app owns its own certificate lifecycle. That is fine for a handful of services and tedious at scale. With Avi at Layer 7, you can terminate TLS at the virtual service, centralise certificate management, and even re-encrypt to the backend if policy requires end-to-end encryption. Pair that with cert-manager inside the cluster for automated certificate issuance and renewal, and the certificate churn that plagues hand-managed setups largely disappears.

My guidance: for more than a few HTTPS services, terminate at L7 and automate issuance with cert-manager, because the alternative is a slow accumulation of expired-certificate incidents, each one an outage that was entirely predictable. The one case to terminate in the pod is a genuine end-to-end encryption mandate where even the load balancer must not see plaintext, and even then Avi can re-encrypt rather than forcing per-pod certificate management.

Sizing and placing the load balancer data plane

One thing the comparison table does not capture is that Avi’s data plane is real infrastructure you have to size and place. The Service Engines that actually move the packets consume CPU, memory and network, and they need addresses and a placement that puts them in the traffic path. For a small platform this is trivial; for a large one carrying serious throughput, it is a capacity-planning exercise in its own right, and undersizing the Service Engines shows up as latency and dropped connections under load, not as an obvious error. The built-in Layer 4 balancers do not impose this because they ride on infrastructure that already exists, which is part of why they are the cheaper default.

So when you decide to bring Avi in, budget for the data plane as well as the license. Size the Service Engines to your peak throughput with headroom, place them so they actually sit in the path of the traffic they serve, and monitor them as the first-class infrastructure they are. The analytics Avi gives you back are excellent for exactly this, per-virtual-service latency and connection data that tells you when you are approaching a limit before it becomes an incident, which is one of the strongest reasons to choose it in the first place.

What I’d Do

For ordinary internal apps, I start with the Layer 4 default, Foundation or NSX native, plus a lightweight ingress controller like Contour for host and path routing. That covers most platforms cheaply. I reach for Avi the moment Layer 7 features, per-VIP analytics or WAF are a genuine requirement, or when I am consolidating load balancing across many clusters and want one control point with real visibility. What I do not do is buy the advanced data plane to serve three internal services, nor hand-roll in-guest kube-vip to dodge a license and then inherit its lifecycle. Match the tool to the actual requirement. When a service of yours sticks in pending, do you know which load balancer is supposed to answer it, and whether it has IPs left to give?

References

• VMware Load Balancing Blog: Modernize and Secure Kubernetes Ingress for VKS with Avi
• Broadcom TechDocs: Deploying vSphere Supervisor with NSX and Avi Load Balancer
• Broadcom TechDocs: Secure VKS Cluster Ingress

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts