TL;DR · Key Takeaways
- VKS (vSphere Kubernetes Service) runs Kubernetes on VCF 9, and NSX is the layer that gives its pods real networking and security, not just an overlay bolted on.
- Antrea is the default CNI for VKS clusters. Its value over a generic CNI is the deep NSX integration through the Antrea-NSX Adapter, which surfaces pod-level objects and policy into NSX.
- The Antrea-NSX Adapter runs as a pod inside the VKS cluster. Its management-plane component talks to the NSX Policy API, the Kubernetes API server, and the Antrea Controller, so NSX sees Kubernetes the way it sees VMs.
- VKS clusters land in NSX VPCs. During Supervisor activation you pick VCF networking with VPC, and the Projects and VPCs from Part 22 become the home for your Kubernetes namespaces and clusters.
- You can bring your own CNI (Calico, Cilium) since VKS 3.6, and VKS stays CNCF-conformant. You trade native NSX visibility and the adapter integration for it. Default to Antrea unless you have a concrete reason not to.
Most Kubernetes-on-vSphere conversations start in the wrong place. Teams argue about the CNI before they have decided whether they want their pod network to be a sealed island or a first-class citizen of the data center network. That ordering is backwards. The interesting question in NSX 9 and VKS is not “which CNI” but “do you want NSX to see and govern your pods the same way it sees your VMs.” Answer that, and the CNI choice mostly answers itself.
Where NSX fits in the VKS stack
VKS sits on top of the vSphere Supervisor, which is the Kubernetes control plane embedded in vSphere. The Supervisor provisions workload clusters, and those clusters need three things from the network: pod-to-pod connectivity (the CNI), north-south access and load balancing, and security policy. NSX provides the substrate for all three. The segments, gateways, and load balancing you have read about for VMs in this series are the same machinery that VKS consumes underneath. What changes is the consumption model: instead of you hand-building segments, the Kubernetes objects drive NSX through automation, so a namespace or a service turns into NSX networking and policy without a network engineer clicking through the UI for each one.
This is the same VKS networking story I opened from the Kubernetes side in the VKS networking Part. Here we look at it from the NSX side: what NSX actually does for Kubernetes, and how the integration is wired.
Antrea and the Antrea-NSX Adapter
Antrea is the default CNI for VKS clusters, and it is a capable CNI on its own merits, built on Open vSwitch with its own NetworkPolicy support. But the reason Antrea is the default in a VCF context is the Antrea-NSX Adapter, which is what turns a generic Kubernetes cluster into something NSX can see and govern. The adapter is deployed as a pod inside the target VKS cluster. Its management-plane component communicates with three things at once: the NSX Policy API (remember, Policy API only in NSX 9), the Kubernetes API server, and the Antrea Controller. That three-way link is what lets NSX learn about pods, namespaces, and Kubernetes NetworkPolicies, and lets NSX-defined intent flow back down into the cluster.
The practical payoff is one policy and visibility plane. Your security team can write and see policy for pods using the same NSX constructs and the same distributed firewall thinking from the micro-segmentation Part, instead of pods being a black box that the VM-centric firewall cannot reach into. For a security org that has standardized on NSX, that single pane is worth more than any individual CNI feature.
VKS clusters live in NSX VPCs
This is where Part 22 pays off. In VCF 9, Supervisor activation offers a networking option called VCF networking with VPC, and choosing it places your Supervisor and its VKS clusters inside the NSX Projects and VPCs you designed earlier. A Kubernetes namespace maps onto VPC networking, the cluster gets native virtual networks from the VPC, and the whole thing consumes the same Transit Gateway and subnet model as everything else. The big simplification in VCF 9 is that Supervisor networking stopped being a bespoke configuration and became just another VPC consumer, alongside vCenter and VCF Automation.
The design consequence is real: your Kubernetes tenancy and your NSX tenancy are now the same model. A team that gets a Project for its VMs can get its VKS clusters in the same Project, under the same RBAC and quotas, on the same subnets typed the same way. That alignment is the point of building on VPCs rather than treating Kubernetes as a separate networking universe.
Antrea or bring your own CNI
VKS is CNCF-conformant, and since VKS 3.6 you can bring your own CNI such as Calico or Cilium inside the guest clusters. That flexibility is real and occasionally the right call. But it is a trade, and you should make it with eyes open. Choose a non-default CNI and you keep its ecosystem features, but you give up the native NSX visibility and the Antrea-NSX Adapter integration that makes pods first-class NSX citizens. Your security team loses the single pane and goes back to governing pod traffic with a separate tool and a separate mental model.
| Dimension | Antrea (default) | Bring your own (Calico/Cilium) |
|---|---|---|
| NSX integration | Native via Antrea-NSX Adapter | None of the adapter benefits |
| Pod visibility in NSX | Yes, pods as NSX objects | No |
| Security policy plane | Unified with VM DFW thinking | Separate tool and model |
| Support / default path | The supported default | Allowed since VKS 3.6, you own it |
| When I pick it | Almost always on VCF | A specific CNI feature you cannot live without |
My take
On VCF, the Antrea default is the right default, and I push back on teams that reach for Cilium reflexively because it is fashionable. If your organization runs NSX and wants one security plane across VMs and pods, that integration is the whole reason you are on this platform instead of rolling vanilla Kubernetes on bare metal. Bring your own CNI only when a specific, named feature gap forces it, and accept that you are stepping outside the integrated story when you do.
What I’d Do
Treat Kubernetes networking on VCF 9 as an NSX design problem, not a separate Kubernetes one. Choose VCF networking with VPC at Supervisor activation so your clusters share the Project, VPC, and RBAC model with your VMs from day one. Keep Antrea as the CNI so the Antrea-NSX Adapter gives your security team one policy and visibility plane across pods and VMs, and only bring your own CNI when a concrete feature gap demands it. The reward for designing it this way is that Kubernetes stops being a networking island your firewall cannot reach, and starts being just another set of workloads under the same NSX governance as everything else. That is the entire reason to run VKS on VCF rather than somewhere else. Are your pods first-class NSX citizens, or a blind spot?
Operating NSX-backed Kubernetes day to day
Once VKS is networked through NSX, the interesting operational questions stop being about the control plane and start being about addresses, visibility and version pairing. These are the things that decide whether the platform scales quietly or surprises you in production.
IP address management is the silent scaler
Pods draw their addresses from the VPC subnet pools, and that is where Kubernetes-on-NSX projects quietly run out of room. If the VPC CIDR or the per-namespace subnet size is set too small at activation, new clusters or scaled-out deployments simply fail to get IPs, and the symptom looks like a scheduling failure rather than a network one. The team chases the wrong layer for hours. Size the VPC address space for the number of clusters, namespaces and pods you expect to run, with real headroom, because every node, every pod and every service consumes from that space and growing the allocation later is disruptive. This is the IPAM planning the VPC design from Part 22 was quietly setting you up for.
Where to look when pod traffic breaks
Pod connectivity issues live in one of three layers, and checking them in order localizes almost everything. First, the Antrea agent and the Antrea-NSX adapter health inside the cluster: if the adapter pod is unhealthy, NSX stops learning about pods. Second, the NSX realized state for the namespace networking: unrealized intent here means the policy you think is enforced is not. Third, the segment and VPC the namespace maps to: the same overlay and firewall realities from the rest of this series apply to pod traffic once it leaves the node. Work top down through those three and you avoid the trap of debugging Kubernetes when the problem is NSX, or the reverse.
Version pairing on upgrades
The CNI rides the cluster and the integration rides NSX. Upgrading a VKS cluster moves Antrea forward, while the Antrea-NSX adapter has a supported pairing with the NSX version underneath. During upgrades, keep both inside their validated compatibility window rather than letting one race ahead of the other. A mismatched adapter is the kind of fault that passes every quick smoke test and then drops a fraction of pod flows under real load, which is the worst kind of fault to find late.
Worked example
Plan for 5 workload clusters, each scaling to roughly 250 pods, plus nodes and services. That is about 1,250 pod addresses before you count services and node IPs, so a VPC private space sized as a /20 (around 4,000 usable) gives comfortable headroom for growth and churn, where a /23 would strand you the first time two teams scale at once. Size for the cluster count times the per-cluster pod ceiling, then add a generous margin, because re-addressing a live Kubernetes estate is exactly the surgery the VPC model exists to avoid.
Bringing your own CNI changes the operating model, not just the data plane
If a team does choose Calico or Cilium over Antrea, the consequence is not only technical, it is operational. Pod visibility and pod-level policy move out of NSX and into that CNI’s own tooling, which means your security and network operations now span two consoles and two mental models instead of one. The platform team watches VM traffic in NSX while the Kubernetes team watches pod traffic somewhere else, and the seam between them is where incidents hide. That split can be the right call when a specific CNI capability is non-negotiable, but it should be a deliberate, documented decision with an owner for each side of the seam, not a default someone picked because a blog post recommended it. Decide who watches pod traffic, where, and how it correlates with the NSX view, before the first cluster goes to production.
Treat the Supervisor networking choice as irreversible
The single decision that is hardest to walk back is the Supervisor networking model chosen at activation. Moving a Supervisor from a legacy networking mode onto VCF networking with VPC later is not a setting you toggle; it is effectively a rebuild of the Supervisor and everything that depends on it. So even if you have one tenant and one cluster today, if multi-tenancy or shared VM-and-Kubernetes tenancy is anywhere on the roadmap, choose VPC networking up front. The cost of choosing it early when you do not strictly need it is near zero; the cost of retrofitting it after you have running workload clusters is a migration project. That asymmetry should decide it for almost everyone building net-new on VCF 9.
References
- Broadcom TechDocs: VKS Clusters Antrea CNI Integration
- Broadcom TechDocs: Enable the Antrea-NSX Adapter for a VKS Cluster
- VMware Cloud Foundation Blog: Architecting VKS on VCF
About The Author
