Dr. Pranay Jha

Picture the call I get more than any other: a segment goes dark, someone restarted an NSX Manager node, and the team is convinced the network is down. It usually is not. Existing flows keep forwarding because the data plane does not stop when the control plane hiccups. Knowing that, cold, at 2 a.m. is the difference between a calm five-minute check and a panicked failover that makes things worse. That instinct comes from understanding how NSX 9 splits work across three planes and what each one is actually responsible for. So before we deploy anything in later parts, this is the map.

Three planes, and why NSX bothers to split them

The separation is not academic. The management plane is where intent lives: the API, the UI, your declared desired state. The control plane computes how to realize that intent and distributes it. The data plane does the actual packet work. Splitting them means the part that forwards traffic at line rate does not depend, moment to moment, on the part that draws the topology in a browser. A Manager node can reboot and your VMs keep talking. That decoupling is the whole reason a software-defined network can be operated without holding your breath every time you log in.

What changed from the NSX-T mental model is consolidation. There is no separate Policy Manager VM and Controller VM to reason about anymore. In NSX 9 the management and control roles are converged onto a single appliance type, deployed as a three-node cluster. You still think in three planes; you just deploy fewer moving parts.

The converged NSX Manager cluster

The NSX Manager cluster is three nodes, and each node carries the full stack of roles: the policy role (the declarative intent layer you and your automation talk to), the manager role (API, UI, and operational tasks against every node), the controller role (the Central Control Plane), an HTTPS front end, and a replicated desired-state datastore shared across the cluster. Three nodes is not arbitrary. The datastore needs a quorum, so you run an odd number, and three is the supported production cluster size. Lose one node and the remaining two keep quorum and the platform stays writable. Lose two and you drop below quorum and are read-only at best.

Clients reach the cluster through a single point, either an NSX cluster VIP (one node owns the VIP at a time) or an external load balancer in front of all three. The VIP is simple and needs all three nodes on the same subnet; the external load balancer is what I recommend when you want real distribution of API and UI load or nodes across subnets. Pick deliberately, because retrofitting a load balancer onto a VIP-based cluster later is avoidable rework. One more non-negotiable: keep all three nodes on solid, identical time. NTP skew across cluster members is a classic root cause of flaky control-plane behaviour that looks like everything and nothing.

How the control plane reaches the data plane

The Central Control Plane runs inside the Manager cluster and does the heavy lifting of turning policy into stateless configuration: which segments exist, which logical ports map where, what the forwarding and firewall state should be. It does not push packets. It computes and distributes. On the receiving end, every transport node, host or Edge, runs a Local Control Plane agent (you will see it as nsx-proxy in the CLI) that takes the CCP’s configuration and programs the local datapath. The CCP is the brain; the LCP is the hands on each node.

This is exactly why a Manager outage is survivable. The CCP distributes desired state, but the LCP has already programmed the datapath, so established forwarding continues. What you lose during a full control-plane outage is the ability to change things and to converge on new state: a freshly powered-on VM may not get its logical port realized, a new segment will not push out, a DFW rule edit will not take effect. That is a serious problem, but it is a different problem from a hard data-plane outage, and treating them the same is how teams turn a minor event into a major one.

What actually survives a failure

Because the planes are decoupled, the honest answer to “is the network down?” depends entirely on which plane is hurting. This is the table I want every operator to have memorized before the pager goes off.

Check the cluster from the CLI

You do not need the UI to know whether the cluster is healthy. SSH to any Manager node as admin and use the NSX CLI. This is the first thing I run when someone reports “NSX is acting up”, before touching anything.

# SSH to any manager node, then use the NSX CLI
ssh admin@nsx-mgr-01.lab.local

# Overall cluster health: every group should read STABLE
get cluster status

# Per-node detail: UUID, IP, and the roles each node holds
get cluster config

# Confirm these group types are all STABLE
#   MANAGER  POLICY  CONTROLLER  DATASTORE  HTTPS  CLUSTER_BOOT_MANAGER

If get cluster status shows every group type STABLE across all three nodes, the control and management planes are healthy and your problem is almost certainly elsewhere (transport node, Edge, routing, or MTU). If DATASTORE or CONTROLLER is degraded, that is your lead. Reading these two commands correctly saves you from rebooting a healthy cluster because a dashboard tile looked yellow.

Sizing the Manager cluster

The Manager appliance comes in four form factors. In VCF 9 deployment, Medium is auto-selected and is the right default for typical production. Small exists for labs and proof-of-concept work and is explicitly not for production. Large and Extra Large are for high object counts and large fleets. The practical detail that saves you grief: you can resize between Small, Medium and Large in place, but moving to or from Extra Large means redeploying the appliance. So if you have any real expectation of hitting XL scale, plan for it at design time rather than discovering the redeploy during a change window.

One more thing people get wrong: all three nodes carry the same roles and the same load profile, so size them identically and give them reserved resources. Starving an NSX Manager node of CPU or memory on a contended management cluster produces exactly the kind of intermittent, hard-to-diagnose control-plane symptoms you do not want to chase. For where these appliances land in the VCF management domain, see NSX in VCF 9 Explained.

The data plane: where packets actually move

Host transport nodes (the distributed half)

Host transport nodes are your ESX hosts, prepared for NSX so the VDS plus EDP carries overlay and VLAN traffic and the Distributed Firewall enforces in the kernel right at the vNIC. East-west forwarding and security happen on the same host the workload runs on, with no hairpin to a central appliance. Add hosts and you add east-west capacity automatically; the firewall and routing scale out with the cluster.

Edge transport nodes (the concentrated half)

Edge transport nodes are the concentrated half: they run the Tier-0 gateways that talk BGP to your physical fabric and host the stateful services (NAT, DHCP, VPN, gateway firewall) that have to live somewhere centralized. Edges come as VM or bare-metal and are grouped into Edge clusters for availability. This is where the scaling asymmetry lives, and it is the single most common design miss I see.

The design consequence worth internalizing: east-west scales with your hosts because it is distributed, but north-south scales with your Edges because it is concentrated. Under-build the Edge tier and you bottleneck every flow leaving the overlay, no matter how much host capacity you have. That trade-off gets its own deep dive later in the series (Part 7 and Part 26), but it starts here, in the architecture.

What I’d Do

Build the three-node Manager cluster at Medium for any real deployment, put it behind an external load balancer if you have one rather than leaning on the VIP, keep the nodes on identical NTP, and reserve their resources so the control plane is never fighting for CPU. Then drill the failure table above with your team until everyone knows, without thinking, that a single Manager reboot does not drop traffic and that get cluster status is the first command, not the reboot button. The architecture already gives you the resilience; most outages I get called into are people not trusting it and forcing an action that was never needed. Next up is Part 3: licensing and editions, and how NSX is consumed in VCF versus standalone. What is your current Manager cluster sitting behind, a VIP or a load balancer?

Why the plane separation matters when something breaks

The management, control and data plane split is not academic taxonomy; it is the first thing that localizes an outage. The management plane is where you configure and where the API lives, the control plane distributes state to the data plane, and the data plane is what actually forwards packets in the hypervisor and on the Edges. The crucial operational consequence is that these fail independently.

An NSX Manager cluster being unreachable is a serious management-plane problem, but your existing workloads keep forwarding because the data plane already has its state and does not need the Manager to move packets. Conversely, a data-plane issue on a host drops real traffic while the Manager looks perfectly healthy. When you internalize which plane owns which symptom, you stop wasting the first twenty minutes of an incident looking in the wrong place. The question I ask first is always the same: is this a configuration-and-management symptom, or a forwarding symptom, because the answer sends me to a completely different part of the stack.

References

• NSX Management and Control Plane Models (Broadcom TechDocs, VCF 9)
• NSX Manager VM and Host Transport Node System Requirements (NSX 9.0)
• Resize an NSX Manager Node (Broadcom TechDocs, VCF 9)

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts