TL;DR · Key Takeaways
- NSX Federation gives you one policy plane across multiple sites. You configure on a Global Manager, set each object’s span, and the config synchronizes to the Local Manager at each location.
- Span is the core concept. A segment, Tier-0, or firewall rule exists at exactly the locations you assign it to. Get span wrong and you either under-protect a site or stretch something that should have stayed local.
- The latency budget decides whether Federation is even on the table. Federation needs RTT under 150 ms between locations and at least 1 Gbps of bandwidth. That is generous, but it is a hard ceiling.
- Stretched networking (a segment live in two sites at once) is powerful for mobility and DR, and it is also how you import a failure in one site into another. Stretch deliberately, not by default.
- Federation is not a high-availability cluster and not a backup. The Global Manager has its own active and standby model, and you still need per-site resilience and backups underneath it.
What is the real cost of running two data centers as one network? Most teams answer that with a wishlist (consistent policy, workload mobility, one firewall rule base) and skip the line item that actually decides it: latency. NSX Federation can absolutely give you a single, consistent network and security policy across sites. Whether it can do that for your sites comes down to a number you measured, or should have, before anyone drew a topology. So let me start where the design really starts, which is the link between the sites, and work back to the architecture from there.
Global Manager, Local Managers, and span
Federation introduces a Global Manager that sits above the Local Manager at each site. You do your networking and security configuration once, on the Global Manager, and assign each object a span: the set of locations where it should exist. Create a Tier-0 and span it across Location 1, Location 2, and Location 3, and that configuration synchronizes down to all three Local Managers. The Local Managers still run the data plane at each site; the Global Manager is the intent layer that keeps them consistent. The mental model that works is this: the Global Manager holds the desired state for the whole estate, and span is how you tell it which slice of that state belongs where.
Span is the concept that earns its own design session. A firewall rule with the wrong span protects the wrong sites. A segment spanned everywhere when it only needed to be local has quietly become a cross-site dependency. I tell clients to treat span as an explicit decision per object, not a default, because the easiest Federation mistake is stretching things out of convenience and discovering the coupling only when one site has a bad day.
The latency budget is the gate
Before you fall in love with a stretched topology, check the link. NSX Federation requires RTT under 150 ms between locations and at least 1 Gbps of bandwidth, and the 150 ms ceiling applies between Edge nodes that carry the cross-site traffic. That number is the gate. If your inter-site RTT is comfortably under it, Federation is viable. If it is borderline or variable, you are designing on sand. And Federation’s 150 ms is only one figure in a stack of latency limits that govern a multi-site VCF, each with a different and stricter threshold depending on what is synchronizing across the link.
| What is crossing the link | Max RTT | Why |
|---|---|---|
| NSX Federation (between Edge nodes) | 150 ms | Cross-site routing and policy sync |
| vSAN stretched cluster (data) | 5 ms | Synchronous mirroring |
| Stretched cluster witness | 200 ms | Metadata only, quorum |
| Workload Domain Day-2 ops, vMotion | 100 ms | Provisioning, migration |
| Fleet to instance (VCF 9) | 300 ms | Management plane reach |
Worked example
Two sites 1,200 km apart on good fiber measure roughly 18 ms RTT. Federation’s 150 ms ceiling is no problem, and a stretched cluster’s 5 ms requirement is also met, so both a stretched cluster and Federation are technically on the table. Now two sites on different continents at 130 ms RTT: Federation is still viable (under 150 ms), but the 5 ms stretched-cluster limit is blown out of the water, so synchronous storage stretching is off, and your DR design has to lean on asynchronous replication instead. Same Federation, completely different storage and DR architecture, decided entirely by one measured number.
Stretched networking: power and the price
The marquee Federation feature is the stretched segment: one logical network live in two or more sites at once, so a VM keeps its IP and its security policy when it moves or fails over. Pair that with a Tier-0 spanning the sites and you get cross-site routing without re-IP’ing anything. For workload mobility and for disaster recovery where you want the network already present at the recovery site, this is genuinely useful and hard to replicate cleanly any other way.
Where stretched networking helps
It shines when the requirement is mobility or recovery without network surgery: live or cold migration between sites, active-passive DR with the segment pre-staged at the standby, or an application that must present the same subnet in two places. In those cases the stretched segment removes the most painful part of multi-site, which is re-addressing.
Where it hurts
A stretched segment is a shared fault domain. East-west traffic between two VMs on the same stretched segment but in different sites crosses the inter-site link every time, so a chatty app that you spread across sites quietly pays the RTT on every packet. Worse, a layer-2 problem or a broadcast storm now has a path between your sites that did not exist before. The honest verdict: stretch the segments that a specific mobility or DR requirement justifies, and keep everything else local. Default-stretch is how a two-site design becomes one big failure domain wearing a multi-site costume.
Federation is not a backup, and not always the answer
Two things teams get wrong about what Federation gives them. First, it is not a backup. The Global Manager keeps your sites consistent, which means a bad configuration is consistently bad everywhere, fast. You still need the per-site backup and restore discipline from Part 19, and you still need each site’s NSX to be resilient on its own. The Global Manager has its own active and standby instances, but that protects the intent layer, not your data.
Second, Federation is not always the right tool for multi-site. If your sites are genuinely independent, with separate teams, separate change windows, and no need for stretched networks or one shared rule base, plain NSX multisite (or just two independent NSX deployments coordinated by automation) can be simpler and more resilient, precisely because it removes the cross-site coupling. I reach for Federation when consistent policy across sites or stretched mobility is a real, stated requirement. I do not reach for it just because there is more than one data center on the diagram. The VCF stretched-cluster and multi-site design from the VCF 9 series is the wider context this sits inside.
Final Thoughts
Federation is the right answer to a specific question: how do I run consistent network and security policy, and optionally stretched networks, across sites that are close enough in latency to act as one. Start by measuring the inter-site link, because the 150 ms ceiling and the stricter storage limits decide your whole architecture before you configure a thing. Treat span as a deliberate per-object decision. Stretch only the segments a real mobility or DR requirement justifies, and keep the rest local so you are not building one large fault domain. And remember Federation makes you consistent, not safe, so the per-site resilience and backups still have to be there underneath. Done with discipline, it is the cleanest way to run a global private cloud network. Done by default, it just spreads your problems faster. Have you actually measured your inter-site RTT, or are you guessing?
Onboarding a location and the order of operations
Building a federation is a sequence, and doing it in the wrong order produces confusing half-states. You deploy the Global Manager, then register each site Local Manager as a location, and only once the location set exists do you start assigning span to objects. The mental model is that you construct the stage first, the set of locations, and then place the actors, the segments and gateways and firewall rules, onto whichever locations they belong to. Trying to stretch objects before the locations are cleanly registered is how you end up with intent that cannot fully realize because it references a location the Global Manager does not yet consider healthy.
That ordering also shapes how you grow a federation. Adding a third site later means registering it as a location first, confirming it is healthy from the Global Manager, and then extending the span of the specific objects that should reach it, deliberately, one decision at a time. Resist the urge to broaden span in bulk just because a new location appeared. Each object that gains a location gains a cross-site dependency, so the onboarding of a site is a good moment to review span rather than to rubber-stamp it. Build the location, then choose, per object, whether it genuinely belongs there.
Version skew across locations is a real constraint
A federation is only as healthy as the compatibility between its locations, and version skew is the constraint that quietly limits you. All locations need to run NSX versions that are compatible with each other and with the Global Manager, which means upgrades across a federation are a coordinated exercise rather than a per-site free-for-all. Letting one location race far ahead on version, or fall far behind, puts the whole federation into an unsupported or degraded posture, and the symptoms of that are exactly the kind of intermittent sync problems that are miserable to diagnose.
The practical discipline is to plan federation upgrades as a fleet activity with a defined order and a bounded window of skew, the same way you would plan any coordinated multi-component upgrade. Keep the locations within their supported version relationship, upgrade them in the prescribed sequence, and treat a location that has drifted out of the supported window as a problem to fix before you do anything else clever. Federation buys you one consistent policy plane across sites, and the price of that consistency is that the sites cannot wander independently on version. Coordinate the lifecycle and the consistency keeps working; ignore it and the consistency is the first thing to break.
Measure the latency you have, not the latency you hope for
Federation lives or dies on the inter-site link, and the number that matters is the one you measured over a representative window, not the one on the circuit order. A link that usually sits comfortably under the latency ceiling but spikes during backups or peak load is not, for design purposes, under the ceiling, because the spikes are exactly when consistency and forwarding will struggle. Measure round-trip time across a real day, including the busy and the batch-heavy parts, before you commit to a federated topology.
The same honesty applies to bandwidth. The cross-site replication and policy sync need real headroom, not the theoretical capacity of the link minus everything else that shares it. Design against the worst representative case you actually observed, leave margin, and treat a link that only sometimes meets the requirement as a link that does not meet it. Federation rewards a sober assessment of the connectivity you genuinely have, and punishes a design built on the optimistic average, because the average is never what is in effect when something goes wrong.
References
- Broadcom TechDocs: Understanding NSX Federation
- Broadcom TechDocs: NSX Multisite
- VirtuallyWired: VCF 9.0 and Global Private Cloud Latency Thresholds
About The Author
