TL;DR · Key Takeaways
- Workload domains are not part of reaching 9.1. They are an optional Day-N job, so your fleet is fully on 9.1 once the management domain is done, and you can upgrade workloads on your own schedule.
- Each workload domain follows the same core order you used on the management domain: NSX Manager, then vCenter, then ESX hosts, then the NSX Edge cluster last to finalize NSX.
- VCF 9.1 raises parallel upgrade capacity to up to 256 clusters at once, a 4x jump. On a large estate that is the difference between one window and several.
- You still plan each domain separately in VCF Operations, set the target version, and run prechecks per domain before you start.
- If an NSX Manager is shared by several workload domains, you upgrade that NSX once, and the domains that share it move together.
- vSphere Lifecycle Manager baselines are gone in 9.x, so any workload cluster still on a baseline has to move to a vLCM image before its ESX can go to 9.1.
Part 11 finished the mandatory scope. Once SDDC Manager and the management domain are on 9.1, your fleet is upgraded. Every workload domain, though, is still sitting on the old version until you get to it, and that is fine.
This is the part where the pressure comes off. Workload domains upgrade as a Day-N procedure, so I can spread them across change windows instead of forcing everything into one long night. What I want in this part is a clear picture of the order per domain, how the new parallelism changes planning at scale, and the handful of workload-specific traps that show up after a management domain is already on 9.1.
Day-N means you choose the schedule
In older VCF, a release upgrade meant every domain moved before you could call the upgrade complete. That is no longer true. The mandatory scope for 9.1 is the fleet level and the management domain. Workload domains are explicitly optional and can be done later. Your management plane, licensing, and lifecycle are already on 9.1 while the workloads keep running on the prior version until you schedule them.
That freedom is useful, but leaving domains behind forever is not the goal. A workload domain on the old ESX and NSX still needs patching, and running a wide version gap across domains complicates interoperability. I treat Day-N as breathing room of weeks, not months, and I upgrade the least critical domain first to build confidence before the busy ones.
The order inside a workload domain
A workload domain upgrades with the same core sequence as the management domain, because the components are the same. You plan the domain in VCF Operations, set the target 9.1 version and component versions, run the domain prechecks, and then work through NSX Manager, vCenter, the ESX hosts, and finally the NSX Edge cluster. The optimized or sequential choice for vCenter and NSX applies here too.
Shared NSX changes the picture
The wrinkle that catches people is a shared NSX Manager. In many designs one NSX Manager backs several workload domains. When that is the case you do not upgrade NSX once per domain, you upgrade it once, and every domain that rides on it is tied to that event. Plan those domains as a group and pick a window that works for all of them, because you cannot leave half of a shared NSX on the old version.
Parallelism is the real 9.1 change at scale
The reason workload domains feel different in 9.1 is capacity. VCF 9.1 raises parallel upgrade capacity to up to 256 clusters at once, roughly four times what came before. On a handful of clusters you will not notice. On an estate with dozens of clusters, that is the difference between draining them one at a time across many nights and moving through them in a few rounds.
Parallelism does not remove the per-host serial reality inside a cluster. Hosts within a cluster still roll one at a time so the cluster keeps capacity. What changed is that many clusters can run their rolling upgrades at the same time, as long as each cluster keeps enough spare hosts to evacuate one. Plan capacity per cluster, then let 9.1 run as many clusters in parallel as your maintenance window and DRS headroom allow.
Parallelism also changes how you watch the run. With one cluster you glance at it now and then. With a dozen clusters moving at once, you need the VCF Operations task view open and you triage by exception, letting the healthy clusters run and stepping in only where a cluster pauses. A single pause does not stop the others, which is the point, but it does mean a stalled cluster can hide in a wall of green if you are not watching the status column. Sort by state, deal with the one that is stuck, and leave the rest alone.
| Approach | How clusters run | Rough ESX window |
|---|---|---|
| Fully serial | One cluster at a time | about 24 h |
| Four in parallel | Two rounds of four clusters | about 6 h |
| All eight in parallel | One round, DRS headroom permitting | about 3 h |
Workload clusters need vLCM images too
The baseline rule from the management domain applies to every workload cluster. vSphere Lifecycle Manager baselines are not supported in 9.x, so any workload cluster still using baselines must transition to a vLCM image before its ESX can go to 9.1. Workload estates are where old baseline clusters tend to hide, because they were stood up years ago and never converted. Audit every workload cluster for baselines during planning, not on the night, or the ESX step will refuse to start on the clusters you forgot.
Workload-specific traps after the core is on 9.1
A few issues show up specifically once the management side is 9.1 and you start on workloads. Broadcom documents these in support articles, and they are worth knowing before you hit them rather than after.
| Symptom | Likely cause | What I check |
|---|---|---|
| Upgrade binaries missing in VCF Operations | Depot still on the old token, or wrong path | Depot uses the activation code, bundles synced |
| vCenter will not license after upgrade | vCenter not connected to the license server | License server reachable, assignment applied |
| Sync error while upgrading an ESX host | Known 9.0.2 to 9.1 host sync condition | The matching support article and retry the host |
| Cluster ESX step will not start | Cluster still on a vLCM baseline | Every cluster transitioned to an image |
Sequence the domains, not just the components
Most guidance stops at the component order inside a domain, but on a real estate the harder question is which domain goes when. I build the domain schedule around three things: shared NSX groupings, business criticality, and capacity for parallel work. Shared NSX comes first because it removes choices for you, several domains have to move together. After that I order the remaining domains and groups from least critical to most, so the riskiest changes happen only after the process has already worked somewhere safe.
The temptation on a big estate is to fan every cluster out at once because 9.1 lets you. I hold back on the first domain. A smaller fan-out on a quiet domain gives you a clean rehearsal, and if a precheck or a baseline surprise shows up, it shows up where it costs an hour, not where it wakes an on-call rotation. Once one domain has gone end to end, I widen the parallelism for the rest.
Run prechecks per domain, every time
A clean management-domain precheck does not vouch for a workload domain. Each domain has its own vCenter, its own clusters, and its own drift, so it gets its own precheck. I run it a day ahead where I can, because the two things it tends to flag, an expiring certificate and a baseline cluster, both take real time to fix and neither is something you want to discover at the start of the window. Rerun the precheck after every fix until it is clean, then schedule the domain.
Backups deserve the same per-domain discipline. Confirm you have a recent backup for each domain vCenter and NSX before you touch it, and confirm the clusters have the DRS headroom to evacuate a host. Those two checks are dull and they are the ones that save the night.
A worked example: one domain, three clusters
Take a workload domain with its own vCenter, a dedicated NSX Manager, and three clusters of four hosts each. I planned the domain in VCF Operations, set the 9.1 target, and ran prechecks, which flagged one cluster still on a baseline. I converted that cluster to a vLCM image the day before, so it was not a surprise on the night.
On the window itself, NSX Manager ran about 80 minutes and vCenter about an hour on the sequential path. Then all three clusters upgraded ESX in parallel. Because each cluster kept a spare host for evacuation, the three ran at the same time and finished in roughly three hours rather than the nine a fully serial run would have taken, which is the shape plotted in Figure 4. The NSX Edge cluster finalized in about 35 minutes. Had I queued the clusters serially out of habit, I would have burned most of a second night for no reason. The lesson I keep from workload upgrades: the parallelism is only as good as your per-cluster DRS headroom, so confirm spare capacity before you fan out.
Common questions
Do I have to upgrade workload domains at all to be on 9.1?
No. Reaching 9.1 requires the fleet and management domain only. Workload domains are optional Day-N work. You should still upgrade them within a reasonable window for patching and interoperability, but there is no hard gate.
How many clusters can I upgrade at once?
VCF 9.1 supports up to 256 clusters in parallel, about four times the prior ceiling. In practice your real limit is DRS headroom, since each cluster must keep enough spare capacity to evacuate a host.
What happens if two domains share an NSX Manager?
You upgrade that NSX once and both domains are tied to the event. Plan the shared-NSX domains as a group and pick one window, because you cannot leave part of a shared NSX behind.
Why did a cluster refuse to start its ESX upgrade?
The usual reason is a vLCM baseline. Baselines are unsupported in 9.x, so the cluster has to move to an image first. Audit every workload cluster for baselines during planning.
My vCenter came up but will not license. Why?
The most common cause is the vCenter not being connected to the license server after the upgrade. Confirm the license server is reachable and the assignment applied, which is a documented condition on 9.1.
References
- Broadcom KB 440630, Upgrade Sequence and Related Issues for VCF and vSphere Foundation 9.1
- Broadcom TechDocs, Upgrading to VMware Cloud Foundation 9.1
- VCF Blog, How to Upgrade to VMware Cloud Foundation 9.1
Related on this site: Part 11, the management domain, VCF 9.0 vs 9.1, and the VCF 9.1 Upgrade Complete Guide.
About The Author
DrJha