, ,

The VCF 9.1 Upgrade Sequence End to End and Why the Order Matters (VCF 9.1 Upgrade Series, Part 7)

The full VCF 9.1 upgrade sequence in the order the platform enforces: VCF Operations first, then the depot, VCF Management Services, SDDC Manager, and the NSX, vCenter and ESX domain core, with rough timings and why each step sits where it does.

VCF 9.1 Upgrade · Part 7 of 14

TL;DR · Key Takeaways

  • VCF Operations upgrades first. In 9.x it is the control point for lifecycle, so nothing else can move until it is on 9.1.
  • After Operations you upgrade SDDC Manager to 9.1, then deploy VCF Management Services and the License Server. Carve a fresh CIDR block on the management VLAN: a /28 gives 14 usable IPs (12 is the minimum), a /27 is better for growth. You cannot reuse your 9.0.2 IPs.
  • Dependency components gate the domain plan: upgrade Avi Load Balancer to a 9.1-compatible build and validate VMware Live Recovery / SRM and vSphere Replication before the core plan builds.
  • With SDDC Manager and Management Services on 9.1, you plan the domain upgrade and run prechecks.
  • The domain core runs in a fixed order: NSX, then vCenter, then ESX, and NSX finishes with the edge cluster. NSX is two steps because the NSX host VIBs now ship inside the ESX image.
  • vSphere Lifecycle Manager baselines are gone in 9.x. Every cluster moves to a vLCM image before ESX can upgrade.

I have upgraded enough VCF fleets to know that the biggest cause of a failed 9.1 upgrade is not a bad bundle or a flaky host. It is doing the right steps in the wrong order. VCF 9.1 moved where lifecycle control lives, so the sequence you used for 5.2, or even for 9.0 at launch, is not the sequence you follow now.

This part maps the whole upgrade end to end, in the order the platform actually enforces, and explains why each step sits where it does. Parts 8 through 14 walk each component in real detail. Keep this one open on a second monitor as the route map while the rest of the series fills in the turns. If you have not read Part 6 on backups and rollback yet, do that first, because a couple of steps below assume you already have those safety nets in place.

Why the order matters

In 5.2, SDDC Manager drove almost everything. You logged into SDDC Manager, it held the inventory, and it orchestrated bundles down to vCenter, NSX and ESX. In 9.1 that job moved up a layer. VCF Operations is now the control point for lifecycle, and a new container cluster called VCF Management Services owns fleet lifecycle, SDDC lifecycle, the software depot and the license server. SDDC Manager is still there, but it takes its cues from above.

That change is the whole reason the sequence looks the way it does. If Operations is not current, it cannot present the 9.1 paths. If Management Services is not deployed, SDDC Manager will not let you proceed with the domain, because the new service layer has to own the inventory before it can orchestrate the rest of the stack. Get this wrong and you do not get a polite warning halfway through. You get a stalled upgrade with half the fleet on new metadata and half on old.

1 · Prerequisites and prechecks2 · VCF Operations 9.1 (upgrades first)3 · Online depot, then download 9.1 binaries4 · Deploy VCF Management Services5 · Avi Load Balancer, SRM / Live Recovery6 · SDDC Manager 9.17 · Plan domain upgrade, run prechecks8 · NSX Manager (management stack)9 · vCenter10 · ESX hosts (assign vLCM image)11 · NSX edge cluster, finalize domain
Figure 1. The 9.1 upgrade sequence from top to bottom. The order is enforced, not a suggestion.

The sequence end to end

Here is the same flow as a reference table. The right-hand column is the part I want you to actually read, because it is the reason each row cannot move.

OrderComponentWhere you drive itWhy it sits here
1Prerequisites and prechecksSDDC Manager and VCF OperationsCatches blockers before anything changes on disk
2VCF Operations 9.1Operations PAK, Software UpdateIt becomes the lifecycle control point, so it goes first
3Online depot and binariesOperations, Build > Depot and Binary ManagementNo active depot means no bundles for the rest of the run
4SDDC Manager 9.1Operations, SDDC Manager UpdatesUpgrades right after Operations and before you deploy Management Services
5VCF Management ServicesOperations, Build > LifecycleNew cluster owns depot, licensing and fleet lifecycle; deploys once Operations and SDDC Manager are on 9.1
6Avi Load Balancer, SRM / Live RecoveryTheir own consolesDependency health has to be good before the core plan builds
7Plan domain upgrade, prechecksOperations, Upgrades tabSets target versions and the vCenter and NSX scenario
8NSX Manager stackOperations lifecycleFirst of two NSX steps; the VIBs ship in ESX now
9vCenterOperations lifecycleNeeds a temporary IP; runs after the NSX manager
10ESX hostsOperations, Image ManagementNeeds a vLCM image; baselines are gone in 9.x
11NSX edge cluster, finalizeOperations lifecycleCloses out NSX and marks the domain 9.1

Get the control plane current first

Steps 2 through 4 are one logical block: bring the management brain to 9.1 before you touch the body. You upload the VCF Operations 9.1 PAK file through the Operations administrator interface under Software Update. From 9.0.x, the upgrade migrates the old fleet management data into the new Management Services cluster, then decommissions and powers off the standalone fleet appliance. Expect the Operations interface to restart and log you out during this; that is normal, log back in and keep watching the cluster status until every node and the cluster show online.

Once Operations is on 9.1, you configure the online depot under Build > Depot Settings with your activation code, confirm it is active, then download the 9.1 bundles from Binary Management. Do the download before your maintenance window. Watching a multi-gigabyte bundle crawl while people stare at the change clock is a waste of everyone. Also confirm you have a Cloud Proxy in the first VCF instance where Operations lives, because it is required for the integration between Operations, SDDC Manager and Management Services.

VCF Operationslifecycle control pointVCF Management Servicesfleet lifecycle · SDDC lifecyclesoftware depot · license serverSDDC ManagerNSXvCenterESX hostsThe top three must be current before the domain core can move
Figure 2. The 9.1 control hierarchy. Operations and Management Services sit above SDDC Manager, which is why they upgrade first.

Deal with dependencies before SDDC Manager

Step 5 is the one people forget, because these components sit slightly outside the normal SDDC Manager path. If Avi Load Balancer (NSX Advanced Load Balancer) is deployed, upgrade it before the SDDC Manager sequence: confirm the controller version, validate against the 9.1 compatibility list, check controller and service engine health, then upgrade. If VMware Live Recovery or Site Recovery Manager and vSphere Replication are present, validate the supported path and confirm pairings, protection groups and replications are healthy first. A sick load balancer or a broken replication pairing has a habit of turning a clean platform upgrade into an incident bridge.

Then SDDC Manager and the domain plan

With Operations current, Management Services running and dependencies healthy, you upgrade SDDC Manager to 9.1, driven from the SDDC Manager Updates tab in Operations. After that you use the Upgrades tab to plan the management domain upgrade: pick the target version, choose custom component targets if needed, and select the upgrade scenario for vCenter and NSX Manager. This is also where you decide between an optimized or a sequential rollout, which comes down to your window size and how much parallel change you are comfortable running.

The step that reshuffles everything: VCF Management Services

If one step deserves extra attention, it is deploying VCF Management Services. This new container cluster replaces two things from 9.0: the standalone Fleet Management Appliance and the VMware Identity Broker. It owns lifecycle, the depot, licensing and fleet operations going forward, and SDDC Manager will not orchestrate the domain until it is up and healthy.

The catch is the network. The deploy wizard asks for an IP pool in CIDR format, not a range, and the cluster needs at least 12 free IP addresses on your management VLAN. In practice that means you must supply a /28, which yields 14 usable addresses, and a /27 (30 usable) is worth it if you have the room to grow. You cannot reuse your existing 9.0.2 IPs, because your old components are still online orchestrating the upgrade while the new cluster comes up. Reuse the same addresses and you get network collisions right in the middle of the run. Prepare forward and reverse DNS records for the new service FQDNs before you start, and run the Management Services precheck. If it flags DNS or reachability, fix it before you click Deploy.

VCF 9.0Fleet Management ApplianceVMware Identity Brokertwo separate components,existing IPsVCF 9.1VCF Management Servicesunified clusterlifecycle, depot, licensingneeds a fresh /28 (14 usable)or /27, plus new DNS recordsYou cannot reuse the old IPs: both sets run at once during the deploy
Figure 3. What Management Services replaces, and why it needs its own subnet.
Seen this go wrong: On one 9.0.2 fleet, the team planned to reuse the fleet appliance addresses for Management Services to save asking the network team for a subnet. The wizard rejected the IP range outright because it wants a CIDR pool, and when they forced addresses that overlapped live services, the deploy collided with the still-running 9.0.2 components mid-run. The fix was boring and should have been done up front: carve a clean /28, add forward and reverse DNS for the new FQDNs, validate reachability, then deploy. It cost them a wasted evening.

Which components upgrade first in the domain?

Inside the management domain the order is NSX, then vCenter, then ESX, then the NSX edge cluster to finalize. NSX is split into two steps for a concrete reason: in 9.x the NSX host VIBs are built into the ESX image by default, so the NSX manager stack upgrades first, and the edge cluster upgrade closes out after vCenter and ESX are done. Trying to finish NSX before ESX would mean upgrading the edge against hosts that are not yet carrying the matching bits.

vCenter comes after the NSX manager and needs a free temporary IP on the same subnet, because the upgrade flow stands up the new appliance on a temporary network before switching over. From 9.0.x you can choose an in-place upgrade or a reduced downtime upgrade; coming from 5.2 the reduced downtime path is required between major versions. ESX is last, and this is where a 9.x change bites people who skipped the reading: vSphere Lifecycle Manager baselines are no longer supported. Every cluster has to be transitioned to a vLCM image, so you import the ESX image into Image Management, assign it to the clusters, then run the host upgrades in a rolling fashion through maintenance mode.

NSX managerstep 1 of 2vCentertemp IP neededESX hostsassign vLCM imageNSX edgestep 2, finalizeNSX brackets the domain: manager first, edge last, because host VIBs live in the ESX image
Figure 4. The domain core order, and why NSX runs at both ends.

A worked example with rough timings

Take a modest management domain: two clusters, eight hosts, one vCenter, an NSX manager cluster and one edge cluster, coming from 9.0.2. Here is roughly how I would spread it. These numbers are estimates from lab and field runs and they move with your hardware, link speed and domain size, so treat them as planning aids, not promises.

StepRough durationNotes
VCF Operations 9.1 and fleet migration2 to 4 hoursCluster restarts, you get logged out mid-run
Depot config and binary download1 to 3 hoursDo it before the window; size depends on the link
Deploy VCF Management Services45 to 90 minutesNeeds the /28 or /27 and DNS ready first
SDDC Manager 9.145 to 90 minutesPrecheck, then update
NSX manager stack1 to 2 hoursEdge cluster finishes later
vCenter1 to 2 hoursReduced downtime or in-place from 9.0.x
ESX, per host20 to 40 minutes eachRolling; maintenance mode plus reboot
NSX edge and finalize30 to 60 minutesCloses the domain to 9.1

The lesson I take from that table: the control-plane work at the top (Operations, depot, binaries, Management Services) is several hours on its own and mostly independent of your production workloads. Eight hosts at half an hour each is roughly four hours of ESX rolling upgrades by itself. Add it up and a single all-night window for the whole thing is tight and stressful. I would rather split it.

What I would do: Upgrade VCF Operations and deploy Management Services in a planned session the week before the domain window, and download all binaries then too. On domain night, run only NSX, vCenter and ESX plus the edge finalize. Splitting it means the risky, production-affecting work happens against a control plane you already proved healthy, and if the pre-work session goes sideways you have not touched a single workload host yet.
Signs it worked: the SDDC Manager management domain shows fully upgraded to VCF 9.1, the Operations cluster and every node read online, the old fleet appliance is powered off, and vCenter and ESX both report 9.1 in two places, the lifecycle view and the component UI. If any one of those disagrees, stop and reconcile before you call it done.

Safety before you touch anything

Caution: the vCenter upgrade uses a temporary network and is production-affecting, and the ESX rolling upgrade puts hosts into maintenance mode and reboots them. Before you start: confirm a recent SDDC Manager backup on external SFTP, take a supported vCenter snapshot, confirm backups exist for the other managed components, and know your rollback boundaries for each step. Run every precheck and fix reds before continuing. Remove snapshots only after post-upgrade validation passes, and never treat any of these steps as risk-free. This is the same discipline covered in Part 6.

Common questions

Can I upgrade SDDC Manager before VCF Operations?
No. Operations is the control point in 9.x and has to be on 9.1 first. The SDDC Manager upgrade is driven from Operations, and it will not offer you the domain path until Management Services is deployed. Doing it out of order is how you stall.

Do I really need a new subnet for Management Services?
Coming from 9.0.2, yes. The wizard wants a CIDR pool with at least 12 free addresses, so a /28 minimum, and your existing components stay online during the deploy. Reusing their addresses causes collisions. Prepare the CIDR block and DNS before the change.

Why is NSX two steps?
The NSX host VIBs now ship inside the ESX image, so the manager stack upgrades first and the edge cluster upgrade finalizes after vCenter and ESX. Splitting it keeps the edge in step with the hosts.

Can I skip prechecks if I am short on time?
No. A red precheck is the platform predicting the failure for you. Fixing it now is far cheaper than untangling a mid-upgrade stall. Review the warnings too, since some of them are outages waiting for their moment.

What about VCF Automation and the other management components?
They come after the core domain is on 9.1, usually as a separate workstream because their integrations take time to validate. Get the core stack current first, then import and upgrade Automation, Operations for Networks, Log Management and Identity Broker. The VCF 9 Operations series and the general VCF 9 series cover those components in depth.

VCF 9.1 Upgrade · Part 7 of 14
« Previous: Part 6  |  Complete Guide  |  Next: Part 8

References

About The Author


Discover more from Journal of Intelligent Infrastructure – By Dr Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure - By Dr Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading