, ,

How to Upgrade VCF 9.0 to VCF 9.1 (VCF 9.1 Day-2 Operations Step by Step Guide, Part 2)

Step by step how-to for upgrading a VMware Cloud Foundation 9.0.x fleet to 9.1, in order: VCF Operations, SDDC Manager, the VCF management services, then the management domain core components.

VCF 9.1 Day-2 Operations · Part 2 of 20
Routine operation
What this does. Moves an existing VMware Cloud Foundation 9.0.x fleet to version 9.1 by upgrading the management and fleet components in the required order.
Depends on. A healthy 9.0.x fleet, a cloud proxy in the first VCF Instance, current backups, and the 9.1 bundles staged in your depot.
Before you begin. Confirm the fleet is healthy and fully backed up before you start. Every appliance you touch should have a recent, verified backup and a snapshot. Work through the VCF 9.x pre-installation checklist so passwords, certificates, DNS, and NTP are all in a known good state first.
TL;DR
  • Prerequisites: fleet at 9.0.x (5.2.x also supported), a cloud proxy in the first VCF Instance, verified SFTP backups, and appliance snapshots.
  • Upgrade order is fixed: VCF Operations first, then SDDC Manager, then the VCF management services, then the management domain core components.
  • You upgrade VCF Operations with a PAK file from the administrator interface at https://node-fqdn/admin.
  • From 9.0.x, the fleet management appliance inventory transfers into VCF Operations, then that appliance is decommissioned automatically.
  • SDDC Manager, vCenter, NSX Manager, and ESX are driven from VCF Operations under Build and Lifecycle.
  • Workload domain upgrades are not mandatory and can run later as a Day-N task.

This part walks you through a fleet upgrade from VMware Cloud Foundation 9.0.x to 9.1. You will run prechecks, upgrade VCF Operations, upgrade SDDC Manager, deploy the new VCF management services, and then upgrade the management domain core components. Each step names the exact console to use, because the upgrade now spans two control planes: VCF Operations owns fleet and SDDC lifecycle, while SDDC Manager and vCenter carry out the domain work.

A short note on scope. Upgrading your management domain and the fleet-level components to 9.1 is mandatory. Upgrading workload domains is not, and you can schedule those later. If you are still on a 5.2.x release, the flow is similar but you plan and start the SDDC Manager upgrade from the SDDC Manager UI rather than from VCF Operations. Both paths are called out below. For the reverse direction, building a fresh 9.1 fleet, see the VCF 9.1 Deployment complete guide, and for how the operating model fits together, see Part 1 of this series.

Plan the work as a maintenance window rather than a single click. A fleet upgrade moves through several restarts, an inventory transfer, and rolling host remediation, so allow enough time for each stage and keep the fleet lightly loaded while it runs. Read this part end to end before you start, confirm your rollback position with current backups and snapshots, and then work through the steps in order.

Prerequisites

Confirm each item below before you plan the upgrade. Resolve any failed precheck rather than working around it.

ItemRequirementWhere to confirm
Source versionFleet at 9.0.x, or at 5.2.x or later before moving to 9.1VCF Operations, SDDC Manager
Cloud proxyA cloud proxy present in the first VCF Instance where VCF Operations runsVCF Operations
BackupsRecent, verified SFTP backups of SDDC Manager and managed componentsSDDC Manager, external SFTP
SnapshotsSnapshot of each VCF Operations node and of the SDDC Manager appliancevSphere Client
Root passwordFleet management appliance root password, needed for the inventory transferYour credential store
Bundles9.1 PAK and upgrade bundles staged in an online or offline depotBroadcom Support Portal, depot
InteroperabilityAvi Load Balancer, if present, at a version compatible with the targetProduct Interoperability Matrix
1. VCF OperationsFleet lifecycle2. SDDC ManagerDomain services3. VCF ManagementServices4. Management DomainvCenter, NSX, ESXUpgrade each stage fully before you start the next one

Step 1, plan the upgrade and run prechecks

Start in VCF Operations, set the target version, and clear every precheck finding before you download or apply anything.

  1. Log in to VCF Operations at https://ops-fqdn with an administrator account.
  2. From the top navigation bar, click Build, then in the left pane click Lifecycle.
  3. Expand VCF Instances and select the instance that hosts your management domain.
  4. Open the Updates view, review the Available updates, and note the 9.1 target.
  5. Click Precheck to validate the environment, and wait for the run to finish.
  6. Review each finding, fix any that fail, and rerun Precheck until the result is green.

Prechecks confirm certificate validity, password state, storage headroom, and connectivity. VCF 9.1 enforces stricter certificate validation for management services, so a certificate that was tolerated on 9.0.x can now block the run.

Step 2, snapshot and back up VCF Operations

Protect the VCF Operations cluster before the PAK upgrade, because the process restarts nodes and transfers inventory.

  1. In the vSphere Client, right click each VCF Operations node, select Snapshot, then click Take Snapshot.
  2. Give each snapshot a clear name that records the pre-upgrade version and date.
  3. In the VCF Operations administrator interface, confirm that recent content and configuration backups exist for the cluster.
  4. Record the fleet management appliance root password so you can enter it during the inventory transfer.

Backup configuration itself is out of scope here. Part 8 owns SDDC Manager and fleet backups and Part 9 owns vCenter and NSX Manager backups, so set those up first if they are not already running.

Step 3, upgrade VCF Operations with the PAK file

You upgrade VCF Operations 9.0.x to 9.1 with a PAK file in the administrator interface. This is the only supported method from 9.0.x.

  1. Download the VCF Operations 9.1 upgrade PAK file from the Broadcom Support Portal under My Downloads.
  2. Log in to the primary node administrator interface at https://primary-node-fqdn/admin with the admin user.
  3. In the left pane, click Software Update, then in the main pane click Install a Software Update.
  4. In the PAK File Upload dialog box, select the confirmation check box and click Next.
  5. Browse to the PAK file, click Upload, and when the upload finishes click Next.
  6. Accept the license agreement and the release information, then click Next.
  7. Click Install to begin. The interface restarts and logs you out.
  8. Reload the administrator interface, log in again, and click Software Update to track progress.
  9. When the Fleet Management v9.0 Node data Migration to VCF Operations dialog box appears, enter the fleet management appliance root password and click OK.
  10. Wait for all nodes and the cluster to return to an online status on the System Status page.

From 9.0.x, the inventory of the fleet management appliance transfers into VCF Operations, and that appliance is then decommissioned and powered off. You can delete it manually afterward. When the cluster upgrade finishes, the cloud proxy for the instance upgrades automatically and is briefly offline while it reboots. Cloud proxies do not collect data during this window, and collection resumes once the upgrade completes. Up to 10 cloud proxy instances upgrade at once; beyond that, the remainder run in sequence. If an automatic cloud proxy upgrade fails, you can upgrade that proxy manually with its command-line interface.

Step 4, upgrade SDDC Manager

With VCF Operations at 9.1, continue with SDDC Manager in the same VCF Instance. From 9.0.x you drive this from VCF Operations.

  1. Log in to VCF Operations and, from the top navigation bar, click Build.
  2. In the left pane, click Lifecycle.
  3. Expand VCF Instances and select the instance that hosts the SDDC Manager to upgrade.
  4. On the SDDC Manager Updates tab, click the Available updates entry.
  5. Click Update Now and monitor the run to completion.

If your source is 5.2.x rather than 9.0.x, upgrade SDDC Manager from its own UI instead: log in to SDDC Manager, open the management domain, select the Updates tab, click Plan Upgrade, choose the target version, click Confirm, then click Update Now or Schedule Update.

Step 5, deploy VCF management services and reconnect the depot

VCF 9.1 replaces the standalone fleet management appliance with two services, Fleet Lifecycle and SDDC Lifecycle, that run inside the VCF management services. Deploy those and the license server, then repoint the depot.

  1. In VCF Operations, follow the guided Deploy VCF Management Services workflow to deploy the management services and license server.
  2. Complete the license server registration so the licenses transfer from the VCF Operations instance to the license server.
  3. Log in to SDDC Manager and go to Administration, then Depot Settings.
  4. Reconfigure the online or offline depot to authenticate with an activation code from the VCF Business Services console, replacing the older download token.

Until the license server is deployed and registered, VCF Operations shows a licenses could not be synchronized message. That clears once the transfer completes. Offline depot setup has its own considerations, which Part 4 covers.

Step 6, upgrade the management domain core components

Now upgrade the remaining core components: NSX Manager, vCenter, ESX, and the NSX Edge cluster. You plan and drive this domain upgrade from VCF Operations.

  1. In VCF Operations, click Build, then Lifecycle, and select the management domain.
  2. Click Plan Upgrade, set the 9.1 target versions, and choose the Optimized or Sequential upgrade option.
  3. Run Precheck again for the domain and resolve any finding.
  4. Start the vCenter and NSX Manager upgrade. With Sequential, NSX Manager upgrades first, then vCenter; with Optimized, you prepare vCenter, upgrade NSX, then switch over to the new vCenter.
  5. Upgrade the ESX hosts cluster by cluster, letting each host enter maintenance mode and remediate against the vSphere Lifecycle Manager image.
  6. Upgrade the NSX Edge cluster and then finalize the NSX upgrade.
  7. Apply the VCF configuration upgrades to reconcile the running configuration with the 9.1 baseline.

Clusters that still use vSphere Lifecycle Manager baselines must transition to vSphere Lifecycle Manager images before ESX can move to 9.1, because baselines are not supported on 9.0 or later. Optional follow-on tasks such as raising the vSphere Distributed Switch version and the vSAN on-disk format can run afterward.

Workload domains follow the same pattern but sit outside this mandatory window. Once the management domain is at 9.1, upgrade each workload domain from VCF Operations under Lifecycle when a maintenance window allows, running its own precheck and plan first. Treat that as a scenario you schedule per domain, not as part of this run.

Management domain core orderNSX Manager and vCenterOptimized or sequentialESX clustersOne cluster at a timeNSX Edge clusterThen finalize NSXApply VCF configuration upgrades

Console map for the upgrade

Each phase runs from a specific console. Use this map to stay oriented as the upgrade moves across control planes.

PhaseConsoleEntry point
Prechecks and planningVCF OperationsBuild, then Lifecycle
VCF Operations upgradeVCF Operations admin interfaceSoftware Update
SDDC Manager upgradeVCF Operations (9.0.x source)Build, Lifecycle, SDDC Manager Updates
Management services and depotVCF Operations and SDDC ManagerDeploy workflow, then Depot Settings
Core component upgradeVCF Operations with vSphere ClientPlan Upgrade, then Precheck

Verification

Confirm each stage before you move on, and run a final set of checks once the domain upgrade completes.

  1. In the VCF Operations administrator interface, open Software Update and confirm the success message for the cluster.
  2. On the System Status page, verify that all nodes and the cluster show online.
  3. In VCF Operations under Lifecycle, confirm SDDC Manager, vCenter, NSX Manager, and ESX all report the 9.1 version.
  4. Confirm the fleet management appliance is powered off and that the license server shows registered with synchronized licenses.
  5. Review the upgrade history and log files for warnings before you clear snapshots.

You can also verify component versions directly in the vSphere Client and NSX Manager. Once every component reports 9.1 and status is healthy, delete the pre-upgrade snapshots, because keeping them degrades performance.

Common errors and fixes

SymptomLikely causeFix
Precheck fails on certificatesStricter 9.1 validation rejects a certificate without required FQDNs and IPsReplace the non-compliant certificate, then rerun the precheck
Cluster goes offline mid-installA node became unavailable during the PAK installOpen the admin interface, take the cluster online, and click Finish Installation
Inventory transfer prompt failsWrong fleet management appliance root passwordRe-enter the correct root password in the migration dialog and continue
Licenses will not synchronizeLicense server not yet deployed and registered after upgradeDeploy and register the license server, then the transfer clears the error
Depot downloads fail after upgradeDepot still set to an old download tokenReconfigure the depot with an activation code in Depot Settings

Common questions

Which component do you upgrade first
VCF Operations. Every 9.1 fleet upgrade starts there, then moves to SDDC Manager, the VCF management services, and finally the management domain core components.

Do you have to upgrade workload domains at the same time
No. Upgrading the management domain and the fleet-level components is mandatory, while workload domain upgrades can run later as a Day-N task.

What happens to the fleet management appliance from 9.0
Its inventory transfers into VCF Operations during the upgrade, then the appliance is decommissioned and powered off. Fleet Lifecycle and SDDC Lifecycle services take over inside the VCF management services.

Why does the SDDC Manager upgrade start in VCF Operations
From a 9.0.x source, VCF Operations owns fleet and SDDC lifecycle, so the SDDC Manager update is driven from Build and Lifecycle. From a 5.2.x source you plan it in the SDDC Manager UI instead.

When is it safe to delete the pre-upgrade snapshots
After every component reports 9.1 and the fleet is healthy. Holding snapshots longer than needed degrades performance.

References

VCF 9.1 Day-2 Operations · Part 2 of 20
« Previous: Part 1  |  Complete Guide

About The Author


Discover more from Journal of Intelligent Infrastructure

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure

Subscribe now to keep reading and get access to the full archive.

Continue reading