, ,

Upgrade VCF Operations to 9.1 with the PAK File and Software Update (VCF 9.1 Upgrade Series, Part 8)

Upgrade VCF Operations to 9.1 first with the PAK file and Software Update, then let the fleet management migration fold the old 9.0 appliance into your new instance.

VCF 9.1 Upgrade · Part 8 of 14

TL;DR · Key Takeaways

  • VCF Operations is the first appliance you upgrade to 9.1, because it holds lifecycle control in the 9.x model. Nothing else moves until it lands.
  • The PAK file is the only supported method from 8.x and 9.0.x sources. Download it from the Broadcom Support Portal, then apply it on the Software Update page of the primary Operations node.
  • From Aria Operations you must be on 8.18 first. From VCF Operations 9.0.x you go straight to 9.1 in a single PAK.
  • If you upgrade from 9.0.x, a fleet management migration runs mid-upgrade. Have the fleet management appliance root password ready or the migration stalls.
  • Expect the cluster to show Offline, then Going Online, then Online. That is normal. The cloud proxy upgrades itself afterward and reboots.
  • Two errors are expected and harmless until later steps: a license sync error and missing identity broker and log management certificates. Do not chase them.

I always give the same warning before a 9.1 upgrade: don’t open SDDC Manager first. The first appliance you touch is VCF Operations, and if that upgrade goes sideways the rest of the fleet won’t move at all. In the 9.x model Operations holds lifecycle control, so it has to be on 9.1 before anything downstream can follow it.

This part covers that first move in detail: the version you need to be on, the PAK file, the Software Update page, and the fleet management migration that runs quietly underneath the whole thing. Part 7 laid out the full sequence and why the order is fixed. If you skipped it, read Part 7 on the upgrade sequence first, then come back here for the hands-on Operations upgrade.

Why VCF Operations goes first

In 5.2, SDDC Manager drove almost everything. It held the inventory and pushed bundles down to vCenter, NSX and ESX. In 9.1 that job moved up a layer. VCF Operations is now the control point for lifecycle, and a new container cluster called VCF Management Services owns fleet lifecycle, SDDC lifecycle, the software depot and the license server. SDDC Manager is still there, but it takes its cues from above.

That reshuffle is the reason Operations upgrades first. It is also the reason Operations is now mandatory in VCF 9.x. If you never ran Aria Operations, you don’t get to skip it. You deploy VCF Operations as part of the upgrade instead. And the Operations upgrade is the step that quietly starts folding the old 9.0 fleet management appliance into Management Services, which you deploy a couple of steps later.

VCF Operationsto 9.1 (first)Depot andbinariesManagementServicesSDDC Managerto 9.1NSX, vCenter,ESXThe Operations upgrade opens every step to its right
VCF Operations is the control point in 9.x. The depot, Management Services, SDDC Manager and the domain core all wait behind it.

Which version you must be on before you start

The 9.1 PAK for Operations does not accept just any source version. From VCF Operations 9.0.x you apply one PAK and you are done. From older Aria Operations you have a stepping stone to clear first.

9.18.18 / 9.0.x8.xAria Ops 8.xreach 8.18VCF Ops 9.0.xVCF Ops 9.19.0.x path: one PAK8.x path: climb to 8.18 firstTwo ways up to 9.1
From VCF Operations 9.0.x the jump to 9.1 is a single PAK. From Aria Operations 8.x you climb to 8.18 first, which may take more than one interim upgrade.

The 8.18 stepping stone

If you are coming from an Aria Operations 8.x install, it must be on 8.18 before the 9.1 PAK will take. If you sit below 8.18 today, that means one or more interim upgrades before you even see the 9.1 bits. Plan for that as its own small project, because it is easy to assume you can jump straight to 9.1 and then lose an evening discovering you can’t.

The PAK file is the only method here

For the jump from 8.x or 9.0.x to 9.1, the PAK upgrade is the only supported path. For later maintenance updates the interface actually steers you away from PAK toward other methods, but a major version move like this one is PAK all the way. You download it from the Broadcom Support Portal: log in, open My Downloads, search for VMware Cloud Foundation, pick VMware Cloud Foundation 9, then 9.1.0.0, and under VCF Operations click View group to grab the upgrade PAK.

0123+Upgrade hops to reach VCF Operations 9.1VCF Ops 9.0.x1 PAKAria Ops 8.181 PAKAria Ops below 8.182+ upgrades
Both 9.0.x and 8.18 reach 9.1 in a single PAK. Anything below 8.18 needs at least one interim climb first.
ItemRequired for the 9.1 PAKWhy it matters
Operations source version9.0.x or Aria Operations 8.18.xPAK is the only supported method from these versions
Aria Operations below 8.18Climb to 8.18 firstThe 9.1 PAK will not apply to an older 8.x build
Cloud proxyVersion 8.18Needed for Operations, SDDC Manager and Management Services to talk
Fleet management root passwordIn hand before you startYou are prompted for it mid-upgrade during the migration
Pre-upgrade snapshotsOne fresh set, deleted afterMultiple lingering snapshots slow the cluster
Operations for networks certificatesFQDN and IP for every platform and collector nodeNon-compliant certs must be replaced before you begin

Get ready before you touch the PAK

Before you start: take fresh snapshots of the Operations nodes, back up any customized content, confirm your certificates carry FQDNs and IPs, and put the fleet management root password in your runbook. Every one of these is faster to do now than to fix while a change window is running.

Snapshots and customized content

Snapshot the Operations nodes before you apply the PAK. A PAK apply is not a light change, and a snapshot is the cleanest way back if a node fails to come online. Separately, back up your customized content: dashboards, alert definitions, custom groups, super metrics. The upgrade should preserve them, but should is not the same as verified, and rebuilding a year of dashboards from memory is nobody’s idea of fun.

Caution: delete those snapshots once the upgrade is validated. Leaving them on a multi-node Operations cluster costs you storage and drags performance, and old infrastructure snapshots have a way of causing confusion weeks later. This is a production system, so treat every snapshot as temporary.

Certificates, passwords and the cloud proxy

If you run VCF Operations for networks, its 9.0.x certificates have to carry FQDNs and IPs for all platform and collector nodes. A certificate with only a short name, or one missing a collector IP, will fail readiness before you can start. Replace any non-compliant certificate with a CA-signed one that lists every node.

You also need the fleet management appliance root password, because the migration prompts for it partway through. And you need a healthy cloud proxy. A cloud proxy in the first VCF instance is required for the integration between Operations, SDDC Manager and Management Services. If your environment has no cloud proxy, or the existing SDDC Manager cloud proxy is on 8.18 with an active Operations for Logs integration and log collection enabled through a one-time key, you deploy a new cloud proxy to finish the upgrade.

Running the software update

The apply itself happens on the primary Operations node. Log in to its administrator interface at the /admin path with the admin user, open Software Update in the left pane, and click Install a Software Update. From there the wizard walks a short, fixed path.

Download PAKfrom portalUpload PAK,accept EULAClick InstallInterface restarts,log back inFleet migration,enter root passwordNodes go OnlineCluster on 9.1,cloud proxy upgradesIf a node stays Offline while others are up:take the cluster Online manually, then click Finish Installation
The PAK path is short: download, upload, install, log back in, feed the migration a password, wait for Online. The dashed branch is the one manual recovery you might need.

In the wizard you tick the confirmation box, browse to the PAK, upload it, and click Next. You accept the license agreement and the release notes, then click Install. As soon as the install begins the administrator interface restarts and logs you out. That is expected. Reload the page, log back in, and open Software Update to watch progress.

Caution: this is a production appliance and the upgrade is not instantly reversible once the migration runs. Confirm your snapshots and backups exist, do it inside a change window, and know your rollback boundary before you click Install.

The fleet management migration

If you upgrade from 9.0.x, the interesting part happens after the install kicks off. A dialog appears asking you to migrate the fleet management appliance inventory into the upgraded Operations instance. You enter the fleet management appliance root password, click OK, and the inventory transfers. When it completes, that old appliance is decommissioned and powered off, and Operations holds the inventory itself until you deploy Management Services.

Once validations pass, the node upgrade runs and the interface restarts again. The System Status page will show the cluster Offline, then Going Online, then Online. Watch it rather than walk away, because a node can get stuck.

OnlineOfflineUploadInstallRestartMigrationGoing onlineOnlineCluster status through the PAK apply
Offline followed by Going Online is the normal shape of a PAK apply. Only a node that stays Offline while the others recover needs a hand.
What you seeWhat it meansWhat you do
Interface restarts, you are logged outThe install has begunReload the page and log back in
Fleet Management migration dialogInventory is moving off the 9.0 applianceEnter the fleet management root password, click OK
Cluster status OfflineNodes are applying the PAKWait, do not intervene yet
Going OnlineInstallation is progressingKeep watching, no action
One node stuck Offline, others upThe PAK apply failed on that nodeTake the cluster Online manually, click Finish Installation
All nodes and cluster OnlineOperations is on 9.1Verify on Software Update, then move to the depot step
Seen this go wrong: two failures show up again and again on this step. The first is a certificate one. An Operations for networks node had a certificate with the short hostname only and no collector IP, and readiness refused to let the upgrade start. Replacing it with a CA-signed certificate that listed the FQDN and IP for every platform and collector node cleared it. The second is simpler and more painful. The migration dialog popped up, and the person running the change did not have the fleet management appliance root password. The upgrade just sat there waiting while the change clock ran down. Put that password in the runbook before the window opens.

After the cluster comes back Online, the cloud proxy for that Operations instance upgrades on its own. It reboots during that process and goes briefly offline, which is expected. Let it settle before you judge the health of the instance.

How you know it worked

Signs it worked: every Operations node and the cluster show Online on the System Status page, Software Update shows a success confirmation for the cluster, the cloud proxy is back up after its reboot, and the old fleet management appliance is powered off. When all four line up, Operations is genuinely on 9.1.

Two errors will appear right after this upgrade and both are expected. You will see a message that licenses could not be synchronized with vCenter instances. That clears once you deploy and register the license server, which rides in with Management Services. You will also notice that Operations does not show certificates for the identity broker and log management components yet. Those appear after you deploy Management Services too. Neither is a real fault, so don’t spend the change window troubleshooting them.

What I’d do: watch this step from start to finish instead of multitasking, because the migration dialog and a stuck node both want a human at the keyboard. Delete the pre-upgrade snapshots once the cluster reports success, and do not open the SDDC Manager upgrade until Operations shows a clean Online cluster. Rushing ahead on a half-finished Operations upgrade is how a tidy maintenance window turns into a long night.

A realistic first run

Here is how a clean 9.0.2 to 9.1 Operations upgrade tends to go when the prep is done. The day before, you pull the VCF Operations 9.1 PAK from the Broadcom Support Portal so nothing is downloading during the window. You snapshot the Operations nodes, export a copy of the customized dashboards and alerts, and confirm the cloud proxy is on 8.18. You also write the fleet management appliance root password into the runbook, because you know the migration dialog is coming.

In the window, you log in to the primary node at the /admin path, open Software Update, and click Install a Software Update. You upload the PAK, accept the license and release notes, and click Install. The interface drops you almost immediately, which still feels wrong every time even though it is expected. You reload, log back in, and open Software Update to follow along.

A few minutes in, the fleet management migration dialog appears. You paste the root password, click OK, and the inventory starts moving. The System Status page goes Offline, sits there long enough that a first-timer would panic, then flips to Going Online and finally Online across every node. The cloud proxy reboots on its own and drops offline briefly. The license sync error shows up in the banner. You leave it alone, because you know it clears when the license server arrives with Management Services. Software Update reports success, the old fleet management appliance is powered off, and you delete the snapshots. Only then do you look at the SDDC Manager step. That discipline is the whole game on this part.

Common questions

Do I have to upgrade Operations if I never ran Aria Operations?
Yes. Operations is mandatory in VCF 9.x. If it is not deployed, you deploy it as part of the upgrade rather than skipping it. The rest of the sequence assumes it is present and current.

Can I use the PAK method for a later 9.1 patch?
No. PAK is for the jump from 8.x or 9.0.x to 9.1. For maintenance updates the interface actively recommends other methods, and once Management Services is deployed later patches flow through it. Reserve the PAK for this one major move.

My cluster went Offline in the middle. Did it fail?
Probably not. Offline followed by Going Online is the normal shape of a PAK apply. The only case that needs a hand is one node stuck Offline while the others recover. Then you take the cluster Online manually and click Finish Installation.

What happens to the old fleet management appliance?
After the inventory migrates, it is decommissioned and powered off. You can delete it once you are satisfied the upgrade is healthy. Its job moves into VCF Management Services, which you deploy in the next part of this series.

Where do I actually get the PAK file?
The Broadcom Support Portal. Open My Downloads, search for VMware Cloud Foundation, choose VMware Cloud Foundation 9, then 9.1.0.0, and under VCF Operations click View group to download the upgrade PAK. Grab it before the change window so you are not waiting on a download at 2 AM.

VCF 9.1 Upgrade · Part 8 of 14
« Previous: Part 7  |  Complete Guide  |  Next: Part 9

References

About The Author


Discover more from Journal of Intelligent Infrastructure – By Dr Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure - By Dr Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading