VCF 9.1 Day-2 Operations · Part 4 of 20
Task type: OPTIONAL. Run this only when your VCF 9.1 fleet cannot reach the Broadcom depot over the internet. An offline depot gives lifecycle operations a local source for install, upgrade, patch and ESX binaries. Depends on a healthy fleet, a reachable internal web server, and a separate machine with internet access to run the VCF Download Tool.
Before you begin. Confirm the fleet is healthy and backed up before you change any depot setting. Work through the VCF 9.x pre-installation checklist and verify recent backups of VCF Operations and SDDC Manager first.
TL;DR
- Prerequisites: a Linux or Windows web server, a dedicated disk of 1 TB or more, a static IP, and a machine with internet access for the VCF Download Tool.
- Offline depot is one of three software depot modes in VCF 9.1: Connected, Offline Depot, and Disconnected.
- Build the web server with a document root such as /var/www/html and serve it over HTTP, or HTTPS with TLS 1.2 and TLS 1.3.
- Install the VCF Download Tool, generate a software depot ID, and register it in the VCF Business Services console for an activation code.
- Download install, upgrade, patch and ESX binaries with the tool, then copy them into the web server document root.
- In VCF Operations, open Build then Software Depot, select Offline Depot, enter the URL, and finish.
An offline depot lets a VCF 9.1 fleet run lifecycle operations without direct internet access. You host the binaries on an internal web server, point the software depot at that server, and VCF Operations proxies binary requests to it. This part builds that web server, downloads binaries with the VCF Download Tool, and connects the software depot in offline mode. Applying those binaries during an upgrade is owned by Part 2, and patching is owned by Part 3, so this part stops once the depot is connected and binaries are visible.
Plan for two machines. One is the internal web server that the fleet reaches, and the second is a machine with internet access that runs the VCF Download Tool. Both roles can live on one host when that host has internet access and is reachable from the fleet, but most sites keep them apart. For the operating model behind this series, start with Part 1.
Prerequisites
| Component | Requirement |
|---|---|
| Web server OS | Linux or Windows VM. Photon OS 5 works well for a small footprint. |
| Storage | A dedicated disk, 1 TB or more, mounted at the document root. |
| Web server | Apache HTTP Server or NGINX. HTTPS needs TLS 1.2 and TLS 1.3 support. |
| Network | Static IP on the web server. DNS records are recommended. |
| Download machine | A separate machine with internet access to run the VCF Download Tool. |
| Entitlement | A VCF Business Services console account able to register a software depot. |
| Fleet access | A VCF Operations user assigned the Administrator role. |
Size the depot disk for growth. A full VCF 9.1 install and upgrade set with ESX images can consume several hundred gigabytes, and each new patch release adds more, so a 1 TB disk gives room for a few releases before you prune. Keep the depot on its own disk so a fill event does not stall the operating system. Plan the network so the fleet reaches the web server on the serving port and the download machine reaches Broadcom, since those two paths are separate and rarely overlap.
Decide early whether to serve HTTP or HTTPS. Plain HTTP with no authentication is valid in VCF 9.1 and removes the certificate work in Step 6, which many internal networks accept because the depot holds signed vendor binaries rather than secrets. HTTPS suits sites with a policy that forbids clear traffic, and it adds a one time certificate import into SDDC Manager. Either choice leads to the same VCF Operations configuration in Step 7.
Step 1, Review the software depot modes and confirm what to build
A software depot instance runs in one of three modes. Confirm the current mode in VCF Operations, then decide whether Offline Depot fits your network.
- Log in to VCF Operations at your VCF Operations FQDN with a user assigned the Administrator role.
- Select Build, then select Software Depot.
- Read the current connection mode shown on the Software Depot page.
- Compare the modes below and confirm Offline Depot for a fleet that reaches an internal web server but not the internet.
| Mode | When to use |
|---|---|
| Connected | Fleet reaches Broadcom directly or through a proxy and downloads binaries itself. |
| Offline Depot | Fleet has no internet, but a local web server holds the binaries and the depot proxies requests to it. |
| Disconnected | No depot server exists, so you upload binaries into the software depot by hand. |
Step 2, Build the offline depot web server
Stand up an internal web server with a dedicated disk. Steps below use a Photon OS 5 VM with Apache HTTP Server and the document root /var/www/html.
- Deploy the Photon OS 5 OVA on a VM and do not power it on yet.
- Add a hard drive of 1 TB or more for depot content.
- Power on, then set a static IP in a file under /etc/systemd/network/ and restart the network service.
- Set the hostname with hostnamectl to your depot FQDN, for example depot.rainpole.io.
- Install the web server and utilities, then reboot.
tdnf install httpd tar jq --assumeyes
tdnf update --assumeyes
reboot- Format the added disk and mount it at /var/www/html, then add it to /etc/fstab.
- For HTTPS, create a private key and a CSR with Subject Alternative Names for the depot FQDN and IP, then sign it with your certificate authority or VMCA.
- Move the signed key and certificate into /etc/httpd/conf/ and reference them in httpd-ssl.conf.
- Open the serving port in the firewall, port 443 for HTTPS or port 80 for HTTP, then restart iptables.
- Enable and start the server with systemctl, then verify from a browser at your depot URL.
httpd -t
systemctl enable httpd
systemctl start httpdVCF 9.1 supports HTTP without authentication, HTTPS without authentication, and HTTPS with basic authentication. Plain HTTP removes the certificate trust step later, which suits many internal networks. If you choose basic authentication, create a password file with htpasswd and note the user name and password, because you enter the same pair in VCF Operations during Step 7. Test the server from a browser before you load any binaries, so a networking or firewall fault surfaces while the tree is still empty and easy to reason about.
Step 3, Install the VCF Download Tool
Download the tool on your internet connected machine. It ships as a tar archive that contains both the VCF Download Tool and the bundle transfer utility. The same offline depot can also feed the VCF Installer during a build, covered in the installer appliance deployment part.
- Log in to the Broadcom Support Portal and browse to My Downloads, then VMware Cloud Foundation.
- Click the 9.1 version to which you are moving.
- Click Drivers & Tools.
- Click the download icon for the VCF Download Tool.
- Copy the archive to the download machine and extract it.
tar -xzf vcf-download-tool-9.1.0.0.<build>.tar.gz- Change to vcf-download-tool-<version.build>/bin/ and confirm you have execute permission on the folders.
Inside the bin folder you find lcm-bundle-transfer-util and vcf-download-tool.
Step 4, Register the depot and get an activation code
VCF 9.1 uses an activation code to authorize downloads. Generate a software depot ID with the tool, then register it in the VCF Business Services console. A download token still works for install and upgrade binaries, but ESX downloads require an activation code.
- Generate a software depot ID on the download machine.
vcf-download-tool configuration generate --software-depot-id- Log in to the VCF Business Services console at vcf.broadcom.com.
- Next to the tenant name, click the arrow to expand the section.
- Select the tenant name and site ID that map to your VCF or vSphere Foundation entitlement.
- Navigate to Software Depot Registrations, then click New Registration.
- Enter the software depot ID to obtain an activation code, then save it to a file such as /path/to/activation-code.txt.
Step 5, Download binaries and ESX data
List the binaries first, then download install, upgrade and ESX content into a single depot store folder. Use –sku VCF for Cloud Foundation, or replace it with –sku VVF for vSphere Foundation.
- List the install binaries for your version.
./vcf-download-tool binaries list --sku VCF --vcf-version 9.1.0 --depot-download-activation-code-file /path/to/activation-code.txt --type INSTALL --automated-install- Download the install binaries into your depot store.
./vcf-download-tool binaries download --sku VCF --vcf-version 9.1.0 --depot-download-activation-code-file /path/to/activation-code.txt --type INSTALL --automated-install --depot-store /var/www/html- Download the upgrade binaries into the same folder.
./vcf-download-tool binaries download --sku VCF --vcf-version 9.1.0 --depot-download-activation-code-file /path/to/activation-code.txt --type UPGRADE --depot-store /var/www/html- Download the ESX binaries and metadata, then watch progress in ./logs/esximage.log.
./vcf-download-tool esx download --depot-download-activation-code-file /path/to/activation-code.txt --depot-store /var/www/html- Reduce ESX size by disabling versions you do not need with esx configuration.
./vcf-download-tool esx configuration -D embeddedEsx-6.7-INTLWhen the download machine reaches the internet through a proxy, add –proxy-server with an FQDN and port, for example –proxy-server proxy.example.com:3128. VCF does not support proxies that use Kerberos or NTLMv2 authentication.
Step 6, Move binaries and trust the certificate
If the tool ran on a separate machine, copy the depot store to the web server document root. For an HTTPS depot, import the web server certificate into the SDDC Manager trust store so the depot connection validates.
- Copy the depot store content into the web server document root, then set ownership and read permission for the web server user.
- SSH to SDDC Manager and log in as the vcf user, then change to root with su.
- Download the depot certificate to a file.
openssl s_client -connect depot.rainpole.io:443 < /dev/null | sed -ne '/BEGIN CERTIFICATE/,/END CERTIFICATE/p' > /tmp/server.crt- Import the certificate into the cacerts store with keytool.
keytool -import -trustcacerts -file /tmp/server.crt -keystore /etc/alternatives/jre/lib/security/cacerts -alias offline_depot_server -storepass changeit -noprompt- Restart the lifecycle manager service so the new trust takes effect.
systemctl restart lcmSkip this whole step when the web server serves plain HTTP, since there is no certificate to trust.
Step 7, Connect the software depot in offline mode
Point the software depot at your web server from the VCF Operations UI. After a fleet upgrade from 9.0.x, this configuration also transfers automatically from the VCF Installer, so a fresh setup is only needed on older fleets or a rebuild.
- Log in to VCF Operations with a user assigned the Administrator role.
- Select Build, then select Software Depot.
- Click Edit.
- Select Offline Depot, then click Next.
- Enter the URL for the offline depot web server.
- If the depot needs credentials, toggle Authenticate on and enter the user name and password.
- Click Finish.
- To confirm content, select Build, then Lifecycle, select a VCF Instance, then open Binary Management and Upgrade Binaries.
Once the depot connects, the fleet reads binaries from your web server for every lifecycle action, and no component reaches the internet. VCF Operations owns this depot at the fleet level, while SDDC Manager continues to run domain operations such as adding hosts and clusters. If you plan an immediate upgrade, move on to Part 2 next, which consumes the binaries you just staged.
Verification
Confirm the depot is connected and the binaries are visible before you hand the fleet to an upgrade or patch run.
- On the Software Depot page, the mode reads Offline Depot and the connection status shows success.
- Under Binary Management and Upgrade Binaries, the versions you downloaded appear for the selected VCF Instance.
- From a browser, your depot URL lists a PROD folder tree with COMP and metadata content.
- On the download machine, esximage.log records finished ESX downloads with no errors.
Treat the depot as an asset you refresh over time, not a one time task. When a new express patch or async patch ships, rerun the VCF Download Tool against the same depot store to pull the added binaries, then let the software depot pick them up. Applying those patches to the fleet stays with Part 3, so this part ends once new content is visible under Binary Management. Keep an eye on free space on the depot disk and remove versions you no longer run from the web server tree when a release ages out.
Common errors and fixes
| Symptom | Cause | Fix |
|---|---|---|
| Invalid credentials when saving depot settings | Web server certificate not trusted by SDDC Manager | Import the certificate into the cacerts store and restart lcm, as in Step 6. |
| Activation code rejected | Wrong tenant or site ID chosen in the Business Services console | Regenerate the software depot ID and register under the correct tenant and site ID. |
| Binaries not listed after connecting | Files outside the document root or wrong permissions | Copy binaries into the document root and grant read access to the web server user. |
| ESX download very large or slow | All ESX platforms enabled | Disable unused platforms with esx configuration -D or edit esxUserConfig.json. |
| Proxy authentication fails | Proxy uses Kerberos or NTLMv2 | Use a proxy without Kerberos or NTLMv2, which VCF does not support. |
Common questions
Support for plain HTTP. VCF 9.1 accepts an HTTP offline depot with no authentication, which removes the certificate trust step. Many internal networks use this to keep setup simple.
One depot for several versions. Point the tool at the same depot store folder for each version, and it keeps them together under one tree. A single web server can serve 9.1 alongside earlier releases.
Difference from disconnected mode. Offline Depot proxies requests to your web server and never stores binaries locally, while Disconnected mode has you upload binaries into the software depot itself.
Reducing ESX download size. List the ESX platforms with esx configuration and disable the ones you do not run, or set disabledPlatforms in esxUserConfig.json before you download.
References
About The Author
