Dr. Pranay Jha

VMware • Cloud • AI • Enterprise Architecture

FORMERLY
VMware Insight & Cloud Pathshala
What began over a decade ago as a passion for sharing knowledge has evolved into a unified platform for Enterprise AI, VMware, Cloud Architecture, Research, and Modern Infrastructure.
, ,

Application and Infrastructure Delivery in VCF Automation: Life After Pipelines (VCF Automation 9 Series, Part 22)

Pipelines, the feature once called Code Stream, is removed in VCF Automation 9. Here is the delivery model that replaces it: external CI/CD driving the vmware/vcfa Terraform provider and the API-first deployment endpoints.

VCF Automation 9 Series · Part 22 of 41
TL;DR · Key Takeaways
  • VCF Automation Pipelines (the feature once shipped as vRealize Code Stream, then Aria Automation Pipelines) is removed in VCF Automation 9. There is no in-product pipeline engine and nothing shipped in its place.
  • The last release that carries Pipelines is VCF 5.2.1 / VCF Automation 8.18.1. A fresh 9 install does not have it, and upgrading Aria Automation 8.x to 9 drops it.
  • Delivery moves out of the platform and into your existing CI runner. VCF Automation 9 is API-first, so GitHub Actions, GitLab CI or Jenkins becomes the orchestrator and the platform becomes the target.
  • Split the work: the vmware/vcfa Terraform provider for the platform config, the deployment and catalog API for app delivery, and Orchestrator for in-platform glue.
  • Broadcom points at Continuous Delivery Director (CDD) as a portfolio replacement. It is a separate licensed product. For most teams, external CI against the API is the cheaper, cleaner answer.
Who this is for:
Platform and cloud admins, automation engineers and release owners who ran pipelines inside vRA / Aria Automation and now have to rebuild that delivery flow on VCF Automation 9.
Prerequisites:
A working VCF Automation 9.x instance, a tenant project you can deploy into, an existing CI system (GitHub Actions, GitLab CI or Jenkins), and a service-account API token. Comfort with Terraform and REST helps.

You finish the upgrade to VCF Automation 9, log in as a tenant admin, and go looking for the Pipelines tab. It is not there. No Code Stream, no executions, no triggers. This is not a missing role or a permissions problem. Broadcom removed the feature. If you ran CI/CD inside vRealize Code Stream or Aria Automation Pipelines, that engine did not make the trip to 9, and it does not come back when you upgrade from 8.x either. So the real question for Part 22 is not how to build a pipeline in the product. It is where delivery lives now that the product no longer does it.

What actually happened to Pipelines

The feature has a long name history and a short ending. It started as vRealize Code Stream, was rebranded VMware Aria Automation Pipelines, then briefly VCF Automation Pipelines. Broadcom announced the deprecation in the VCF 5.2.1 / VCF Automation 8.18.1 release notes and followed up with a formal advisory. VCF 5.2.1 / VCF Automation 8.18.1 is the last version that ships it. Three facts matter for planning: it is not installed with VCF Automation 9, it is not carried forward when you upgrade Aria Automation 8.x to 9, and if you stay on 8.x it keeps working only for as long as your support agreement is valid.

The Pipelines lineage, and where it ends One feature, four names, removed in VCF 9 vRealize Code Stream vRA 7.x / 8.x Aria Automation Pipelines Aria 8.x VCF Automation Pipelines 8.18.1 (last) Removed VCF Automation 9 no replacement shipped Upgrading 8.x to 9 does not migrate pipelines. They simply stop existing.
The Pipelines feature across rebrands, ending with removal in VCF Automation 9.

What Broadcom suggests, and what I tell clients

The deprecation advisory points at Continuous Delivery Director (CDD) as the portfolio replacement, and otherwise tells you to use external tools. CDD is a real product and a reasonable fit if you need release orchestration across many applications with approval gates and audit. But it is a separate purchase with its own license, install and learning curve. The mistake teams make is treating CDD as a drop-in for a handful of Code Stream pipelines that only ever called vRA. In production, most of those pipelines were thin: clone a template, deploy a blueprint, run a smoke test. That workload belongs in the CI system you already run, talking to the VCF Automation API. Reach for CDD when release governance is the actual problem, not when you just need to deploy a catalog item from a build job.

The delivery model is API-first now

VCF 9.1 was shipped explicitly as an API-first private cloud. Everything Automation Assembler and Automation Service Broker expose is backed by an API contract, and the consumption tooling is built on those contracts. Read that as a design instruction. Delivery does not live inside the platform anymore. It lives in your CI runner, and the platform is the thing it acts on. That is a cleaner separation than Code Stream ever offered: your release logic, secrets, history and rollback live in Git and your pipeline tool, version-controlled and auditable, instead of inside a vRA appliance that you also had to back up and upgrade.

Where delivery runs in VCF Automation 9 The CI runner orchestrates. The platform is the target. Git repo HCL, YAML, scripts CI runner Actions / GitLab / Jenkins vmware/vcfa platform config (IaC) VCF Automation API catalog / deployments Orchestrator / ABX in-platform glue VCF Automation orgs, projects, deployments
Three lanes into the platform: Terraform for config, the API for deployments, Orchestrator for glue.

Infrastructure delivery with the vmware/vcfa provider

Platform configuration, the orgs, projects, regions and content libraries, is the part that should be code, not click-ops. The vmware/vcfa Terraform provider reached v1.0.0 and targets VCF Automation 9 and later. It is distinct from the vmware/vcf provider, which manages SDDC Manager and fleet infrastructure. I covered the provider in depth in Part 21; here it is the first lane of your pipeline. Pin the version and authenticate against the provider (System) org.

terraform {
  required_providers {
    vcfa = {
      source  = "vmware/vcfa"
      version = "~> 1.0"
    }
  }
}

provider "vcfa" {
  url                  = var.vcfa_url
  user                 = var.vcfa_user
  password             = var.vcfa_password
  org                  = "System"
  allow_unverified_ssl = false
}

# Expected: terraform init pulls vmware/vcfa v1.x; plan shows the
# org/project diff. Failure mode: a 401 here means the provider org or
# the System credentials are wrong, not your tenant login.

Application delivery through the deployment API

Requesting a catalog item, the thing a Code Stream stage used to do, is two API calls: get a bearer token from a refresh token, then POST the request. Tokens are short-lived, so the build job mints one per run. This is the same auth flow Part 20 walks through, and the org-type trap from that post applies here too. Keep the deployment payload in a file so it is reviewable in Git.

# 1. Exchange a refresh token for a short-lived access token
ACCESS=$(curl -sk -X POST 
  'https://automation.vcf.example.com/csp/gateway/am/api/auth/api-tokens/authorize' 
  -H 'Content-Type: application/json' 
  -d "{"refreshToken":"${VCFA_REFRESH_TOKEN}"}" 
  | jq -r '.access_token')

# 2. Request the catalog item. This is your deploy stage.
curl -sk -X POST 
  "https://automation.vcf.example.com/catalog/api/items/${ITEM_ID}/request" 
  -H "Authorization: Bearer ${ACCESS}" 
  -H 'Content-Type: application/json' 
  -d @request.json

# Expected: HTTP 200 with a deploymentId you can poll. Failure mode:
# 403 usually means the service account lacks a role in the project,
# not that the catalog item is missing. Confirm the path against your
# release API reference; endpoints shifted between 9.0 and 9.1.
In practice
The first thing I check on a migrated pipeline is token lifetime. Code Stream held credentials for you. A CI job does not, so a long terraform apply followed by an API call can outlive the access token it minted at the top of the run. Mint the token in the deploy stage, not the validate stage.

Wiring it into a real pipeline

Put the two lanes together and you have what Code Stream gave you, with the logic in version control. The pattern is a two-job pipeline: an infra job that runs Terraform against vmware/vcfa to make sure the org, project and catalog state is correct, then a deploy job that requests the catalog item through the API and waits for the deployment to settle. Secrets live in the CI vault, not in the repo. Here it is in GitHub Actions; the GitLab CI and Jenkins shapes are the same idea with different syntax.

The pipeline stages, rebuilt in your CI 1 commit 2 validate / tf plan 3 provision (vcfa) 4 deploy (API) 5 verify / smoke
The same five stages a Code Stream pipeline modeled, now expressed as CI jobs.
name: deliver-to-vcf-automation
on:
  push:
    branches: [ main ]

jobs:
  infra:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: hashicorp/setup-terraform@v3
      - name: Apply platform config (vmware/vcfa)
        working-directory: ./infra
        env:
          TF_VAR_vcfa_url: ${{ secrets.VCFA_URL }}
          TF_VAR_vcfa_user: ${{ secrets.VCFA_USER }}
          TF_VAR_vcfa_password: ${{ secrets.VCFA_PASSWORD }}
        run: |
          terraform init
          terraform apply -auto-approve

  deploy:
    needs: infra
    runs-on: ubuntu-latest
    env:
      VCFA_REFRESH_TOKEN: ${{ secrets.VCFA_REFRESH_TOKEN }}
      ITEM_ID: ${{ vars.CATALOG_ITEM_ID }}
    steps:
      - uses: actions/checkout@v4
      - name: Request catalog item and wait
        run: ./scripts/deploy.sh

# Expected: infra runs first, deploy only on its success. Failure mode:
# if you drop the needs: infra line, deploy races provisioning and
# fails with a 403 against a project that does not exist yet.
Gotcha
The single biggest difference from Code Stream is that nothing waits for you. A pipeline stage in vRA polled the deployment until it finished. Your CI job returns the instant the POST returns, which is before the VM exists. You have to write the poll loop yourself: take the deploymentId from the request, poll the deployment status endpoint until it reads CREATE_SUCCESSFUL or fails, and time it out. Skip this and your pipeline goes green while provisioning is still running.

Where Orchestrator and ABX still fit

External CI is the right home for release flow, but some logic belongs inside the platform. VCF Operations Orchestrator and ABX actions, covered in Parts 17 and 18, handle the work that has to run in response to a platform event rather than a Git push: register a deployed VM in IPAM, write a CMDB record on provision, run a day-2 reconfigure. The clean split is this. If a human or a build triggers it, it runs in CI. If a deployment lifecycle event triggers it, it runs in Orchestrator or ABX through an Event Broker subscription. Trying to do event-driven extensibility from an external pipeline means polling the API on a timer, which is the kind of thing that looks fine in a demo and rots in production.

Code Stream versus what replaces it

There is no single successor, and that is the point. Code Stream bundled three jobs that are better done by three tools. Map your old pipelines against this and most of them collapse into the CI + API row.

What you did in Code Stream Where it goes in VCF Automation 9 Why
Deploy a blueprint / catalog itemCI job calling the deployment APIAPI-first; logic stays in Git
Create / update orgs, projects, contentvmware/vcfa TerraformDeclarative, drift-checked config
Event-driven post-provision tasksOrchestrator / ABX + Event BrokerTriggered by lifecycle, not a push
Release orchestration across many appsCDD (licensed) or your CIBuy only if governance is the need
Application build / test / artifactYour existing CI (always was)Code Stream rarely did this well
What should deliver it? A decision tree for migrated pipelines What are you delivering? Platform config orgs/projects/catalog App deployments request catalog items Event-driven tasks post-provision hooks vmware/vcfa CI + API Orchestrator / ABX
Sort each old pipeline by what it delivers, then route it to the right tool.
Worked example
A tenant with 3 projects ran 12 Code Stream pipelines on 8.x. Sorting them by the tree: 4 were pure blueprint deploys (move to CI + API), 5 created or updated project and catalog config (move to one vmware/vcfa Terraform repo), 2 fired CMDB updates after provisioning (move to an ABX action behind an Event Broker subscription), and 1 was a cross-app release with a manual approval gate. That last one is the only candidate for CDD. Eleven of twelve cost zero extra license. Budget for the platform you already pay for before you buy a new one.
Disclaimer: The Terraform and API examples here change live tenant state. Run them against a non-production project first, keep a service account scoped to the smallest role that works, and confirm the exact API paths against your release reference, since deployment and catalog endpoints shifted between VCF Automation 9.0 and 9.1.

What I would do

Do not look for the Pipelines tab and do not assume CDD is mandatory. Rebuild delivery in the CI system you already run, with platform config in a vmware/vcfa repo and app deployments behind the API. Keep Orchestrator and ABX for event-driven work. Buy CDD only when release orchestration and approval governance across many applications is the actual requirement, not because a few pipelines lost their home. The honest downside: you now own the poll loops, the token handling and the retry logic that Code Stream hid from you. That is more glue code than before. It is also code you can read, test and version, which on balance I would take every time over a pipeline engine locked inside an appliance. If you migrate one pipeline this week, make it a blueprint deploy and prove the token-and-poll pattern end to end. Everything else is a copy of that. Most teams that move off the built-in pipeline engine report the same arc: once the token-and-poll deploy works inside their existing CI, the rest of the migration is mechanical, and the embedded pipelines stop being missed within a sprint.

VCF Automation 9 Series · Part 22 of 41
« Previous: Part 21  |  VCF Automation Guide  |  Next: Part 23 »

References

By Dr. Pranay Jha · VCF architect and long-time vExpert. Verify version-specific behavior against current Broadcom documentation before production change.

About The Author


Discover more from Dr. Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

VCF Automation 9 Series

Discover more from Dr. Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading