Dr. Pranay Jha

VMware • Cloud • AI • Enterprise Architecture

FORMERLY
VMware Insight & Cloud Pathshala
What began over a decade ago as a passion for sharing knowledge has evolved into a unified platform for Enterprise AI, VMware, Cloud Architecture, Research, and Modern Infrastructure.
, ,

VCF Operations Orchestrator in VCF Automation: Workflows for Extensibility (VCF Automation 9 Series, Part 17)

How VCF Operations Orchestrator (ex-vRO) plugs into VCF Automation 9 for extensibility: instance models, standing up an external Orchestrator, the rights-bundle trap, event subscriptions and when to use ABX instead.

VCF Automation 9 Series · Part 17 of 41
TL;DR · Key Takeaways
  • VCF Operations Orchestrator is the workflow engine you reach for when a Cloud Template and an event subscription cannot do the job alone. It is the same product you knew as Aria Automation Orchestrator and vRealize Orchestrator (vRO), renamed and folded into VCF.
  • It attaches to VCF Automation three different ways, and the org type decides which: embedded in the Provider Consumption org, embedded or external in a VM Apps org, external-only in an All Apps org.
  • In an All Apps org the Orchestrator tab stays empty for everyone except the provider admin until you publish the Orchestrator rights bundle. This is the number one reason teams think the integration is broken.
  • The external appliance in VCF 9 is a manual OVA deploy, not a Fleet-managed component. Authentication is wired from the appliance CLI with vracli vro authentication set.
  • Use Orchestrator for stateful, multi-system, long-running logic. Use ABX for short, stateless functions. Picking the wrong one is a maintenance tax you pay for years.
Who this is for:
Cloud and platform admins who run VCF Automation and need to extend it past what blueprints offer. Automation engineers writing the workflows. Architects deciding the extensibility pattern before a tenant goes live.
Prerequisites:
A working VCF Automation 9.x instance with at least one organization, provider (system) admin access, and a vCenter the Orchestrator can talk to. Familiarity with projects, cloud templates and the Service Broker catalog helps.

You integrate an external Orchestrator into a fresh All Apps tenant, run a data collection, log in as the org admin, and the Orchestrate tab is not there. Not greyed out. Gone. You check the integration: connected, green, workflows collected. You check the role: Administrator. Everything looks right and nothing works. That gap, between a technically correct integration and a usable one, is where most Orchestrator projects stall in VCF Automation 9. The fix is not in the integration screen at all, and I will get to it. First, what this engine actually is now and where it bolts on.

The engine, renamed but not rewritten

VCF Operations Orchestrator (was Aria Automation Orchestrator, was vRealize Orchestrator) is a JavaScript-based workflow engine with a plug-in model. You build workflows out of scriptable tasks, decisions, loops and waits, and you reach into other systems through plug-ins. It ships with a standard set, including vCenter and a VCF Automation plug-in, so a workflow can find a VM, attach a tag, call a REST endpoint, or drive a downstream system without you writing the transport layer.

Nothing about the authoring model changed in the rename. If you wrote vRO workflows for vRA 8.x, those skills carry over one to one. What changed is how the engine is packaged, how it authenticates, and how it is exposed to tenants. That packaging is the part people get wrong, because it is genuinely different from the 8.x world where Orchestrator was just embedded and always there.

Orchestrator is one of three extensibility surfaces in VCF Automation. The catalog and forms (covered in Part 14) shape what users request. The Event Broker fires events at points in a deployment lifecycle. Orchestrator and ABX are what those events call out to. Hold that picture; the rest of this post lives inside it.

Where Orchestrator plugs in Request to provisioning, with extensibility hanging off the Event Broker Service Broker catalog + forms Assembler provision engine Event Broker lifecycle topics 1 2 Orchestrator workflows ABX stateless actions Plug-ins vCenter, NSX, REST, ServiceNow
The Event Broker fires lifecycle topics; Orchestrator workflows and ABX actions subscribe and act through plug-ins.

Three ways it attaches, and the org type decides

This is the part that does not match your 8.x muscle memory. The organization model in VCF Automation 9 (see Part 3) changes how Orchestrator is wired, and a feature that works one way in a VM Apps org behaves differently in an All Apps org.

Org typeEmbedded?External?Instances
Provider ConsumptionYes, by defaultn/aOne embedded
VM AppsYesYesMultiple, mix embedded + external
All AppsNoYes, requiredExactly one, per org

VM Apps org: the familiar model

A VM Apps org is close to Aria Automation 8.x, and Orchestrator behaves accordingly. You can register multiple instances, embedded and external, against the same org. When you upgrade an 8.x deployment, your old embedded vRO and its content land as an embedded integration automatically, and any external vROs you had become external integrations with their content intact. When several instances are registered, you tell them apart with capability tags, the same tagging mechanism covered in the org model post.

All Apps org: external only, one to one

An All Apps org is the Kubernetes-API architecture, and it only works with an external Orchestrator. The provider deploys, configures and registers that instance per request, and there is exactly one per org. You cannot add a second, and you cannot use the embedded engine. Want to swap it? Delete the existing integration first, then add the new one. The one-to-one binding is deliberate: each All Apps org gets its own Orchestrator so tenant content and credentials stay isolated. That isolation is the whole point, and it is also why the embedded model is off the table here.

Provider Consumption org: the embedded default

The Provider Consumption org ships with one embedded Orchestrator, reached on the Orchestrate tab in the provider portal. The provider can publish workflows from that embedded instance as catalog items down to All Apps orgs, which is a clean way to offer a vetted workflow to tenants without handing them the engine. Broadcom is explicit that embedded instances are meant for the provider org; for VM Apps and All Apps tenants, integrate an external Orchestrator. I agree with that guidance, and the reason is operational, not ideological: an external appliance you can scale, back up and patch on its own cadence beats sharing the provider plane.

Instance topology by org type One platform, three attachment patterns Provider Consumption Embedded vRO Publishes workflows as catalog items down VM Apps org Embedded vRO External vRO (n) Multiple, capability-tagged All Apps org External vRO x1 Exactly one, isolated per tenant
Embedded for the provider, multiple instances for VM Apps, a single isolated external instance for each All Apps tenant.

Standing up an external Orchestrator

Here is the first surprise for anyone coming from Fleet-managed thinking: in VCF 9 the external Orchestrator is not a managed solution. It does not roll out from Fleet Management the way the core appliances do. You download the OVA from the Broadcom portal and import it into a vCenter yourself. It can even be a different vCenter, as long as the network path exists.

The deploy itself is ordinary. One field is a trap on 9.0: do not set an NTP server during the OVA wizard, because the services may not start afterward. Set NTP from the CLI once the appliance is up. On 9.1 that bug is fixed and you can set NTP in the wizard. I still set it from the CLI out of habit, because it is one verifiable command and time skew is a miserable thing to debug after the fact.

# On the Orchestrator appliance CLI
# 1. Set and verify NTP (skip the OVA NTP field on 9.0)
vracli ntp systemd --set ntp1.corp.local,ntp2.corp.local
vracli ntp show-config

# 2. Wire authentication to an All Apps org (no UI yet until this runs)
vracli vro authentication set -p tm 
  -u provider-admin 
  -hn https://automation.corp.local 
  --tenant tenant-payments

# 3. Activate the config and start the auth service
/opt/scripts/deploy.sh

A detail worth internalizing: the Orchestrator may show no UI at all until authentication is configured. People deploy the OVA, browse to it, get nothing, and assume the deploy failed. It did not. The web client comes up after the auth step, not before. The -p tm flag selects VCF Automation as the auth provider, -u is a provider (system) admin, -hn is the VCF Automation FQDN or load balancer, and --tenant is the All Apps org name, which must already exist.

With auth set, register the instance inside the org under Administer, then Integrations, pointing at the Orchestrator FQDN. No credentials are needed there because VCF Automation is already the auth provider. That same screen has a manual data collection button. Use it whenever you author a new workflow and want Automation to see it without waiting; the scheduled collection runs about every ten minutes, which is fine for steady state but slow when you are iterating.

In practice
The first thing I check after a deploy is not the workflow library, it is time sync and authentication. Nine out of ten new-instance tickets I see trace back to one of those two, not to anything in the workflow logic. Get vracli ntp show-config clean before you touch a single workflow.

The rights-bundle trap (this is the empty-tab fix)

Back to the opener. The integration is green, the role is Administrator, and the Orchestrate tab is still missing for everyone except the provider admin. The reason is a VCF Automation 9 capability: external components can contribute their own rights bundles, sets of permissions tailored to that component, which you then attach to Automation roles. Orchestrator ships such a bundle, and until it is published to the org, no tenant user, not even an org Administrator, sees the Orchestrator menu items.

Two steps fix it, both in the provider (system) tenant. First, enable Advanced Rights Bundles in the Feature Flags section. Second, find the Orchestrator Rights Bundle under Access Control, then Rights Bundles, and publish it to the orgs that need it. By default that bundle is published only to the Provider Consumption org, so a brand-new tenant has nothing. This connects directly to the governance and role model from Part 16: in VCF Automation 9, visibility is a function of published rights, not just role names.

Gotcha
Symptom: the Orchestrate tab is visible to the provider admin but missing for org admins. Cause: the Orchestrator rights bundle is not published to that org. Fix: enable Advanced Rights Bundles (Feature Flags), then publish the Orchestrator Rights Bundle to the target org. The role was never the problem.

Making a workflow do real work

A registered Orchestrator is inert until something calls it. There are three doorways: an event subscription fires a workflow at a lifecycle point, a custom resource maps create/read/update/delete to workflows, or a day-2 resource action exposes a workflow as a button on a deployed item. The most common starting point is the event subscription, so make that concrete.

Event subscriptions: workflows on lifecycle topics

Assembler exposes event topics across the provisioning lifecycle. You create a subscription that binds a topic to a workflow, optionally with a condition so it only runs for some requests. A blocking subscription pauses provisioning until the workflow returns, which you use when the workflow must finish before the VM is considered ready, for example registering it in a CMDB or reserving an IP. The architecture behind these topics is in Part 2.

A workflow bound to compute.provision.post receives the provisioned resource in its payload. A scriptable task inside it might tag the VM in vCenter and hand off to an external system:

// Scriptable task on a workflow bound to compute.provision.post
// Inputs (from the event payload): resourceNames (Array/string), owner (string)
for each (var vmName in resourceNames) {
    System.log('Post-provision tag for ' + vmName + ' owner=' + owner);
    var vms = VcPlugin.getAllVirtualMachines(null,
        'xpath:name="' + vmName + '"');
    if (vms == null || vms.length == 0) {
        throw 'VM not found in vCenter inventory: ' + vmName;
    }
    // attach owner tag, then call the CMDB (REST plug-in) ...
}

That throw matters. On a blocking subscription, an unhandled error fails the deployment, which is exactly what you want if the CMDB registration is mandatory: no silent half-provisioned VMs. On a non-blocking subscription the same error is logged and provisioning continues. Decide which behavior you want before you write the workflow, not after a tenant files a ticket about a stuck request.

Custom resources and day-2 actions

Past event subscriptions, workflows back custom resources, where you model a non-native object (a DNS record, a database, a load-balancer member) and wire its lifecycle to workflows so it can sit in a Cloud Template like any other resource. Day-2 resource actions surface a workflow as an operation on an already-deployed item, a Resize, a Reset Password, an Add Disk. Both are deeper topics; the point here is that they all bottom out in the same engine you just stood up.

Worked example
A tenant provisions 40 VMs a week. You want every one registered in ServiceNow before it is handed over. Bind one workflow to compute.provision.post as a blocking subscription scoped to that project. Set a workflow timeout of 5 minutes so a CMDB outage fails fast instead of hanging the request for an hour. Result: 40 clean CMDB records a week, and on the rare CMDB outage, a handful of clearly failed deployments you can resubmit, rather than 40 ghost VMs nobody recorded. The first time a CMDB outage hits a non-blocking design you stop second-guessing the choice: deployments keep flowing and you reconcile the few missed records afterward, which is exactly the failure mode you want on a dependency you do not control.
Request to workflow, end to end A blocking subscription on compute.provision.post Catalogrequest Assemblerprovision Event topicprovision.post Workflow(blocking) ServiceNow return blocks until workflow completes
A blocking subscription holds provisioning until the workflow returns, so the CMDB record exists before handover.

When not to reach for Orchestrator

Orchestrator is not the answer to every extensibility need, and reaching for it reflexively is a mistake I see often. If your logic is a short, stateless function, a few lines that call one API and return a value, ABX (covered next in this series) is the lighter tool and the one your team can maintain without learning the workflow editor. Orchestrator earns its weight when the logic is stateful, spans multiple systems, needs to wait on something, or has to be reused across many subscriptions and custom resources. The plug-in ecosystem and the inventory model are what justify it.

The case against it is real. An external Orchestrator is another appliance to deploy, authenticate, patch and back up, and in an All Apps world that is one per tenant. That is genuine operational cost, and for a single small function it is not worth paying. What to validate first: confirm whether ABX can do the job, confirm the Orchestrator can reach every system it needs to drive, and confirm your content lifecycle (Git branching, packages, version history) is in place before you have ten workflows scattered across tenants with no source of truth.

My take
I default to ABX for anything that fits in one function and reach for Orchestrator the moment a second system, a wait state, or reuse across orgs enters the picture. The deciding question is not capability, both can technically do most things, it is who maintains it in two years and whether the operational overhead of another appliance is justified by the logic it runs.
Disclaimer: Enabling feature flags, publishing rights bundles, and binding blocking event subscriptions change production behavior for real tenants. Test in a non-production org first, confirm workflow timeouts and error handling, and roll out per project rather than platform-wide.

What I’d Do

Stand up one external Orchestrator per All Apps tenant, wire authentication from the CLI, and publish the rights bundle before you tell anyone the integration is live, so the first thing a tenant admin sees is a working Orchestrate tab and not an empty one. Keep the embedded provider instance for provider-published catalog workflows only. Then write your first workflow against a single blocking event subscription with a real timeout, prove the failure path, and only then expand into custom resources. If you take one thing from this post: the integration being green is not the same as the integration being usable, and the gap is almost always the rights bundle.

Standing up an Orchestrator this week? Start with the empty-tab fix above before you write a line of workflow code, and save yourself the support ticket.

VCF Automation 9 Series · Part 17 of 41
« Previous: Part 16  |  VCF Automation Guide  |  Next: Part 18 »

References

About The Author


Discover more from Dr. Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

VCF Automation 9 Series

Discover more from Dr. Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading