An alert that only ever pages a human is half a tool. The other half is what happens next, and for a surprising number of alerts the next step is the same boring fix every time. Those are the ones worth automating. The trick is knowing which alerts are boring and reversible, and which ones are telling you something you had better look at yourself.

This part is about closing the loop from alert to action in VCF Operations without handing the environment a loaded gun. I will walk the chain that connects a symptom to an automated fix, show where I draw the line on what to automate, and give you the numbers from a real cleanup where automation turned a 42-minute chore into a 4-minute one.

The chain from symptom to fix

Automation in VCF Operations rides on the alerting model I covered earlier in the series. A symptom defines a condition, an alert definition groups symptoms and attaches recommendations, and a recommendation is the advice for resolving it. What people miss is that a recommendation does not have to be text a human reads. It can be mapped to an orchestrator action, and when the alert fires, the action runs on the next collection and the alert clears if the fix worked. That is the whole loop: a condition triggers, a fix executes, the next cycle confirms.

An alert definition can carry more than one recommendation, ranked in the order you want them tried or considered. The top one might be an automated action while the ones below it are written guidance for the cases the automation does not cover. That ranking is worth setting deliberately, because it is also the order a human reads under pressure. Put the fix that resolves the common case first and the caveats below it, so the person who lands on the alert at 2am sees the right move before the footnotes.

The Management Pack for Orchestrator is what wires a recommendation to a workflow or action, so when the mapped alert fires, the automation is invoked without anyone clicking. Compliance remediation works the same way, which is how a configuration drift alert can put a setting back on its own. The mechanics are not the hard part. Deciding what deserves that treatment is.

Actions or workflows?

VCF Operations Orchestrator is the same engine that used to be called vRealize Orchestrator and then Aria Automation Orchestrator, now folded into VCF. It gives you two ways to build automation, and they are not interchangeable. Actions are pure JavaScript. You can read them, lint them, unit test them and keep them in version control like any other code. Workflows are the visual drag-and-drop canvas, built originally for admins who wanted orchestration without writing much code. My rule is that anything I will maintain for years gets written as an action, because code I can test ages better than boxes I have to click through to understand. Workflows earn their place as the glue that sequences several actions and external calls.

What to automate, and what to leave alone

Here is where I disagree with the instinct most teams have. The loud alerts, high CPU, high memory, are the ones people want to automate first, and they are exactly the ones you should not. A high-CPU alert is a symptom with a dozen possible causes, and rebooting the VM to clear it can turn a slow afternoon into an outage. Automate the boring, reversible, high-frequency toil instead: stale snapshot cleanup, reclaiming VMs that have been powered off for weeks, putting a drifted config setting back. The test is simple. If the fix is always the same, low blast radius, and reversible, automate it. If the right fix depends on judgment, leave a human in the loop and let the recommendation be text.

I think of it as three tiers. Notify-only, where the alert just tells someone. One-click, where a person reviews and triggers the mapped action by hand. And fully automatic, where the action runs on its own. Most alerts should stay in the first two tiers. The gauge below is one site where 72 percent of alert definitions had a recommendation attached at all, and only a slice of those were fully automatic. That ratio is healthy. An environment where most alerts auto-remediate is one that has stopped thinking.

A worked example: snapshot sprawl

Snapshots are the perfect automation candidate: the fix is always the same, the scope is easy to bound, and leaving them costs real storage. I built a symptom for snapshots older than 72 hours, scoped the alert to VMs carrying a dev tag so nothing in production was ever in range, and mapped the recommendation to an orchestrator action that consolidates and removes them. In the first week it cleared 140 stale snapshots and reclaimed 2.1 TB. More to the point, the time from the snapshot ageing out to it being gone dropped from roughly 42 minutes of someone noticing and handling it to about 4 minutes on the next collection.

Guardrails that make automation safe

The difference between automation you trust and automation you fear is guardrails, and they are cheap to add. The first is scope, which I keep coming back to because it is the one that saves you. Every action should be bound to a tight group, and a dev tag is a better filter than a promise to be careful. The second is a promotion path. A new action starts as a one-click recommendation a human triggers and watches, and only after it has run cleanly enough times does it earn the fully automatic tier. Skipping straight to fully automatic on day one is how the snapshot cleaner you were proud of deletes something it should not have.

The third guardrail is a record. Every automated run should write a log entry saying what fired, what it touched, and whether the alert cleared afterward. Without that, an action that quietly misbehaves is invisible until someone goes looking, and by then the trail is cold. Where a workflow makes an irreversible change, put an approval step in front of it so a human signs off before it runs, which is one of the cases where a visual workflow beats a bare action. Reversible fixes can run on their own. Irreversible ones should ask first.

Promote in stages, never in one jump

The safest automations still earn trust in stages. Start notify-only, so the recommendation appears but nothing runs. Then allow it as a one-click action a human confirms. Only after it has run cleanly on a tight, low-risk group for a few weeks should it become fully automatic, and even then scope it to tags you trust before production. In 9.1 the move to OAuth 2.0 API tokens lets you give an external trigger its own scoped credential instead of a shared admin login, which keeps the blast radius of an automated action small. Log every run and review the log before you widen the scope.

Measuring whether it works

Automation is not a set-and-forget. An action that ran is not the same as a problem that was fixed, and the only way to tell them apart is to measure. I watch three numbers. Mean time to remediate tells me the automation is faster than the human path it replaced, which for the snapshot job was 42 minutes down to 4. Action success rate tells me how often the mapped action actually cleared the alert on the next cycle rather than firing and leaving the condition in place. And the count of actions that ran but did not clear their alert is my early warning that a fix has stopped matching the problem it was written for.

That last number is the one people forget, and it is the most useful. An action whose success rate is drifting down is telling you the environment moved out from under it. Maybe the setting it corrects is now managed elsewhere, maybe the scope has grown to include objects it was never meant to touch. Put these three on a small dashboard, using the widgets from the dashboards part of this series, and review them every so often. Automation you do not measure is automation you are trusting on faith.

Two things that changed in 9.1

Two smaller updates are worth knowing if you build automation. The orchestrator UI session timeout is now configurable in minutes, which sounds trivial until you have lost half an hour of unsaved workflow to a short timeout. Set it sensibly for your developers. And the Management Pack Builder can now pull metrics from Prometheus servers, so you can build a management pack on Prometheus data and reuse it in VCF Operations without writing a new integration through the SDK. If your automation depends on a metric that only lives in Prometheus today, that is a shorter path than it used to be. If you are planning the move to 9.1, fold both into the same change window I set out in the VCF 9.1 upgrade series.

Common questions

Where does the automation actually run?
In VCF Operations Orchestrator, the engine that used to be vRealize Orchestrator. A recommendation on an alert is mapped to an action or workflow there, and it runs when the alert fires and is confirmed on the next collection.

Action or workflow for a new fix?
Write it as an action if it has real logic you will maintain, because JavaScript is testable and versioned. Use a workflow when you are sequencing several steps, approvals or external calls with little custom logic.

Should I auto-remediate high CPU or memory?
No. Those are symptoms with many causes. Automating a reboot or a resize on them can worsen an incident. Keep them notify-only and automate the boring, reversible toil instead.

How do I stop an action from running too widely?
Scope the alert to a tight group, dev tags before anything in production, and reason about the blast radius before you promote it from one-click to fully automatic. Log every run so you can see what it did.

Can compliance drift self-heal?
Yes. A configuration drift alert can map to a remediation action that puts the setting back, the same pattern as any other auto-remediation. Keep it scoped and logged like the rest.

References

• VCF Operations: Configuring and Executing Auto Remediation
• VCF Operations 9.1: What’s New
• VCF Operations Orchestrator: Workflows for Extensibility

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts