A misconfigured retry loop in a batch job pushed a NAT gateway from 90 US dollars a day to 2,400. The anomaly detector caught it on the second day, exactly as designed, and sent an email. The email went to a distribution list created eighteen months earlier by somebody who had since left. The charge ran for nineteen more days before it appeared as a line item somebody questioned in the monthly review. Total damage, around 44,000 US dollars, of which roughly 4,000 was unavoidable.

The detection worked. The plumbing after the detection did not exist. That ratio holds almost everywhere I have looked, which is why this part spends more time on thresholds, routing and measurement than on how the models work. Part 8 built a forecast for spend you can anticipate. This is the capability that catches the spend you cannot.

What counts as an anomaly

The FinOps Foundation defines an anomaly as a level of spending that is different, usually higher, from normal historical or expected spend. Anomaly Management is the capability that detects, identifies, clarifies, alerts on and manages those events in time to limit the damage. It sits in the Understand Usage and Cost domain, which tells you something important: it is a visibility capability, not an optimisation one. Detecting an anomaly saves nothing. Somebody acting on it does.

Worth being precise about what an anomaly is not. It is not a budget breach, because a budget is a number you chose and an anomaly is a deviation from a pattern the system learned. A team can be 40 percent under budget and still have an anomaly, if a service that normally costs 60 US dollars a day suddenly costs 900. It is also not the same as a forecast miss, which is a failure of the intake process described in the previous part. Anomaly detection compares yesterday against a learned pattern. Forecasting compares next quarter against a plan.

The Foundation is direct about a consequence that trips up new practices: detection works by comparing current usage against past usage, so anticipated new spend will trigger alerts. A team standing up a training environment in an account that has never carried real cost will fire an anomaly, and it will be correct to do so. The answer is not to suppress it. The answer is a documented route for teams to declare planned spikes in advance, and a discipline of recording why each alert was closed.

One more distinction that matters operationally. Most anomalies are not incidents. A 300 US dollar spike in a development account is a note in a log. A 30,000 US dollar spike in a production account is a page. If your process treats both the same, you will either page people constantly or miss the one that mattered, and both outcomes end with people ignoring the alerts.

flowchart LR
  A[Usage lands in billing data] --> B[Detector scores the day]
  B --> C{Above threshold}
  C -->|No| D[Visible in console only]
  C -->|Yes| E[Alert raised]
  E --> F[Route by resource owner]
  F --> G[Triage and root cause]
  G --> H{Legitimate}
  H -->|Yes| I[Document and adjust expectation]
  H -->|No| J[Remediate the workload]
  I --> K[Feed outcome back to tuning]
  J --> K

How AWS, Azure and Google Cloud detect anomalies

All three are free, all three are machine learning based, and all three differ in ways that change how you design the process around them.

AWS Cost Anomaly Detection

You create monitors, which define what to watch, and alert subscriptions, which define who hears about it. Monitors come in two flavours. AWS managed monitors track every value in a dimension automatically and pick up new accounts, tag values or cost categories as they appear, up to the top 5,000 values by spend. Customer managed monitors track up to 10 values you pick by hand and aggregate them together. The dimensions available are AWS services, linked accounts, cost allocation tags and cost categories. You can have one AWS services monitor and up to 500 custom monitors.

Subscriptions carry a frequency of IMMEDIATE, DAILY or WEEKLY. Only IMMEDIATE delivers as soon as an anomaly is detected, and it requires an Amazon SNS topic rather than email. Daily summaries are generated at 00:00 UTC and carry the top 10 alerts from the previous day sorted by cost impact, which means an anomaly detected at 04:30 UTC on the fourteenth arrives in the summary sent at 00:00 UTC on the fifteenth. Each subscription takes up to 10 email recipients or one SNS topic. AWS reports cost impact as actual spend minus expected spend, and impact percentage as cost impact divided by expected spend, times 100.

Microsoft Cost Management anomaly detection

Azure evaluates anomalies daily at subscription scope, comparing the day total against a forecast built from the last 60 days so that recurring patterns such as a Monday peak do not read as anomalies. The model is a univariate time series, unsupervised, reconstruction based model that uses a deep learning algorithm called WaveNet, and it is a different mechanism from the Cost Management forecast. Detection runs 36 hours after the end of the day in UTC, so a complete data set is available.

Alerts are created under Cost alerts with the alert type set to Anomaly, and require the Cost Management Contributor role or the Microsoft.CostManagement/scheduledActions/write permission. Three constraints shape the design. Anomaly alert rules can only be created at subscription scope. There is a limit of five alert rules per subscription. The alert email is sent once, at detection, and there is no repeat. Automation is possible through the Scheduled Actions API with the kind set to InsightAlert, and the documented pattern for anything richer is a Logic App watching a mailbox.

Google Cloud Cost Anomaly Detection

Generally available, no setup, no charge. The Anomalies dashboard lives under Cost management in the Billing console and each anomaly carries a root cause analysis panel naming the top services, regions and SKUs behind the spike, with links straight into a filtered billing report. Thresholds are set automatically from your usage pattern and can be overridden by a Billing Account Administrator, and Google gives you two dials rather than one: a cost impact amount and a deviation percentage. Notifications go to Billing admins, Essential contacts or Project owners, at either individual anomaly or daily summary frequency, or to Pub Sub for anything programmatic. The dashboard also takes structured feedback on whether each anomaly was unexpected, expected, or insignificant, which the model uses to adapt.

Verified against AWS, Microsoft and Google documentation in July 2026. Console behaviour and limits change, so re check before you standardise on any of it.

Setting thresholds people will not ignore

A single absolute threshold is the most common configuration and the worst one. Set it at 5,000 US dollars and you will miss a development account whose spend went from 40 to 900 a day, which is a 2,150 percent increase and a genuine engineering defect. Set it at 200 and a large production account will alert on ordinary weekday noise until people build a mail rule to hide it.

Use both dials. An absolute threshold catches anything materially expensive regardless of the base. A percentage threshold catches proportional runaways in small accounts that would never trip a dollar figure. Combine them with OR rather than AND, because AND means an anomaly has to be both large and proportionally large, and the small account defect is exactly the case AND throws away. AWS supports this directly through a ThresholdExpression using ANOMALY_TOTAL_IMPACT_ABSOLUTE and ANOMALY_TOTAL_IMPACT_PERCENTAGE nested under Or. Google Cloud exposes both dials in the console. Azure does not let you set thresholds at all, which is a real gap and the reason most Azure practices run budget alerts alongside anomaly alerts rather than instead of them.

As starting values for an estate in the low hundreds of thousands per month, I use 1,000 US dollars absolute or 75 percent, and then tune once a month against how many alerts actually produced an action. If more than about a third of alerts are closed as expected spend, the threshold is too tight or your intake process from Part 8 is not capturing planned work. Both are fixable, and both are cheaper than the alternative, which is teams learning that these emails do not matter.

Because AWS is the only one of the three where the threshold is properly programmable, that subscription is worth defining as code rather than clicking. The following creates a subscription that fires on either dial.

aws ce create-anomaly-subscription --anomaly-subscription '{
  "SubscriptionName": "platform-team-immediate",
  "Frequency": "IMMEDIATE",
  "MonitorArnList": ["arn:aws:ce::111122223333:anomalymonitor/EXAMPLE-MONITOR-ID"],
  "Subscribers": [
    {"Type": "SNS", "Address": "arn:aws:sns:us-east-1:111122223333:finops-anomalies"}
  ],
  "ThresholdExpression": {
    "Or": [
      {"Dimensions": {"Key": "ANOMALY_TOTAL_IMPACT_ABSOLUTE",
        "MatchOptions": ["GREATER_THAN_OR_EQUAL"], "Values": ["1000"]}},
      {"Dimensions": {"Key": "ANOMALY_TOTAL_IMPACT_PERCENTAGE",
        "MatchOptions": ["GREATER_THAN_OR_EQUAL"], "Values": ["75"]}}
    ]
  }
}'

Expected output is a single JSON object containing SubscriptionArn. Cost Explorer API calls run against us-east-1.

Two failures are common here. Setting both Threshold and ThresholdExpression is rejected, because Threshold is deprecated and is only kept as shorthand for the expression form; use one or the other. And IMMEDIATE frequency requires an SNS subscriber, so passing an email address with IMMEDIATE will not give you real time alerts. If the monitor ARN is wrong you get UnknownMonitorException rather than a validation error, which sends people looking in the wrong place.

Route the alert to somebody who can act

This is where the capability is won or lost, and it depends entirely on allocation. If a resource has no owner tag, an anomaly on that resource has no recipient, and it goes to the central FinOps mailbox where it becomes somebody else’s afternoon. The Foundation is explicit that allocation metadata is critical to effective anomaly detection, and specifically to determining who can best evaluate and resolve what was detected. Everything you built in Part 5 pays out here.

The practical shape on AWS is a managed monitor on your owner tag key, so that every tag value is tracked independently, with an IMMEDIATE subscription publishing to SNS. From SNS you fan out: a Lambda that reads the anomaly payload, looks up the owner from the dimensional value, and opens a ticket in the team’s own queue rather than sending an email. The payload gives you what you need, including anomalyId, the dimensional value, impact totals and a ranked rootCauses array. On Google Cloud the equivalent path is Pub Sub. On Azure, with email as the only channel, the documented pattern is a Logic App or Power Automate flow watching a shared mailbox and creating the ticket from there, which is less elegant but works.

Route by severity as well as by owner, but do not conflate the two. Severity decides the channel, a ticket versus a page. Ownership decides the recipient. A practice that routes only by severity ends up paging a central team about a workload they have never seen, and a practice that routes only by owner sends a 40,000 US dollar production anomaly as a low priority ticket into a backlog.

flowchart TD
  A[Anomaly above threshold] --> B{Owner tag present}
  B -->|No| C[Central FinOps queue and tagging defect raised]
  B -->|Yes| D{Impact and environment}
  D -->|Under 1k or non prod| E[Ticket in the team queue]
  D -->|Over 1k in production| F[Page the on call engineer]
  D -->|Over 25k anywhere| G[Page plus finance and leadership notified]
  E --> H[Close with a documented reason]
  F --> H
  G --> H

Measure detection in hours, not in count

Reporting the number of anomalies found is the vanity metric of this capability. It goes up when the estate grows, up when you tighten thresholds, and tells you nothing about whether money was saved. The Foundation lists better options: mean time to detect, mean time to notify the owner, duration of unresolved anomalies, the count of actioned anomalies with the spend avoided, and the count of unactionable ones categorised by why they were ignored.

Mean time to notify, abbreviated MTTN, is the one I would put on the wall. Mean time to detect, or MTTD, is largely bounded by the provider and there is only so much you can do about it. The gap between detection and the owner acknowledging is entirely yours, and in most practices it is far larger than the detection lag itself. In the NAT gateway story that opened this part, MTTD was about 30 hours and MTTN was nineteen days.

The Foundation also publishes an anomaly cost percentage KPI: total cost of anomaly spikes divided by total spend, with bands of green below 2 percent, yellow between 2 and 7, and red above 7 where you have runaway costs. It was written with AI spend in mind and adapts cleanly to general cloud spend. It is the one number a leadership audience understands without preamble, because it converts a detection story into a proportion of the bill.

Illustrative figures from one quarter, chosen to show a realistic spread. Note that detection time barely varies while notification time varies by a factor of eighteen, and that the two worst cases are both routing failures rather than detection failures.

Why anomaly programmes stall

Four patterns cover most of it. The first is alert fatigue from a threshold set once and never revisited, usually too low, which teaches an entire engineering organisation that cost emails are noise. Recovering that attention takes months and a visible change to the process, not a quiet threshold edit.

The second is the orphaned recipient, which is the story this part opened with. Audit your alert destinations quarterly, treat a distribution list with no active members as a defect, and prefer a ticket queue over a mailbox because queues have owners and mailboxes do not. The third is no record of outcomes. If nobody writes down why an alert was closed, the same expected spike alerts every month and the model never learns from your feedback. Google Cloud makes this concrete by letting you mark an anomaly as expected with a business reason, and the model adapts. Use it.

The fourth is treating detection as a FinOps team activity rather than an engineering one. The Foundation puts the investigation and resolution firmly with engineering, and the FinOps role with tooling, thresholds and communication. A central team that investigates every anomaly itself becomes the bottleneck within a quarter, and the metric that reveals it is duration of unresolved anomalies climbing while the count stays flat.

Two thresholds, owner based routing, and MTTN on the wall

Here is the recommendation for this part, stated plainly. Turn on the native detector in every provider you use, because it is free and there is no reason not to. Configure two thresholds rather than one, an absolute amount and a percentage, combined with OR so that either can fire. On AWS, run a managed monitor on your owner tag key so each team is tracked independently, and use IMMEDIATE frequency into SNS rather than daily email summaries. On Google Cloud, set both the cost impact and deviation dials and send to Pub Sub. On Azure, accept that you cannot tune the threshold and pair anomaly alerts with daily budget alerts to cover the gap.

Then spend the rest of your effort after the alert. Route to the resource owner, escalate by impact and environment, land it in a queue rather than an inbox, and require a documented close on every alert including the ones that turn out to be legitimate. Report mean time to notify and anomaly cost percentage, and stop reporting how many anomalies you found. If you do only one thing from this part, make it the routing, because a mediocre detector wired to the right person beats an excellent detector wired to a mailing list every time.

Part 10 moves from unexpected cost to meaningful cost, with unit economics: cost per customer, per transaction and per feature, which is what turns a cloud bill into a business number. The groundwork is in Part 5 on allocation and Part 7 on reporting, and every part so far is indexed on the Cloud FinOps guide.

One thing to do this week: open your anomaly alert configuration, find every destination address, and send a test message to each one. Count how many bounce or reach nobody. That number is your real detection capability.

References

• Anomaly Management capability, FinOps Framework, FinOps Foundation
• Getting started with AWS Cost Anomaly Detection, AWS Cost Management User Guide
• AnomalySubscription and ThresholdExpression, AWS Billing and Cost Management API Reference
• Identify anomalies and unexpected changes in cost, Microsoft Cost Management
• View and manage cost anomalies, Google Cloud Billing documentation

Monetary figures, detection times and monthly percentages are illustrative and chosen to show realistic movement. Framework wording, provider limits and API field names were verified in July 2026 and change over time.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts