,

Cloud Cost Forecasting and Budgeting That Survives Contact With Reality (Cloud FinOps Series, Part 8)

A forecast is a promise somebody has to keep, and a budget is the money that backs it. Here is how to build one that holds, how accurate it has to be, and why you should alert on forecast rather than actual.

Cloud FinOps Series · Part 8 of 20

The forecast said 412,000 for the quarter. The bill came in at 511,000. Nobody had lied, nobody had made an arithmetic error, and four people had signed the number off. What had actually happened was that a data platform team switched on a managed streaming service in week three, and no part of the forecasting process had any way to hear about it before the invoice did.

That is the shape of almost every forecast failure I have seen. The model is rarely the problem. The problem is that a forecast is a statement about what the organisation intends to do, and most organisations have no reliable channel between the people who decide to do things and the people who wrote the number down. Part 7 ended with reporting that describes the past. This part is where the same data starts committing you to a number in the future, which is a considerably harder thing to get right.

Who this is for: The assumed starting point is that you can read a provider bill (Part 4), most of your spend carries an owner through tags or account structure (Part 5), and you have reporting that people read (Part 7). No finance background is assumed. Terms such as amortised, driver, variance and prediction interval are defined the first time they appear.

Forecast and budget do different jobs

People use these two words as if they were the same thing, and the confusion causes real damage in meetings. A forecast is a model of what you expect to spend, built from history plus known plans. A budget is the money that has been allocated to back that expectation. The forecast is a prediction. The budget is a constraint. When the two diverge, only one of them has a treasury department attached to it.

The FinOps Foundation separates them into distinct capabilities for exactly this reason, and adds a third in front of both. Planning and Estimating is investigative work, usually driven by engineers asking what a proposed system would cost. Forecasting takes those estimates and establishes an expectation among all stakeholders. Budgeting then uses the forecast as the baseline against which finance allocates funding. The Foundation makes the consequence explicit: once a forecast model is agreed for an application, the budget owner for that application has to live up to it, or be responsible for seeking variance funding to cover the shortfall.

That last sentence is the whole discipline in one line. A forecast without an owner who is accountable for the gap is not a forecast, it is a chart. I have watched practices spend six months improving statistical accuracy while the actual failure mode sat untouched, which was that nobody was ever asked to explain a variance. Fixing the accountability is cheaper and works faster than fixing the model.

One definition to fix now, because it decides whether your forecast is comparable to your bill at all. Amortised cost spreads the up front payment for a commitment across the months it covers, rather than landing the whole charge in the month you bought it. The Foundation lists forecasting against discount adjusted, amortised usage as a measure of success. If you forecast on amortised numbers and then compare against an invoice that shows a single large purchase in March, March will look like a catastrophic miss and it will not be one.

flowchart LR
  A[Engineering estimate] --> B[Forecast model]
  B --> C[Budget approved]
  C --> D[Actual spend]
  D --> E{Variance inside threshold}
  E -->|Yes| F[Keep the model]
  E -->|No| G[Explain and revise]
  G --> B
  F --> B
  G --> H[Variance funding request]
Three capabilities, not one. Most practices build the middle box and leave the estimate feed and the variance loop unbuilt.

Four ways to build a forecast

There is no single method that fits every workload, and the Foundation says so directly. What there is, is a small set of methods with well understood blind spots, and the skill is knowing which blind spot you can afford in a given part of the estate.

Trend based forecasting projects history forward, ideally with seasonality accounted for, meaning the annual peaks around a holiday and the daily peaks when your users wake up. It is cheap, it is what every provider console gives you free, and it is blind to anything that has not happened yet. Driver based forecasting ties spend to a business quantity such as active accounts, orders processed or ad impressions, and grows the cost as the business grows the driver. It is far better at explaining a forecast to a finance audience, because the conversation becomes about customer growth rather than about instance hours, and it cannot see workloads that do not yet exist in the cloud.

Rolling forecasts re extend the horizon every period, so you always have the next twelve months in view and the model absorbs a change in the business a few weeks after it happens rather than at the next fiscal year. Static forecasts fix a number for the fiscal year and do not adjust, which is administratively simple and, in cloud, close to useless after the first quarter. Special projects are the patch for everything the other methods cannot see: a planned launch, a migration, a licensing fee, a workload sitting on a provider you do not have automation for.

My recommendation, stated plainly because this series does not do it depends: run a rolling forecast built on a driver based model for anything that scales with the business, keep trend based projection for the stable platform and shared services underneath it, and maintain an explicit special projects line that engineering owns and updates monthly. That combination is what the Framework describes at Run maturity, and it is reachable by a two person team inside a quarter.

MethodWhat it usesUse it forWhat it cannot see
Trend basedHistoric spend with seasonalityStable platform, shared services, storageLaunches, new regions, anything out of band
Driver basedA business KPI such as active accountsWorkloads that scale with demandWorkloads not yet running in the cloud
RollingContinuous re extension of the horizonAny business that changes shape mid yearNothing structural, but it costs more effort
StaticOne fiscal year number, no revisionRegulatory or grant funded fixed envelopesEvery change after the day it was signed
Special projectsEngineering estimates layered on topLaunches, migrations, licences, off cloud spendOnly what nobody told you about

Method names and blind spots follow the FinOps Framework forecasting methodologies. The use it for column is my own recommendation.

Worked example

A subscription product runs at 312,000 USD a month, amortised. Finance forecasts active accounts growing 9 percent next quarter. Roughly 100,000 of the monthly bill scales with account count, the rest is fixed platform, so the driver adds about 28,000. Engineering has one special project, a second region, estimated at 41,000 a month once live. Procurement is closing a three year commitment expected to reduce compute rate by 33,000. A rightsizing programme already approved removes another 18,000.

The forecast is 312 plus 28 plus 41 minus 33 minus 18, which is 330,000 a month. Note that only the first component came from history. Everything else came from a person, which is why the forecast meeting matters more than the forecasting tool.

Building one month of forecast, component by componentThousands of US dollars, amortised. Only the first bar comes from historic data.0100200300400312+28+41-33-18330BaselineDriverNew regionCommitmentRightsizingForecastAdds to spendReduces spendFour of the six bars depend on a commitment somebody has to keep.
If a component has no named owner, treat it as zero until it does. Optimism in a forecast is a form of debt.

How accurate does a forecast need to be?

Accurate enough that finance stops holding a contingency reserve against your number, and no more accurate than that. Chasing the last two percentage points of variance is one of the most common ways a FinOps practice burns a quarter with nothing to show for it.

The Foundation gives you a usable yardstick. The earlier version of the Forecasting capability, citing survey data from the community of practitioners, put acceptable variance at a maximum of 20 percent for a practice at Crawl maturity, 15 percent at Walk and 12 percent at Run. The current version of the page has moved to saying each organisation defines its own acceptable variance, which is more honest but less useful when you are trying to set a target this week. My advice is to adopt the older numbers as a starting ladder and replace them with your own once you have twelve months of history.

The Foundation publishes the Forecast Accuracy Rate as forecast minus actual, divided by forecast. I invert the sign in my own reporting so that overspend comes out positive, because every audience I have presented to reads a positive number as bad news and reversing that costs you thirty seconds of confusion in every meeting. Whichever convention you choose, write it at the top of the report and never change it.

Track the absolute variance, not the average. A practice that runs 15 percent over in January and 15 percent under in February has an average variance near zero and a forecasting problem in both months. Averaging is how a broken model hides.

MonthForecast, USDActual, USDVarianceMain cause
January296,000331,400plus 12.0 percentUnannounced streaming service
February318,000342,700plus 7.8 percentSame service, now in the baseline
March330,000349,100plus 5.8 percentData transfer under modelled
April344,000352,600plus 2.5 percentSpecial projects line introduced
May351,000344,900minus 1.7 percentRightsizing landed early
June349,000356,300plus 2.1 percentSeasonal peak, partly modelled

Illustrative figures from one practice over two quarters. Variance is actual minus forecast over forecast, so a positive number means overspend. The improvement came from the April change, not from a better algorithm.

Forecast variance against the maturity thresholdsAbsolute variance per month. The reference lines are the Crawl, Walk and Run levels from the FinOps Framework.05101520percentCrawl 20 percentWalk 15 percentRun 12 percent12.07.85.82.51.72.1JanFebMarAprMayJunThe step down happens in April, when an explicit special projects line was added to the model.No statistical technique changed in those six months. A process did.
Most forecast accuracy problems are intake problems wearing a statistics costume.

Turning a forecast into budgets people own

A single budget for the whole cloud bill is the same mistake as a single dashboard for the whole organisation. Nobody owns it, so nobody acts on it, and the only person who ever looks at it is the person who built it.

Set budgets at the boundary where an owner has real authority to change spend. In practice that means a budget per team, product or environment, mapped onto whatever allocation construct you settled on in Part 5, plus one deliberately separate budget for shared platform costs that no single team can influence. Mixing shared costs into team budgets produces the worst conversation in FinOps, which is an engineer being asked to explain a variance caused by a load balancer they do not control.

Write the variance process down before the first breach, not after. It needs three things: a threshold that triggers a conversation, a named person who owes an explanation, and a route to request variance funding when the overspend is legitimate. That third item is the one teams skip, and skipping it is why engineers learn to hide spend rather than declare it. If the only response to a forecast breach is criticism, you have built an incentive to stop telling you things.

Budget periods deserve a decision too. Monthly budgets catch problems fast and generate false alarms in months with a genuine seasonal peak. Quarterly budgets absorb that noise and give a runaway workload eleven more weeks to run. I use monthly budgets for alerting and quarterly budgets for accountability, so that a spike triggers a look but only a sustained quarter triggers a funding conversation.

Budget alerts on AWS, Azure and Google Cloud

All three providers give you a forecast and a budget mechanism at no charge, and all three have a limitation that will bite you if you do not know it in advance.

On AWS, Cost Explorer produces the forecast and AWS Budgets does the alerting. Cost Explorer forecasts at an 80 percent prediction interval by default, meaning the range within which the true value is expected to fall 80 percent of the time, and it will decline to produce a forecast at all if it does not have enough history, which is normal for an account with less than one full billing cycle. Through the GetCostForecast API you can request a different confidence level anywhere from 51 to 99, and the higher you set it the wider the interval gets. AWS segments the history by charge type and runs a mix of machine learning and rule based models across each one. AWS Budgets then lets you set fixed or variable amounts and filter by service, Region or tag.

On Azure, budgets live in Microsoft Cost Management and the alert Type field switches a threshold between Actual and Forecasted cost. The catch worth knowing early is that action groups, which are how you trigger anything automated rather than just an email, are supported only at subscription and resource group scopes. If your budget hierarchy sits at management group level, your automation has to live somewhere else.

On Google Cloud, budgets carry threshold rules that fire on either actual or forecasted cost, and the defaults are 50, 90 and 100 percent against actual spend. Forecasted rules are stated as a percentage of the budget projected to the end of the calendar period, so a 110 percent forecast rule warns you when you are on track to finish 10 percent over. The limitation to note is that a budget configured for a custom time range cannot trigger forecast alerts at all. Programmatic notifications go through Pub Sub, which is the route to take if you want anything beyond email.

CapabilityAWSAzureGoogle Cloud
Forecast surfaceCost ExplorerCost Analysis in Cost ManagementBudgets and Cloud Billing reports
Alert on forecastYes, via AWS BudgetsYes, Type set to ForecastedYes, forecasted threshold rules
Default thresholdsYou define themYou define them50, 90 and 100 percent on actual
Programmatic routeGetCostForecast, confidence 51 to 99Budgets API, Bicep, action groupsBudget API with Pub Sub notifications
Limitation to plan aroundNo forecast under one full billing cycleAction groups only at subscription and resource group scopeCustom time range budgets cannot alert on forecast

Verified against provider documentation in July 2026. Console behaviour and default thresholds change, so re check before you standardise on any of it.

flowchart TD
  A[Budget threshold crossed] --> B{Which kind}
  B -->|Forecast at 100 percent| C[Notify the team owner only]
  B -->|Actual at 80 percent| D[Notify owner and FinOps]
  B -->|Actual at 100 percent| E[Notify owner finance and leadership]
  C --> F[Owner responds within two working days]
  D --> F
  E --> G[Variance funding or an optimisation plan]
  F --> G
Route by severity, not by convenience. A forecast breach is a private conversation. An actual breach at 100 percent is not.

Gotcha

Forecast based alerts are unstable in the first week of a billing period, because a few days of data get projected across a whole month. A single heavy batch job on the second of the month can push a projection 40 percent over budget and fire every threshold you own.

Suppress forecast alerts for the first five to seven days of each period, or raise the forecast threshold well above 100 percent, and keep actual cost alerts running throughout. Teams that get three false alarms in the first month stop reading budget emails permanently, and winning that attention back is much harder than tuning the threshold correctly the first time.

Where forecasts go wrong

Five failure modes account for nearly everything I have seen, and only one of them is statistical.

The first is the intake gap from the opening story: a team ships something and the forecast finds out from the invoice. The fix is a standing five minute item in each team ceremony asking whether anything landing next month changes spend, which is cheap and catches most of it. The second is unallocated spend. If 20 percent of your bill has no owner, then 20 percent of your forecast has no owner either, and the Foundation is direct that comprehensive allocation is a foundational input to this capability.

The third is commitment accounting. A forecast built on amortised numbers compared against an unblended invoice will produce variance that means nothing, and a commitment expiring in month seven that nobody modelled will produce a step change that looks like a runaway workload. The fourth is currency and rate change: if you forecast in one currency and are billed in another, the exchange rate is a forecast input whether you treat it as one or not. The fifth is optimism about savings. A rightsizing programme in the forecast that has not started is a wish, and the Foundation makes usage optimisation inclusion a Run level trait precisely because getting the timing honest is difficult.

Notice that four of the five are process failures. Buying a forecasting tool solves none of them, which is worth remembering the next time a vendor demonstration makes accuracy look like a purchasing decision.

In practice

The single change that improved forecast accuracy most in the practices I have worked with was not a model. It was a monthly forecast review of forty five minutes with one engineering representative per budget, where the only question asked was what is changing next month that is not in this number.

Run it on the same day every month, keep it short enough that people attend, and record every answer as a line in the special projects component. Within two quarters, the meeting also becomes the place teams volunteer optimisation work, because they can finally see what their own decisions cost.

Run a rolling driver based forecast and alert on forecast rather than actual

Here is the recommendation for this part, without hedging. Build a rolling forecast that re extends its horizon every month rather than a static annual number. Use a driver based model for anything that scales with customers or transactions, trend based projection for the stable platform underneath, and a special projects line that engineering updates in a standing monthly review. Forecast on amortised, discount adjusted cost so the number is comparable to what you will later measure. Set budgets at the level where somebody has authority to act, keep shared platform costs in their own budget, and write the variance funding route down before you need it.

On alerting, prefer forecast based thresholds over actual based ones, with the first week of each period suppressed. An actual cost alert at 100 percent of budget tells you the money is already gone. A forecast alert at the same threshold tells you three weeks earlier, while there is still a decision worth making, and the entire point of this capability is producing decisions rather than explanations.

Start at a 20 percent variance target, tighten to 15 and then 12 as the intake process matures, and expect the improvement to come from the meeting rather than the mathematics. Part 9 picks up what happens when spend moves in a way no forecast could have anticipated, which is anomaly detection and cost alerting. The groundwork is in Part 3 on the Framework and Part 7 on reporting, and every part so far is indexed on the Cloud FinOps guide.

One thing to do this week: take last month, calculate your variance for a single team budget, and find out whether anyone was ever asked to explain it. The answer tells you which half of this part to work on first.

Cloud FinOps Series · Part 8 of 20
« Previous: Part 7  |  Guide  |  Next: Part 9 »

References

Monetary figures and monthly variance percentages are illustrative and chosen to show realistic movement. Framework wording, variance thresholds and provider behaviour were verified in July 2026 and change over time.

About The Author


Discover more from Journal of Intelligent Infrastructure – By Dr Pranay Jha

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure - By Dr Pranay Jha

Subscribe now to keep reading and get access to the full archive.

Continue reading