The line item that got my attention last quarter was not a model I meant to run. It was a fine tuned deployment somebody spun up for a demo in March, left deployed, and forgot. It billed an hourly hosting charge every single day it sat idle, calling nothing, serving no one, until the invoice made a finance analyst ask what it was. Nobody had done anything wrong except leave a switch on. That is what GenAI cost governance on Azure actually protects you from, and almost none of it is about being clever with prompts.

I will start where the money starts, which is the token, then work up to reservations, discounts, tagging, and the alerts that page you before the invoice does. If you have read Part 25 on observability, treat this as its financial twin. Observability tells you what your models did. Cost governance tells you what that cost and who pays.

Three ways Azure charges you for a token

Every Azure OpenAI charge is built from a few facts at once: how many tokens moved, which model, and which deployment type served the request. A token is the subword unit a model reads and writes; roughly 1,000 tokens is about 750 words. Input and output tokens are metered on separate rates, and output is usually the pricier side. That is the whole billing atom. Everything else is a wrapper around it.

The deployment type is what changes the shape of the bill. Standard (pay as you go) charges per token, so a quiet hour costs nothing. Provisioned Throughput reserves a fixed slice of model capacity measured in PTUs, short for Provisioned Throughput Units, and bills a flat hourly rate for that slice whether you send one token or a billion. Batch runs your requests asynchronously within a 24 hour window and takes 50% off the Standard token rate for the privilege of waiting. Same model, three completely different cost curves.

When does a PTU reservation beat pay as you go?

This is the decision people get wrong most often, in both directions. A Provisioned Throughput reservation is a monthly or annual commitment on a pool of PTUs at a discounted flat rate. Because the flat cost does not move with traffic, it only wins when you keep the pool busy. The lever is utilization, not raw volume. A pool you use at 30% is a pool you are overpaying for.

The math is a straight line crossing a flat one. Pay as you go rises with token volume. A reserved PTU pool is flat until you hit its throughput ceiling. Where they cross is your break even. Below it, Standard is cheaper and you keep the flexibility. Above it, and only if utilization stays high, the reservation pays for itself and buys you predictable latency as a bonus. Do not commit off a spreadsheet guess. Run 30 to 60 days of real Standard telemetry first, then model the crossover against actual traffic.

Here is a small calculator I keep in the repo so nobody argues from memory. It prints the crossover and checks three volumes.

# ptu_breakeven.py  PAYG vs reserved PTU pool
# Rates are ILLUSTRATIVE. Replace with Azure OpenAI pricing page numbers.

payg_per_million = 12.00      # blended input+output USD per 1M tokens
ptu_monthly_flat = 6000.00    # reserved PTU pool, USD per month

def monthly_cost(tokens_million):
    payg = tokens_million * payg_per_million
    ptu  = ptu_monthly_flat            # flat within pool capacity
    return payg, ptu

breakeven = ptu_monthly_flat / payg_per_million
print('break-even at', round(breakeven), 'million tokens per month')

for v in (200, 500, 800):
    payg, ptu = monthly_cost(v)
    cheaper = 'PTU' if ptu < payg else 'PAYG'
    print(v, 'M  PAYG $', payg, ' PTU $', ptu, ' ->', cheaper)

Expected output: break-even at 500 million tokens per month; 200M favours PAYG, 500M ties (script prints PAYG on a tie), 800M favours PTU. Failure mode: feed a volume above the pool ceiling and the flat $6,000 is fiction, because real traffic spills to Standard at list price. Always cap the model at measured pool throughput before trusting the crossover.

Batch and caching, the discounts you barely have to ask for

Two of the biggest savings on Azure OpenAI need no negotiation and no commitment. The Batch deployment takes 50% off the Standard token rate in exchange for a completion window of up to 24 hours. If a workload is offline, evaluation runs, nightly summarisation, bulk classification, then Batch is close to free money. Prompt caching is the second one. When a request repeats a prompt prefix Azure has seen recently, above a minimum length near 1,024 tokens, the cached portion of the input bills at a fraction of the normal input rate. You get it automatically on supported models; you just have to structure prompts so the stable part comes first.

They stack. A batch job over prompts that share a long system preamble gets the Batch discount on the whole request and the cache discount on the repeated prefix. For high reuse input that can land around 75% off the list input rate. The one design rule that unlocks it: keep the fixed instructions and context at the front of the prompt and the variable user text at the end, so the cacheable prefix stays identical across calls.

Read the bill in Cost Management

Cost Management is the Azure service where spend actually lands, and Azure OpenAI hides in it in a way that trips up first timers. If you filter Cost analysis by service looking for Azure OpenAI, you will not find it. Its usage rolls up under the broader Cognitive Services classification. The trick is to filter by Service tier and pick Azure OpenAI, or scope Cost analysis to the resource group where the Foundry resource lives and group by Meter. Group by Meter shows you the per model input and output lines. Group by Resource shows you which deployment generated them.

One scoping detail matters for mixed catalogs. Models sold by Azure, which includes Azure OpenAI, bill directly through Microsoft and show as meters under your Foundry resource. Models served through Azure Marketplace show under Global resources instead, so if you run partner models, scope to the resource group, not just the resource, or you will miss half the spend. Reporting has ingestion lag too, so reconcile against trend windows and the invoice, never minute by minute.

Tag everything, or chargeback is guesswork

Chargeback means allocating a shared bill back to the team or workload that ran it up. Showback is the softer version where you report the split without moving budget. Neither works if you cannot tell whose tokens these were. Foundry helps here: every Foundry project is automatically stamped with a project tag on its underlying usage, and in Cost analysis you filter by that project tag to split spend per project with no manual tagging. That attribution currently covers models sold by Azure, including Azure OpenAI, and does not yet reach Marketplace models, so plan your own tags for those.

For anything the automatic project tag does not cover, apply your own consistent tags at deployment time: team, environment, cost center. Tags that are added by hand after the fact are the ones that go missing, and a tag filter that returns incomplete results is almost always a tag that was never applied at creation. Bake tagging into the deployment pipeline, not the runbook. Then export cost data on a daily schedule to a storage account so finance can slice it in Power BI without touching the portal.

Budgets, alerts, and the hard cap Azure will not give you

A budget in Cost Management is a threshold with alerts, not a stop valve. You set a monthly amount on a subscription or resource group, add filters to narrow it to your GenAI resources, and get notified at percentage thresholds. Here is the part people assume wrong: OpenAI direct offers a hard limit that blocks calls once you cross it, but Azure OpenAI does not. A budget alert emails you; it does not stop the spend. If you want an actual cutoff you have to wire an action group to automation that disables the deployment, and you build that yourself.

So set two thresholds, a warning around 60% and a critical around 90%, and split them to different recipients so the critical one is not lost in the noise. Recalibrate after a full billing trend so the alerts are not crying wolf on day three of every month. And give finance read only cost visibility with a least privilege custom role rather than handing out broader access. The Microsoft docs publish a working starting point.

{
  "Name": "Foundry Cost Reader",
  "IsCustom": true,
  "Description": "Can see cost metrics in Foundry",
  "Actions": [
    "Microsoft.Consumption/*/read",
    "Microsoft.CostManagement/*/read",
    "Microsoft.Resources/subscriptions/read",
    "Microsoft.CognitiveServices/accounts/AIServices/usage/read"
  ],
  "AssignableScopes": [
    "/subscriptions/<subscriptionId>/resourceGroups/<rg>/providers/Microsoft.CognitiveServices/accounts/<foundryResource>"
  ]
}

Expected result: a role that grants cost read access without touching resources. Failure mode: this role alone does not grant Foundry resource visibility, so pair it with the Foundry User role when the person also needs to see usage context. Validate in a nonproduction tenant first, since resource provider actions can change.

The AWS side of this series covers the same discipline from the other platform, and the parallels are close enough to be useful if you run both clouds. See Bedrock Cost Governance and FinOps for the Amazon equivalent of PTUs and Cost Explorer. The next Azure part, on responsible AI and governance, builds directly on the tagging and role work here, and the guide tracks where each piece fits.

Batch and caching first, PTU only after the telemetry

If I inherit a running Azure OpenAI bill, the order is always the same. Move offline jobs to Batch and reorder prompts to cache, because those cut spend this week with no commitment. Turn on tagging and a budget with two alert thresholds so you stop flying blind. Sweep for idle fine tuned deployments, because one of them is usually the mystery line. Only then, with 30 to 60 days of real utilization in hand, do I model a PTU reservation, and I commit only if the pool stays busy above the crossover. Predictable cost is a side effect of high utilization, never a reason to reserve capacity you will not use.

Pick one workload this week, find its true cost per 1M tokens in Cost analysis grouped by meter, and decide whether Batch or caching applies before you touch anything else.

References

• Plan and manage costs for Microsoft Foundry, Microsoft Learn
• Provisioned throughput for Foundry Models, Microsoft Learn
• Azure OpenAI Service pricing, Microsoft Azure

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts