A model call that worked fine in testing threw ThrottlingException: Too many requests the first afternoon real traffic hit it. Nothing about the prompt changed. What changed is that we crossed a per minute quota in the one Region we had wired up, and every extra request past that ceiling bounced. This part is about the plumbing that decides whether your request runs at all: which Region processes it, what quota it counts against, and how cross-Region inference buys you headroom without a support ticket.

Where your data runs, and why the Region matters

A Region is one geographic cluster of AWS data centers, named like us-east-1 (Northern Virginia) or eu-west-1 (Ireland). For Bedrock, three things follow from the Region you pick. First, you get a Region specific API endpoint, bedrock-runtime.us-east-1.amazonaws.com for inference calls and bedrock.us-east-1.amazonaws.com for control plane work like listing models. Second, the model catalog differs by Region. A model that is generally available in Northern Virginia may not be enabled in Frankfurt for weeks, and some models never land in every Region. Third, and this is the one that bites people, your quotas are counted per Region. The TPM ceiling you have in us-east-1 does nothing for traffic you send to us-west-2.

That last point is why so many first production incidents look like the one I opened with. You test against one Region, throughput is tiny, everything passes. Then real users arrive, you cross the per minute limit in that single Region, and there is no automatic spillover to anywhere else unless you have set it up. Picking a Region is a data residency decision, a model availability decision, and a capacity decision all at once. Treat it as all three.

One practical habit. Before you commit an application to a Region, confirm the exact model you need is available there and that the quota you will get is enough, rather than assuming parity with Northern Virginia. Northern Virginia and Oregon (us-west-2) usually get models first and carry the highest default limits. If you have a residency requirement that forces you into an EU or APAC Region, check availability early, because it changes your model shortlist.

How RPM and TPM quotas actually work

On-demand inference is metered by two quotas that both apply per foundation model, per Region, per account. Requests per minute (RPM) caps how many API calls you can make. Tokens per minute (TPM) caps the total token throughput. Both are measured across the combined traffic of the Converse, ConverseStream, InvokeModel, and InvokeModelWithResponseStream operations for that model. Whichever ceiling you hit first is the one that throttles you, and in most real workloads that is TPM, because a single request with a long document can carry thousands of tokens.

Here is the part that surprises people. TPM is not counted one token to one unit for every model. Bedrock applies a burndown multiplier to output tokens on the higher end reasoning models, so an output token can cost several units against your TPM budget. The published rates as of mid 2026 put Anthropic Claude 4.8 at a fifteen times burndown on output tokens, Claude Sonnet 5 at ten times, Anthropic models at version 4.7 and below at five times, and everything else at one to one. If you plan capacity assuming raw token counts on a heavy reasoning model, your real ceiling arrives far sooner than the number on the quota page suggests.

The numbers behind that chart are worth keeping in a table you can hand to whoever sizes your capacity.

Reading your real quota numbers

Default quotas are not published as one fixed table you can trust for your account, because AWS adjusts them by Region, payment history, and usage. So do not plan against a number you read in a blog post, including this one. Read your own account. Bedrock now surfaces its per model quotas through the Service Quotas console and API, which AWS expanded in 2026, so you can query the applied value directly instead of inferring it from throttling.

The snippet below lists the cross-Region inference profiles available to you, then pulls the applied Bedrock quotas from Service Quotas so you can see your real RPM and TPM ceilings. Run it against the Region you actually serve from.

#!/usr/bin/env bash
# List system-defined inference profiles in your Region
aws bedrock list-inference-profiles 
  --type-equals SYSTEM_DEFINED 
  --region us-east-1 
  --query "inferenceProfileSummaries[].{id:inferenceProfileId,status:status}" 
  --output table

# Pull applied Bedrock quotas (RPM and TPM) from Service Quotas
aws service-quotas list-service-quotas 
  --service-code bedrock 
  --region us-east-1 
  --query "Quotas[?contains(QuotaName, 'Claude')].{name:QuotaName,value:Value}" 
  --output table

Cross-Region inference, and what the routing does

Cross-Region inference lets one API call be served from any of several Regions instead of just the one you called. You invoke an inference profile ID rather than a bare model ID, and Bedrock picks the destination Region per request. That is how you get past a single Region ceiling without buying Provisioned Throughput or filing a ticket. The system-defined profiles are named with a geography prefix, us., eu., or apac., followed by the model ID, for example us.anthropic.claude-sonnet-5-.... You can list them with the command above and confirm their type is SYSTEM_DEFINED.

A few facts settle most of the questions people ask. There is no extra routing charge; you pay based on the Region you called the profile from. Traffic between Regions stays on the AWS network and is encrypted in transit, so it never crosses the public internet. Requests can be routed to Regions you have not manually enabled in your account, so you do not need to turn each one on. And every call is logged in CloudTrail in your source Region, with the actual processing Region recorded in additionalEventData.inferenceRegion, which you will want for audits.

Should you pick geographic or global routing

Bedrock offers two flavours of cross-Region inference and the choice is mostly about compliance versus cost. Geographic profiles keep processing inside a named boundary, US, EU, APAC, and so on, which is what you want when a regulation says data may not leave a region. Global profiles let Bedrock pick the best commercial Region anywhere in the world, which gives the highest throughput and, per AWS, roughly ten percent lower cost. The trade is that you give up the geographic guarantee.

Watch the service control policy row. If your organization uses SCPs that restrict which Regions are allowed, a geographic profile needs every destination Region in that geography allowed, and a global profile needs the account to permit aws:RequestedRegion with the value unspecified. This is the most common reason a cross-Region call fails with an access denied error in a locked down landing zone. Fix the SCP, not the code.

Sizing a quota increase that gets approved

When cross-Region headroom is still not enough, you request a quota increase through Service Quotas. AWS guidance is to request an increase on the Cross-Region InvokeModel tokens per minute quota first, after which the team reaches out and can raise the related quotas together. The requests that get approved quickly are the ones with a number derived from measured demand, not a round figure someone liked. Measure your peak minute, apply the burndown multiplier for your model, add headroom, and ask for that.

What throttling looks like, and how to absorb it

Cross-Region routing and a sized quota reduce how often you hit a ceiling, but they do not remove throttling entirely. When you cross a per minute limit, Bedrock returns a ThrottlingException with HTTP 429, and the right response is to retry with exponential backoff rather than hammer the endpoint. The AWS SDKs give you this for free through retry modes. The default standard mode retries throttled calls with backoff; adaptive mode adds client side rate limiting that slows your own sending when it detects throttling, which smooths a bursty caller. Set the mode explicitly so behaviour does not drift with SDK versions.

One distinction worth holding onto. Cross-Region inference is a throughput tool, not a resilience tool you control. It raises your effective ceiling by spreading load, but you do not choose failover behaviour and you should not treat it as your Region outage plan. If you need true resilience against a Region going down, that is a separate architecture decision: an active setup in more than one Region with your own routing in front, which I cover with the reference architectures later in the series. Keep the two goals separate in your head, because conflating them leads to designs that satisfy neither.

Where I would start on Bedrock capacity

Pick your Region for residency and model availability first, then assume you will outgrow its single Region quota and design for cross-Region routing from the start. Use a geographic profile if a rule pins your data to a boundary; use a global profile when cost and peak throughput matter more and you have no residency constraint. Size every quota request from measured peak with the burndown multiplier applied, because output tokens on reasoning models are where the real cost hides. Read your own account with Service Quotas rather than trusting a default table, and keep CloudTrail on so you can prove where each request ran.

Next in the series I get into the network layer: how to reach Bedrock privately with PrivateLink and VPC endpoints so your traffic never touches a public route. That builds directly on the routing and residency choices here. Before you read it, run the two commands above so you know your real ceilings.

Related reading: the vendor-neutral NVIDIA AI Series covers the GPU side of throughput planning if you run your own inference stack.

References

Increase throughput with cross-Region inference, Amazon Bedrock User Guide
Quotas for Amazon Bedrock, Amazon Bedrock User Guide
How tokens are counted in Amazon Bedrock, token burndown rates

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts