A team I worked with ran their first Bedrock call, got an AccessDeniedException, and spent an afternoon convinced their IAM policy was wrong. It was not. They had never requested access to the model in the Bedrock console, which is a separate step from any permission you write. That afternoon is the whole reason this series exists. AWS gives you a lot of generative AI surface, and almost none of it fails in the place you expect. Before you write a line of code, you need the map.

Three layers, not one product

When people say they are "using AWS for AI" they usually mean one of three very different things, and the confusion costs them weeks. Here is the split I draw on a whiteboard every time.

The top layer is Amazon Bedrock, a fully managed service that hands you a single API in front of foundation models from many providers. A foundation model is a large model trained on broad data that you adapt to your task instead of training from scratch. You do not manage servers. You call an endpoint and pay per token, where a token is roughly three quarters of a word. This is where most teams should start and where most should stay.

The middle layer is Amazon SageMaker AI, the platform for building, training and hosting your own models. AWS renamed the old Amazon SageMaker to SageMaker AI in December 2024 when it folded the classic studio into a larger data and AI environment. You reach for this layer when a hosted model is not enough and you need to train, fine tune at depth, or serve an open weight model on hardware you control.

The bottom layer is silicon. AWS designs its own AI chips, Trainium for training and Inferentia for inference, and also rents NVIDIA GPUs. You rarely pick chips directly when you use Bedrock, because AWS runs that for you. You pick them constantly once you drop into SageMaker. Keep these three layers separate in your head and the rest of this series clicks into place.

Amazon Bedrock, the managed front door

Bedrock is the service you will touch first, so it is worth being precise. It is a fully managed, serverless API. You enable access to a model, send it text or images, and get a response. AWS handles the GPUs, the scaling and the patching. Your data is not used to train the base models, and traffic can be kept inside your virtual private cloud with PrivateLink, which is a private network path that never crosses the public internet. Those two facts are why regulated teams start here.

The single most useful thing to learn about Bedrock is the Converse API. Early Bedrock made you format a different request body for every model vendor. Converse replaced that with one schema that works across models and supports tool use, system prompts, multi turn chat and document inputs without custom parsing. Learn Converse once and you can swap the underlying model by changing a single string. I have moved a production workload from one vendor to another in an afternoon because of that.

Model catalog and Amazon Nova

Bedrock is a catalog, not a single model. As of mid 2026 it exposes models from Anthropic, Meta, Mistral, Cohere, AI21, DeepSeek, Stability AI and others, plus a large marketplace of specialist models. The set changes often, so treat any list as a snapshot and check the console. The point that matters for design is that you choose a model per task and can run several behind one application.

Amazon Nova is AWS’s own family of foundation models, built to be cheap and fast. The text tier runs from Nova Micro, the smallest and cheapest, up through Nova Lite and Nova Pro to Nova Premier for the hardest reasoning. There are also Nova Canvas for images and Nova Reel for short video. Nova is not always the smartest option, but it is usually the cheapest credible one, which makes it the right default to test against before you pay for a premium model.

Here is the text lineup with the launch on demand prices. Treat the numbers as a starting shape and confirm them in the console, since AWS adjusts pricing and adds tiers.

Bedrock or SageMaker AI

This is the fork that decides your operational life, so I will be blunt. Start on Bedrock. Move to SageMaker AI only when you hit a wall you can name. SageMaker AI is the machine learning platform: notebooks, training jobs, model hosting, and features like JumpStart, which is a library of ready to deploy models, and HyperPod, which is managed infrastructure for large distributed training. It gives you total control and hands you the whole operational bill in return.

The walls that justify the move are specific. You need an open weight model that Bedrock does not host. You need deep fine tuning or continued pre training beyond what Bedrock offers. You need an endpoint pinned to particular hardware for latency or compliance. Short of one of those, a managed Bedrock endpoint will be cheaper to run and far cheaper to babysit. I have watched teams stand up SageMaker endpoints for a task Bedrock already served, then pay for idle GPUs every night.

The silicon underneath

You do not have to care about chips to use Bedrock. You do have to care the moment you host your own model, so meet the players now. Trainium is the AWS chip built for training, on Trn1 and the newer Trn2 instances, with Trainium3 announced at re:Invent 2025. Inferentia is the inference chip, on Inf2 instances. Both are AWS designed and priced to undercut GPUs for the workloads they fit. NVIDIA GPUs remain the default when you need the broadest framework support or a kernel that only exists for CUDA.

The trade is real and it is not free. AWS silicon can cut cost per token meaningfully, but it needs the Neuron SDK, which is the software layer that compiles your model for Trainium and Inferentia. Not every model or custom kernel compiles cleanly, and porting time is a cost people forget. My rule is simple. Prototype on GPU where everything just runs, then port the steady state, high volume path to Trainium or Inferentia once the model is settled and the savings are worth the engineering.

What a real bill looks like

Layers are abstract, so price a real workload. Take a support assistant handling 200,000 requests a month. Each request averages 2,000 input tokens of context and 400 output tokens. That is 400 million input tokens and 80 million output tokens a month. Run it on Nova Lite and on Nova Pro and the difference is stark.

Pieces that turn a model into an application

A raw model call is not a product. Bedrock ships four managed pieces that close that gap, and each gets its own part later in this series. Knowledge Bases handle retrieval augmented generation, the pattern where you fetch relevant documents and feed them to the model so it answers from your data instead of guessing. Agents let a model plan steps and call tools. Guardrails filter inputs and outputs for safety and topic rules. And AgentCore, which reached general availability in October 2025, runs production agents with long session windows and enterprise controls.

You do not need any of these on day one. You need to know they exist so you do not rebuild them by hand. The most expensive mistake I see is a team writing their own retrieval and safety plumbing, badly, before they have checked what Bedrock already manages.

Where Amazon Q fits

One more name, because it confuses people. Amazon Q is not a model you call. It is a finished assistant AWS sells in two flavours. Amazon Q Developer helps you write, debug and understand code and AWS itself. Amazon Q Business answers questions over your company data with connectors to common systems. If your goal is to buy a working assistant rather than build one, Q is the shortcut. If you are building a product, you will live in Bedrock, not Q. Keep the two apart.

A first call that works

Enough theory. Here is the smallest useful thing: a Converse call to Nova Lite from Python. Two steps trip up beginners, so I have called them out below the code. First enable model access in the Bedrock console. Second make sure your credentials carry the bedrock:InvokeModel permission.

# pip install boto3, then set your AWS credentials first
import boto3

client = boto3.client("bedrock-runtime", region_name="us-east-1")

response = client.converse(
    modelId="amazon.nova-lite-v1:0",
    messages=[
        {"role": "user", "content": [{"text": "Name the three layers of the AWS GenAI stack."}]}
    ],
    inferenceConfig={"maxTokens": 200, "temperature": 0.2},
)

print(response["output"]["message"]["content"][0]["text"])
print("tokens in/out:", response["usage"]["inputTokens"], response["usage"]["outputTokens"])

Expected output, roughly:

Amazon Bedrock for managed models, Amazon SageMaker AI for
building your own, and custom silicon (Trainium and Inferentia)
underneath.
tokens in/out: 16 38

Failure mode: if you see AccessDeniedException with a message about model access, you skipped the console step, not a policy. Enable Nova Lite under model access in the Bedrock console for us-east-1 and run it again. If the region has no Nova access, switch region or request it.

Regions and quotas will bite you first

Two operational facts surprise almost every team, so learn them now rather than during an outage. First, model availability is per region. A model you see in US East may not exist in the region your data has to stay in, and Bedrock will simply refuse the call. I always confirm the model, the exact model ID and the region together before I design anything, because "it works on my laptop" usually means "it works in us-east-1" and nowhere else.

Second, you start with quotas, not unlimited capacity. Bedrock caps how many requests and tokens per minute you can push at a given model, and the default is lower than a busy application needs. The fix is cross region inference, which spreads calls across regions to lift throughput, and a quota increase request when that is not enough. Neither is instant. A launch plan that assumes day one throughput without checking the quota is a launch plan that throttles under its first real load. Read the quota page before you promise a date.

These two points sound like housekeeping. They are the difference between a demo and a service. The parts ahead on regions, quotas and private networking go deep on both, but even at this map stage, write down the region and the quota next to every model you plan to use.

My take

If you remember one thing, remember the three layers. Bedrock for managed models, SageMaker AI to build your own, silicon underneath. Start at the top and only go down when a named wall stops you. Default to the Converse API so you are never locked to one vendor, and test Nova before you pay for anything pricier. Most teams that struggle on AWS are not short on options. They picked the wrong layer for the job and paid for it in idle GPUs and lock in. Get the map right and the next twenty nine parts are detail, not surprise.

Next I will open up Amazon Bedrock itself and the shared responsibility model, so you know exactly what AWS secures and what stays yours. If you want the vendor neutral foundations first, my Generative AI guide covers tokens, prompting and hallucination without the AWS lens, and the NVIDIA AI guide goes deep on the GPU side that AWS silicon competes with.

References

Supported foundation models in Amazon Bedrock, AWS docs
Converse API reference, AWS docs
Amazon Bedrock AgentCore general availability, AWS blog
What is Amazon SageMaker AI, AWS docs
Amazon EC2 Trn2 instances with Trainium2, AWS

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts