A team came to me with one line of doubt: "we picked SageMaker for our chatbot and the bill is 3,000 dollars a month before we shipped anything." They had spun up two GPU instances to host an open model, kept them running day and night, and were serving a few thousand messages a day. The same workload on Bedrock would have been a rounding error. They did not choose wrong because SageMaker is bad. They chose wrong because nobody told them the two services answer different questions.

Bedrock answers a simple ask: I want to call a good model right now. SageMaker AI answers a different one: I want to own the model and the machine it runs on. Get that split right and the rest of this decision is mechanical.

The split in one line

Amazon Bedrock is a managed service that gives you foundation models, which are large pretrained models like Amazon Nova, Anthropic Claude, and Meta Llama, behind one API. You send text or images, you get a response, and you never see a server. There is nothing to provision. Amazon SageMaker AI, renamed from plain SageMaker, is the platform underneath: notebooks to experiment, training jobs to fine tune or build a model, and endpoints where your model runs on instances you select and pay for by the second. One is a menu. The other is a kitchen. If your job is done by ordering from the menu, ordering is cheaper, faster, and less to maintain. If the menu does not carry your dish, you need the kitchen, and then you also own the cleanup.

What Bedrock actually gives you

Bedrock hands you a catalog of models from Amazon and third parties, reachable through two calls: InvokeModel, the low level per model call, and Converse, a single message shaped API that works the same across models so you can swap one for another without rewriting your code. On top of the raw models sit three features you would otherwise build yourself. Knowledge Bases give you retrieval augmented generation, meaning the model answers from your documents instead of guessing. Agents let a model call tools and take steps. Guardrails filter unsafe input and output. You configure all of this; you host none of it.

Billing is the part people miss. On-demand Bedrock charges per token, and when no one is calling, you pay nothing. There is no instance sitting idle burning money. That single property is why bursty traffic, internal tools, and early products belong on Bedrock. When you need guaranteed capacity and steady latency you can buy Provisioned Throughput, which reserves model units billed hourly, closer to how a dedicated server works. Most teams never need it.

What SageMaker AI actually gives you

SageMaker AI is the full lifecycle: prepare data, train or fine tune, then host. For hosting it offers four endpoint shapes. Real time endpoints keep an instance warm for low latency. Serverless inference scales to zero for spiky traffic but has a cold start. Asynchronous inference queues large or slow requests. Batch transform runs a model over a whole dataset with no live endpoint at all. SageMaker JumpStart is a catalog of pretrained and open models you can deploy to your own endpoint in a few lines, which is the closest SageMaker gets to the Bedrock experience, except the instance is yours.

The reason to accept that extra work is control. You can run any open weight model, not only the ones Bedrock lists. You can ship a custom container, pin a library version, keep everything inside your own VPC, and fine tune on data you never want to leave your account. For large training you can reach for SageMaker HyperPod, a managed cluster for long running jobs. All of that power comes with one hard rule: a real time endpoint bills every second it exists, busy or idle, until you delete it. I have seen more money wasted on forgotten endpoints than on any model choice.

Decision, side by side

Here is the comparison I actually pull up in design reviews. Read the last row first. It settles most arguments.

Cost question, with real numbers

Cost is where the choice usually gets decided, so let me put a workload through both. On-demand Nova pricing is public: Nova Lite is 0.06 dollars per million input tokens and 0.24 dollars per million output tokens. SageMaker charges by instance second; ml.g5.xlarge runs about 2.03 dollars per hour for inference. [VERIFY exact ml.g5.xlarge inference rate against the SageMaker pricing page for your Region]

The self host number only starts to make sense at scale that most teams never hit. If I hold the two instance cluster fixed at about 2,964 dollars and let Bedrock cost climb with volume, the lines cross near 26 million requests a month. Below that, Bedrock is both cheaper and simpler. Above it, dedicated capacity, whether SageMaker instances or Bedrock Provisioned Throughput, begins to pay off.

Calling each one

The code makes the difference concrete. Bedrock is one call and no infrastructure. The Converse API returns the text and a token count you can log for cost.

import boto3

brt = boto3.client('bedrock-runtime', region_name='us-east-1')

resp = brt.converse(
    modelId='amazon.nova-lite-v1:0',
    messages=[{'role': 'user',
               'content': [{'text': 'Summarize this ticket in one line.'}]}],
    inferenceConfig={'maxTokens': 200, 'temperature': 0.2},
)
print(resp['output']['message']['content'][0]['text'])
print(resp['usage'])   # inputTokens, outputTokens, totalTokens

Expected output: a one line summary, then a usage dict like {‘inputTokens’: 40, ‘outputTokens’: 12, ‘totalTokens’: 52}. Failure mode: AccessDeniedException means the model is not enabled in that Region; open the Bedrock console and turn on model access first. A wrong modelId raises ValidationException.

SageMaker JumpStart is a few lines too, but the last line creates a running, billing endpoint that is now yours to watch and delete.

from sagemaker.jumpstart.model import JumpStartModel

model = JumpStartModel(model_id='meta-textgeneration-llama-3-8b-instruct')
predictor = model.deploy(instance_type='ml.g5.xlarge',
                         initial_instance_count=1)

print(predictor.endpoint_name)
# ... use it ...
predictor.delete_endpoint()   # do not skip this

Expected output: an endpoint name after a few minutes of provisioning. Failure mode: ResourceLimitExceeded means the account has no ml.g5 quota; request an increase before you deploy. Skip the delete call and the endpoint bills every second until someone notices. The model_id string varies by JumpStart version. [VERIFY current JumpStart model_id in your Region]

Latency and cold starts

Cost is not the only axis. Response time splits the two services in a way the bill does not show. A Bedrock on-demand call goes to a warm, shared fleet, so first token latency is steady and you never wait for a machine to boot. The price you pay is that you share that fleet, and under a regional spike you can see throttling before you see slowness. A SageMaker real time endpoint is the opposite: it is your instance, warm and waiting, so latency is yours to control and does not depend on anyone else traffic, but you pay for that instance every second whether a request arrives or not. SageMaker serverless inference sits in between, scaling to zero like Bedrock, but a request after an idle period pays a cold start while the container spins up, sometimes several seconds. For a user facing chat box that pause is unacceptable, so serverless suits back office jobs that tolerate it. My rule is simple. If a human is waiting on the response, use Bedrock on-demand or a warm SageMaker endpoint, never serverless. If a queue or a batch is waiting, serverless and asynchronous inference save real money. Match the endpoint shape to who is on the other end, not to a number on a spreadsheet.

SageMaker Unified Studio blurs the line

The clean split I drew is getting fuzzier, and that is a good thing. SageMaker Unified Studio, which AWS has rolled out across 2025 and 2026, puts both services in one project. You can call a Bedrock model and train a custom one from the same workspace, without hopping between two consoles. The practical read for a design decision has not changed: you still pick per token managed access or per instance owned hosting for each workload. Unified Studio just means you no longer have to commit the whole team to one console to get there. Pick the billing model that fits the workload, not the tab you happen to have open.

How I actually choose

My default is Bedrock. Start there for any new generative AI feature. It ships faster, it costs nothing when idle, and Knowledge Bases plus Guardrails cover retrieval and safety without a platform team. I move a workload to SageMaker AI only when I can name the reason: Bedrock does not host the model I need, I must fine tune on data that cannot leave my account, I need a custom container or library pin, or I am genuinely saturating dedicated capacity around the clock. If none of those are true and someone still wants SageMaker, that is usually habit, not a requirement.

One more practical note. These are not mutually exclusive. A common pattern is Bedrock for the main chat model and a small SageMaker endpoint for one custom classifier that Bedrock cannot host. Mixing them per workload is fine, and Unified Studio makes it easier. What you should not do is default the whole stack to instances because one component needed them.

Bedrock first, SageMaker when you outgrow it

Bedrock is the menu, SageMaker AI is the kitchen. Order from the menu until the menu cannot serve you, then walk into the kitchen on purpose and clean up after yourself. Next in this series I take Bedrock pricing apart on its own, on-demand against Provisioned Throughput against Batch, so you know exactly when reserving capacity actually saves money. If you have a workload you are unsure about, tell me the request volume and the model, and I will tell you which side of the line it falls on.

References

• AWS decision guide: Amazon Bedrock or Amazon SageMaker AI
• Amazon Bedrock pricing
• Amazon SageMaker AI pricing
• SageMaker JumpStart foundation models
• Related: the vendor neutral GenAI series

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts