Here is a request that looks right and returns nothing useful:
POST https://my-aoai.openai.azure.com/openai/deployments/gpt-4o/chat/completions
{"messages":[{"role":"user","content":"ping"}]}
HTTP/1.1 404 Resource not foundThe endpoint is correct, the deployment exists, the key is valid, and it still fails. The classic Azure path wants an api-version query parameter, and without it the router does not know which contract you are speaking, so it answers with a 404 that has nothing to do with a missing resource. That one confusing error is the whole reason this part exists. Once you know what a call is actually made of, the three ways to send one stop looking interchangeable and start looking like a choice.
What a call to an Azure model is made of
Every request to an Azure model, whichever library you use, is five things stacked together. The endpoint is the host name of your resource, something like my-aoai.openai.azure.com, which pins the request to your subscription and region. The deployment name is the alias you gave a model when you deployed it, and it is not the same as the base model name. You deploy gpt-4o and can call the deployment gpt-4o, prod-chat, or anything else you typed. The auth credential is either an api-key header or an Entra bearer token, covered last part. The API surface is the path segment that decides the request and response shape. The api-version, on the classic path, tells the router which dated contract to apply.
Miss any one of the five and you get an error that rarely names the real cause. A wrong deployment name returns DeploymentNotFound. A missing api-version on the classic path returns the 404 above. A base model name where a deployment name belongs looks like a missing deployment. Hold this picture in your head and most Azure calling errors debug themselves.
Three surfaces for sending a prompt
An API surface is the request and response contract you code against, independent of which library you hold. Azure OpenAI exposes three that matter, plus the raw REST under all of them. Chat Completions is the stateless workhorse: you send the full message list every turn, you get one reply, the server remembers nothing. The Responses API is the newer stateful surface: you send one turn, the server can store the thread, and you continue by referencing the last response instead of resending history. The Assistants API was an earlier stateful attempt that is now on a deprecation path, so I do not start new builds on it.
The practical split is portability against convenience. Chat Completions is the syntax every other provider copied, so code written against it moves to another vendor with a URL and a key change. The Responses API is richer and less portable, with built-in tools, server-side state, and reasoning support that pay off when you commit to Azure. Pick the surface for how locked-in the workload is allowed to be, then pick a library.
| Surface | State | Best for | Status |
|---|---|---|---|
| Chat Completions | Stateless, you resend history | Portable code, multi-provider | GA, widely supported |
| Responses API | Optional server-side state | New Azure builds, tools, reasoning | GA on v1, Microsoft recommended |
| Assistants API | Server-side threads | Existing apps only | Deprecation path, do not start here |
| Raw REST | Whatever the surface is | Debugging, thin clients, curl | Always available |
Assistants status changes over time; confirm the current retirement date before you plan a migration off it [VERIFY].
Chat Completions, the surface you already know
Chat Completions takes a list of messages, each with a role of system, user, or assistant, and returns one assistant message. Because it keeps no memory, a five-turn conversation means you send all five turns of history on the fifth call. That is simple to reason about and simple to scale, since any replica can serve any request with no shared session. The cost is that your token bill grows with conversation length, because you pay to resend the whole history each turn.
Here is the raw call on the classic path, the one that fails without api-version. This is worth seeing once in curl, because every SDK you use is a wrapper over exactly this.
curl https://my-aoai.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2024-10-21
-H "Content-Type: application/json"
-H "api-key: $AZURE_OPENAI_API_KEY"
-d '{
"messages": [
{"role": "system", "content": "You are terse."},
{"role": "user", "content": "ping"}
]
}'Expected output: a JSON body with a choices array, where choices[0].message.content holds the reply, plus a usage block counting prompt and completion tokens.
Failure mode: drop the api-version and you get the 404 from the opener. Swap the deployment name for the base model name and you get DeploymentNotFound. Send an expired or wrong key and you get 401. None of these mention the actual field that is wrong, so check the five parts in order.
Responses API and what it changes
The Responses API takes a single input instead of a full message list, and it can store the result server side for a default of thirty days. You continue a conversation by passing the id of the previous response in a previous_response_id field, and the service reattaches the prior context for you. You stop shipping the whole transcript on every turn. That is the headline difference from Chat Completions, and it changes both your token math and where conversation state lives.
It also folds in capabilities that used to need separate surfaces. Built-in tools, function calling, image input, and reasoning items all live in one contract, which is why Microsoft steers new Azure OpenAI work here. The trade is portability and control. Server-side state is convenient until you need to run in a region that does not offer the Responses API yet, or a compliance rule forbids the service retaining conversation content. For those, you either turn off storage and chain by resending, or stay on Chat Completions.
Which SDK to use across Python, C#, and JavaScript
The v1 API changed the answer here, and for the better. Before, Azure needed its own client, the AzureOpenAI class in Python, so you could not lift OpenAI sample code without edits. On the v1 path you use the plain OpenAI() client and point its base_url at your resource with /openai/v1 appended. The same client refreshes Entra tokens automatically when you pass a token provider as the key, which used to be the one job the Azure-specific client did for you. Python, C#, JavaScript, Go, and Java all follow this shape.
Here is the keyless Responses call in Python. It builds on last part: a managed identity, the Cognitive Services OpenAI User role, and no secret in the code.
from openai import OpenAI
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
# v1 path: plain OpenAI client, Azure endpoint, no api-version.
token_provider = get_bearer_token_provider(
DefaultAzureCredential(),
"https://ai.azure.com/.default",
)
client = OpenAI(
base_url="https://my-aoai.openai.azure.com/openai/v1/",
api_key=token_provider, # token, auto refreshed by the client
)
resp = client.responses.create(
model="gpt-4o", # your deployment name
input="Give me one word.",
)
print(resp.output_text)Expected output: a single short string printed to stdout, read straight off resp.output_text with no digging through a choices array.
Failure mode: a 401 means DefaultAzureCredential found no identity, so run az login locally or attach a managed identity. A 403 means the identity lacks the OpenAI User role. A NotFoundError on the model field means the deployment name is wrong, or the model is not offered in your resource region for this surface.
How api-version works, and how v1 retires it
On the classic path, api-version is a dated string like 2024-10-21 that names a frozen contract. Microsoft shipped a new one roughly monthly, and any new feature meant editing that string across your code and config, then testing that nothing else moved with it. It was busywork that broke builds. The v1 path removes the requirement. You call /openai/v1 and get ongoing access to new features with no dated version to chase.
Preview features that are not yet generally available opt in a different way. Some want a feature-specific preview header, and some signal preview status through the path itself, for example an alpha segment in the URL, so you enable only what you want without swapping your whole API version. My rule is plain: build production against the GA v1 surface, and reach for a preview header only for a feature you have decided to depend on, knowing it can change under you.
| Mode | What you pass | When to use |
|---|---|---|
| v1 GA | Nothing, api-version is gone | Default for all new work |
| v1 preview feature | A preview header or alpha path | One feature you accept as unstable |
| Classic dated | api-version=2024-10-21 style string | Existing code not yet migrated |
The v1 base_url also accepts the services.ai.azure.com host form. Both reach the same resource.
Streaming and the latency your users feel
A non-streaming call blocks until the whole reply is generated, then hands it over. For a two sentence answer that is fine. For a long answer, the user stares at a spinner for the full generation time, even though the first words were ready almost immediately. Streaming flips this. You set stream to true and the server sends tokens as they are produced, as a sequence of server-sent events. On the Responses API you read events and act on the ones of type response.output_text.delta.
stream = client.responses.create(
model="gpt-4o",
input="Summarize the v1 API in one sentence.",
stream=True,
)
for event in stream:
if event.type == "response.output_text.delta":
print(event.delta, end="")The number that matters for felt speed is time to first token, not total time. Streaming does not make the model faster; it makes the wait visible and short. The chart below uses round figures from a long generation to show the gap.
Worked example
Take a support answer of about 400 output tokens at roughly 60 tokens per second, so near 6.7 seconds of generation. Without streaming the user waits the full 6.7 seconds, then the whole block appears. With streaming the first token lands in about 0.4 seconds and the rest scrolls in as it is produced. Same total work, same token bill. The felt wait dropped from about 6.7 seconds to under half a second, which is the difference between an app that feels stuck and one that feels alive.
Handling 429s without hammering the endpoint
The first production error most teams meet is not a 404, it is a 429, too many requests. Each deployment has a quota measured in tokens per minute and requests per minute, and once you cross it the service rejects calls until the window resets. The response carries a Retry-After header telling you how many seconds to wait. Honoring that header is the whole game. If you retry immediately on a tight loop, you extend the throttle and turn a brief spike into a sustained outage of your own making.
The OpenAI client already retries a handful of times with backoff, and you tune that with a max_retries setting rather than writing your own loop. For anything past a demo, add jitter so a fleet of replicas does not retry in lockstep, and split traffic so a batch job cannot starve your interactive chat of quota. That usually means a separate deployment, or provisioned throughput, for the latency-sensitive path. Deployment types and how quota is carved up are Part 6, and regional quota limits are Part 8. Treat 429 as normal backpressure to absorb, not an exception to crash on.
My recommendation for wiring the first call
Start every new Azure OpenAI build on the Responses API against the v1 path, with the plain OpenAI() client and an Entra token provider. You get server-side state when you want it, one contract for tools and reasoning, and no dated api-version to maintain. Keep Chat Completions for one specific reason, which is code that must run against more than one provider without a rewrite, and accept that you then manage history yourself. Keep raw REST in your pocket for debugging, because when an SDK error is vague, one curl tells you whether the problem is your request or your library.
Stream user-facing responses, leave machine-consumed ones whole, and always send the deployment name in the model field. If you also run on AWS, the same decision over there is InvokeModel against the Converse API, which I cover in the Bedrock Converse part. Next in this series is retrieval: grounding a model in your own documents with Azure AI Search, which is where these calls start returning answers about your data. For now, make one keyless Responses call today and read the reply off output_text.
References
- Azure OpenAI v1 API and the api-version lifecycle
- Use the Azure OpenAI Responses API
- Supported programming languages for the v1 API
About The Author
DrJha