The API key sitting in your app configuration is a bearer credential with no expiry, no name attached, and full data-plane access to your model. Anyone who reads it can call your deployment as you, from anywhere, until you notice and rotate it. That one string is the first thing I take out of a production Azure OpenAI design, and this part is how I replace it.
Where identity sits in an Azure OpenAI call
Microsoft Entra ID, renamed from Azure Active Directory, is the identity service that issues and checks tokens for everything in Azure. An API key, by contrast, is a symmetric secret the resource itself mints, with no link to any person or workload. Every call to your endpoint carries one of these two credentials in a header. Key auth sends api-key with the raw secret. Entra auth sends Authorization with a Bearer token. The service validates whichever arrives before it routes the prompt to the model.
The difference is who the credential represents. A key represents nothing, so a leaked key is anonymous access that logs show as the resource calling itself. An Entra token represents a named identity, a user or a workload, and it carries role assignments that decide what that identity may do. When something goes wrong at 2am, one of these tells you who did it and the other does not. That is the whole argument in one sentence, and everything below is how you act on it.
Two ways to prove who you are
Every Azure OpenAI resource ships with two API keys, named key1 and key2. Two, so you can rotate one while the other keeps serving traffic. They are long strings that never expire on their own. Convenient on day one, a liability by month three, because they get pasted into notebooks, checked into a repo by accident, and shared in chat threads that outlive the project.
An Entra token is the opposite in every way that matters for security. It is short-lived, with a default lifetime somewhere between 60 and 90 minutes, after which it is useless and the client silently fetches another. It is tied to a specific identity, and it carries the role-based access control decisions attached to that identity. Your code asks Entra for a token scoped to cognitiveservices.azure.com, presents it, and the gateway decides based on the role, not on possession of a secret.
The security payoff shows up the moment a credential leaks. A stolen key works until a human notices and rotates it, which in practice can be hours or days. A stolen token stops working at its expiry, and if you turn on Continuous Access Evaluation, Entra can revoke it far sooner when it sees a risky sign-in. The chart below is the same leak, three different exposure windows.
Which role grants what
Entra auth is only half the job. The token gets you in the door; the role decides which rooms you enter. This trips people up because being Owner or Contributor on the subscription does not, by itself, let you call the model. Those are management-plane roles. They let you create and delete the resource, read its keys, and change its settings, but the data plane, the inference API, is gated by a separate set of Cognitive Services roles you assign explicitly.
For an application that only needs to send prompts and get completions, assign Cognitive Services OpenAI User. That is the least-privilege role for inference. Reserve Cognitive Services OpenAI Contributor for identities that also create deployments, upload fine-tuning files, or manage Knowledge Base ingestion through the data plane. Keep the broad Cognitive Services Contributor role for platform automation that provisions resources, and do not attach it to a chat app.
| Role | Plane | Grants | Assign to |
|---|---|---|---|
| Cognitive Services OpenAI User | Data | Call inference, read deployments | Your app identity |
| Cognitive Services OpenAI Contributor | Data | Inference plus create fine-tunes, upload files | Build and training jobs |
| Cognitive Services Contributor | Management | Create, delete, configure the resource | Provisioning automation |
| Owner | Management | Everything above plus role assignment | A named human, sparingly |
Management-plane roles do not grant data-plane access. An Owner who never assigned themselves OpenAI User will get a 403 on the inference API.
Managed identities, system-assigned or user-assigned?
A managed identity is an Entra identity that Azure creates and rotates for you, attached to a compute resource so your code never handles a secret. Your app running on a VM, a container app, or a function gets a token from the local identity endpoint, and you assign that identity the OpenAI User role. No client secret, no certificate, nothing to leak. This is the piece that makes keyless real rather than a slogan.
There are two kinds. A system-assigned managed identity is created with the resource and deleted with it, one to one, and it cannot be shared. A user-assigned managed identity is a standalone object with its own lifecycle that you can attach to many resources at once. For a single app, system-assigned is simplest. For a fleet, or when you want to grant the OpenAI User role once and reuse it across ten services, user-assigned saves you ten separate role assignments and the drift that comes with them.
One caveat worth flagging before you standardize on user-assigned everywhere. Some service-to-service flows, notably Azure OpenAI On Your Data reaching into Azure AI Search, have historically supported only the system-assigned identity on the Azure OpenAI resource. Check the current On Your Data docs for the flow you are wiring before you assume the user-assigned one will be accepted [VERIFY]. For the plain inference path in this part, either works, and DefaultAzureCredential picks up whichever is present.
Worked example
A platform team runs one chat API across three regions, so three container apps. With system-assigned identities that is three separate identities and three OpenAI User role assignments to keep in sync. Create one user-assigned identity instead, attach it to all three apps, assign OpenAI User once on the resource, and a fourth region later is one attach, zero new role assignments. When they retire a region, the identity and its role survive for the next one.
Turn off key access without locking yourself out
Assigning roles does not stop the keys from working. Both credential types stay valid side by side until you explicitly disable local authentication. You do that by setting the resource property disableLocalAuth to true, either in your ARM or Bicep template, through the Azure Policy named Foundry Tools resources should have key access disabled, or with a single CLI call. After it takes effect, Entra is the only way in, and every anonymous key is dead.
az resource update
--ids /subscriptions/<SUB>/resourceGroups/<RG>/providers/Microsoft.CognitiveServices/accounts/<NAME>
--set properties.disableLocalAuth=trueThe order matters more than the command. Assign the OpenAI User role to your app identity first, confirm a real Entra call succeeds, and only then disable local auth. Flip that order and you take down production while you scramble to fix a role assignment. Keep a break-glass identity, a named admin with the role, out of band from the app so a bad deploy cannot orphan you.
How encryption at rest works here
Identity controls who calls the model. Encryption controls who can read the data the service stores. The good news is that you start protected. Every Azure OpenAI resource encrypts its data at rest by default with Microsoft-managed keys, using AES-256, with the key handling done entirely by the platform. There is no switch to flip and no cost line for it. Uploaded files, fine-tuning training data, and stored artifacts all land encrypted. In transit, calls to the endpoint use TLS 1.2 or higher.
For most workloads that is the correct and final answer. The keys are rotated and protected by Microsoft, and an auditor asking whether data is encrypted at rest gets a clean yes. You only move off the default when a specific control demands that your organization, not Microsoft, hold the key that can make the data readable, and can revoke it on demand. That control has a name and a real cost, which is the next section.
When customer-managed keys earn their place
Customer-managed keys, CMK, put your own key in front of the platform key. The service still encrypts data with a fast data-encryption key, then wraps that key with your key held in Azure Key Vault or Azure Key Vault Managed HSM. This is envelope encryption. The lever it gives you is revocation. Cut access to your key and the wrapped data-encryption key cannot be unwrapped, so the stored data becomes unreadable, even to the service. That is the whole reason CMK exists, and it is worth exactly as much as your requirement to hold that lever.
The setup has hard requirements. The key vault must have Soft Delete and Purge Protection enabled, so a deleted key cannot be permanently lost and take your data with it. The resource needs a managed identity granted permission to wrap and unwrap your key, through the Key Vault Crypto Service Encryption User role under RBAC or the wrapKey and unwrapKey permissions under a vault access policy [VERIFY exact role name]. Enabling CMK also switches on a system-assigned managed identity on the resource and registers it with Entra so it can reach the vault.
Now the cost, because it is the part teams skip until the invoice arrives. CMK backed by a standard Key Vault key adds only a few dollars a month in key operations. CMK backed by Managed HSM, which some regulated mandates require for a single-tenant hardware boundary, is billed per hour for the HSM pool and lands near 2,300 dollars a month before you have encrypted a single byte [VERIFY current Managed HSM rate]. Same revocation lever, two very different bills.
| Option | Who holds the key | You can revoke | Setup and cost |
|---|---|---|---|
| Microsoft-managed keys | Microsoft | No | None, included |
| CMK in Key Vault | You | Yes | Vault plus role, a few dollars |
| CMK in Managed HSM | You, single-tenant HSM | Yes | HSM pool, hundreds to thousands |
Revocation is the only column that justifies leaving the default. If you cannot name a rule that needs it, you do not need it.
A keyless call from a Python service
Here is the whole pattern in one file. The azure-identity library builds a token provider from the ambient identity, and the OpenAI client asks it for a fresh token whenever the old one nears expiry. There is no key in the code, no key in an environment variable, no key anywhere. DefaultAzureCredential resolves your identity from the environment, a managed identity in Azure or your az login locally, and if you attached a user-assigned identity you point at it with the AZURE_CLIENT_ID variable.
from azure.identity import DefaultAzureCredential, get_bearer_token_provider
from openai import AzureOpenAI
# No key anywhere. The workload identity supplies a short-lived token.
token_provider = get_bearer_token_provider(
DefaultAzureCredential(),
"https://cognitiveservices.azure.com/.default",
)
client = AzureOpenAI(
azure_endpoint="https://my-aoai.openai.azure.com",
azure_ad_token_provider=token_provider,
api_version="2024-10-21",
)
resp = client.chat.completions.create(
model="gpt-4o", # your deployment name, not the base model
messages=[{"role": "user", "content": "ping"}],
)
print(resp.choices[0].message.content)Expected output: the model reply printed to stdout, a short string such as pong or a greeting. If you see text, the identity, the role, and the endpoint all lined up.
Failure mode: a 401 means no valid token was obtained, so DefaultAzureCredential found no identity, or you are local and never ran az login. A 403 means the token is valid but the identity lacks Cognitive Services OpenAI User on this resource, which no code change fixes, only a role assignment does. A DeploymentNotFound means you passed the base model name instead of your deployment name in the model field.
Managed identity first, keys behind a break-glass
My recommendation is one line. Give every workload a managed identity, assign it Cognitive Services OpenAI User, run the keyless pattern above, and disable local auth only after a real Entra call has succeeded. Keep one break-glass admin identity out of band so a bad deploy cannot lock you out. On encryption, stay on Microsoft-managed keys unless a written policy names key revocation, use standard Key Vault CMK when it does, and reserve Managed HSM for the rare mandate that spells out a single-tenant hardware boundary and its bill.
You now have who can call the model and who can read its data. The next question is what you actually send, which is the request shape itself, covered in Part 11 on calling models with REST, the SDKs, and the Responses API. If you also run AWS, the parallel key-control story lives in the Bedrock data residency and KMS part. Before you move on, assign OpenAI User to one app identity and make a keyless call today.
References
- Disable local authentication in Foundry Tools
- Use keyless connections with Azure OpenAI
- Customer-managed keys for Microsoft Foundry
About The Author
DrJha