You can ship a working agent on Azure without importing a single orchestration framework. No LangChain, no home-grown state machine parking conversation history in Redis. The Agent Service keeps the thread, runs the model, invokes the tool, and hands you back the result. I moved a support-triage prototype off a hand-written loop last quarter and deleted about two hundred lines of glue code the same afternoon. What follows is what that runtime is, how its parts fit, and where it will surprise you once real traffic arrives.

What an agent on Azure actually is

A chat completion is stateless. You send a list of messages, you get one reply, and the service forgets everything the instant it responds. Every follow-up means resending the whole history yourself and tracking it in your own code. An agent flips that. You register the model, the instructions, and the tools once, and the service remembers the conversation and decides when to reach for a tool.

So an agent here is three things bound together: a deployed model such as gpt-4o, a set of standing instructions that describe its job, and an optional list of tools it may call. Foundry stores that definition and gives it an ID. From then on you talk to the agent, not to the raw model endpoint. The rename matters if you are reading older material. What launched as Azure AI Agent Service is now Foundry Agent Service, part of Azure AI Foundry. The general availability API version is 2025-05-01, with a 2025-05-15-preview track for newer features. Same runtime, new label, plus a split between a Classic experience and a newer runtime that matters when you size storage later on.

Agents, threads, runs, and tools

Four nouns carry the whole model. An agent is the definition. A thread is one conversation, a container that holds every message in order. A message is a single turn, from the user or the assistant. A run is one execution of the agent against a thread. During a run the service reads the thread, calls the model, executes any tool the model asks for, writes the reply back to the thread, and reports a status such as completed or failed.

That separation is the useful part. One agent definition can serve thousands of threads at once, one per user session, with no state leaking between them. You create the agent once at deploy time and open a fresh thread per conversation. Memory lives in the thread, not in the agent. Keep that distinction straight and most of the confusing questions answer themselves. Where does history go? The thread. What do I reuse across users? The agent. What do I create per chat? A thread, then a run on it.

Basic setup or standard setup?

Two setup modes decide where that thread actually gets stored, and the choice has real consequences. Basic setup uses Microsoft-managed, multitenant storage. You create a Foundry project, deploy a model, and start building agents in minutes. Conversation history sits in Microsoft resources. For a prototype or an internal demo this is the right call, and I reach for it first every time.

Standard setup uses customer-managed, single-tenant resources. Agent state lands in your own Azure Cosmos DB for NoSQL, uploaded files in your own Azure Storage account, and vector stores in your own Azure AI Search resource. Nothing about the conversation leaves your tenant. This is what you want the moment real customer data is in play, or when a compliance reviewer asks where the transcripts live. The trade is provisioning effort: the docs budget roughly thirty to forty-five minutes for a full standard setup, and that is if the role assignments go right the first time.

The bridge between the two is a sub-resource called a capability host. There is one on the account and one on the project. The project capability host names the storage, Cosmos, and search connections the project uses. It reads like plumbing, and it is, but one property is worth knowing before you touch anything: you cannot update a capability host after it is set. Change your mind about which Cosmos account backs a project and the fix is to delete and recreate the project. Decide the storage story up front. There is also a Classic versus New runtime split under the hood, and the two use different Cosmos containers, so mixing guides written for each is a good way to chase a phantom error.

Wiring your first agent in Python

Here is the whole loop in one file: create an agent, open a thread, drop a message, run it, and read the reply. It uses the projects client and DefaultAzureCredential, so no API key sits in the code.

from azure.ai.projects import AIProjectClient
from azure.identity import DefaultAzureCredential

project = AIProjectClient(
    endpoint='https://YOUR-RESOURCE.services.ai.azure.com/api/projects/YOUR-PROJECT',
    credential=DefaultAzureCredential(),
)

# 1. Create the agent: a model, a name, and instructions.
agent = project.agents.create_agent(
    model='gpt-4o',
    name='refund-triage',
    instructions='You classify refund requests. Reply with one word: approve, deny, or escalate.',
)

# 2. Open a thread. This is the conversation container.
thread = project.agents.threads.create()

# 3. Add a user message, then run the agent over the thread.
project.agents.messages.create(
    thread_id=thread.id, role='user',
    content='Customer wants a refund 40 days after purchase. Policy is 30 days.',
)
run = project.agents.runs.create_and_process(
    thread_id=thread.id, agent_id=agent.id,
)

print('run status:', run.status)
for m in project.agents.messages.list(thread_id=thread.id):
    print(m.role, m.text_messages[-1].text.value)

Expected output

run status: completed
user Customer wants a refund 40 days after purchase. Policy is 30 days.
assistant escalate

Failure mode: if DefaultAzureCredential resolves to an identity without the Foundry User role on the project, create_agent returns AuthorizationFailed before a single token is spent. Assign the role, wait a minute for it to propagate, then rerun. The exact sub-client method names have moved between SDK releases, so pin your azure-ai-projects version and check the changelog if a call goes missing. [VERIFY: azure-ai-projects method paths against your installed version]

Tools an agent can call

A model on its own can only produce text. Tools are how the agent reaches past that: run code, search documents, hit the live web, call your own API. You attach tools to the agent definition, and the model decides during a run whether to use them. These are the ones I reach for most.

Connected agents instead of an orchestrator

One agent is often enough. When it is not, the temptation is to bolt on an external framework to route between specialists. Foundry offers connected agents instead: you build task-specific agents and attach them to a primary agent, which delegates to them during a run. No separate orchestrator process, no message bus you have to run yourself. A triage agent can call a billing agent and a returns agent, collect their answers, and reply once.

My take: connected agents earn their keep when each sub-agent maps to a team or a data boundary that a single set of instructions cannot hold. If the split is only cosmetic, one agent with good instructions and a couple of tools is cheaper to run and far easier to debug, because you have one thread to read instead of five. This is the same trade AWS draws with supervisor and collaborator agents on Bedrock, which I covered in the AWS series. The names differ; the design question is identical.

What standard setup provisions, and what it costs

Standard setup provisions three of your resources: Storage for files, Azure AI Search for vectors, and Cosmos DB for NoSQL for threads and agent metadata. Cosmos is where sizing bites. The account must carry at least 3000 RU per second, and the setup lays down containers at 1000 RU per second each. Classic uses three containers, the new runtime uses two, and both keep the 3000 RU per second floor. Provisioned and serverless modes both qualify.

On billing, the agent object itself is free. Microsoft does not charge you for creating or running a native agent. You pay for what the agent consumes: model tokens through Foundry Models, per-transaction tool charges such as Bing grounding, and the underlying Cosmos, Storage, and Search resources you brought. That last line is the one teams forget. A quiet agent still runs a Cosmos account at 3000 RU per second, and that meter never stops.

Failure modes I keep seeing

Almost every standard-setup failure I have hit is a role assignment or a throughput shortfall, not a code bug. The agent runs under the project managed identity, and if that identity is missing one role, you get a specific error rather than a vague one. Here are the ones worth memorizing.

One naming trap sits under all of this. The Foundry RBAC roles were renamed. Foundry User, Foundry Owner, Foundry Account Owner, and Foundry Project Manager were previously Azure AI User, Azure AI Owner, Azure AI Account Owner, and Azure AI Project Manager. The IDs and permissions did not change, but you will still see the old names in some blades while the rename rolls out. Match on the role ID when a guide and the portal disagree.

Basic setup to prototype, standard setup before anyone real logs in

Here is where I land. Start on basic setup and build the agent, the tools, and the instructions until it behaves. You will iterate a dozen times, and Microsoft-managed storage keeps that loop fast. The instant you have a real user or real data, provision standard setup, size Cosmos for the projects you actually plan to run, and settle the storage story before you create the capability host, because you cannot change it later. Keep one agent until a genuine team or data boundary forces a second, then use connected agents rather than a framework you have to operate. The runtime is doing the hard part; your job is the sizing and the roles.

An agent that can call tools also needs a way to stop it saying the wrong thing. That is guardrails, and on Azure it runs through Content Safety, which is Part 14. Provision a basic-setup project today, paste in the Python above, and get one agent answering before you read on.

References

• What is Foundry Agent Service (Microsoft Learn)
• Set up standard agent resources (Microsoft Learn)
• Foundry Agent Service pricing (Microsoft Azure)

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts