Search the Azure portal for a Maia virtual machine. Nothing comes back. Microsoft runs Maia in its own datacenters and never hands you an SSH key to one. The GPUs you can rent are the ND-series, and here is the part nobody says out loud: most teams shipping on Azure OpenAI should never rent those either. This part is about the metal under the managed endpoints you have been calling since Part 3, what the ND-series SKUs are, what Maia is, and the short list of cases where dropping down to raw GPUs is the right call.
Where GPU VMs and Maia sit in the stack
Start with the layer you already use. When you call an Azure OpenAI deployment, your request lands on a model that Microsoft is running for you on GPUs and, increasingly, on Maia. You never see the hardware. You pay per token or per PTU, the way Part 6 laid out, and capacity, patching, and driver versions stay Microsoft’s problem.
Drop one layer down and you reach the ND-series. ND is the GPU-accelerated VM family built for training and large-model inference. Rent an ND VM and you get the whole node: eight GPUs on most SKUs, the CPUs and memory around them, and an InfiniBand fabric to cluster nodes together. You also inherit every job that used to be Microsoft’s problem. You install drivers, you keep CUDA or ROCm current, you handle a node that drops in the middle of a training run.
Maia sits on the same physical layer as the ND GPUs, but you reach it differently. Maia is Microsoft’s in-house AI accelerator, a chip Microsoft designed itself. It is not sold as a VM SKU and it is not in the size list. It runs first-party and platform workloads: Copilot, and portions of Azure OpenAI capacity. When you call a model on Azure, some of that traffic may be served by Maia and you would never know. That is the whole point of it. Microsoft controls the stack from the chip to the API, which is how it holds latency and cost steady while demand keeps climbing.
What the ND-series actually gives you
Three ND-series SKUs matter for generative AI in 2026, and they split by who makes the accelerator.
ND H200 v5 is the NVIDIA H200 node. One VM, the Standard_ND96isr_H200_v5 size, carries eight H200 GPUs with 141 GB of HBM3e each, wired together with 900 GB/s NVLink inside the box. Across boxes, each GPU gets its own 400 Gb/s Quantum-2 InfiniBand link, so a full VM has 3.2 Tb/s of scale-out bandwidth and clusters reach into the thousands of GPUs. The host is 96 Intel Sapphire Rapids vCPUs and 1850 GiB of RAM. This is the workhorse for training and for hosting large open-weight models.
ND GB200 v6 is the Blackwell generation, and it changes the unit of thought. Instead of eight discrete GPUs in a box, GB200 is rack-scale: NVIDIA Grace CPUs and Blackwell GPUs joined by fifth-generation NVLink across a whole rack. Each VM pairs Grace CPUs with Blackwell GPUs at 192 GB of HBM3e per GPU and roughly 8 TB/s of memory bandwidth per GPU, a real step over H200. It reached general availability on Azure and is aimed at the biggest training runs and the highest-throughput inference.
ND MI300X v5 is the AMD option. One VM, the Standard_ND96isr_MI300X_v5 size, packs eight Instinct MI300X GPUs for 1.5 TB of HBM per VM and 5.3 TB/s of aggregate HBM bandwidth, the most GPU memory per VM of the three. It runs on ROCm rather than CUDA, and Microsoft already serves parts of Azure OpenAI on MI300X. If your model is memory-bound and your stack is ROCm-clean, this is the value play.
| ND SKU | Accelerator | Per VM | HBM each | Interconnect |
|---|---|---|---|---|
| ND H200 v5 | 8x NVIDIA H200 | 96 vCPU, 1850 GiB | 141 GB HBM3e | 900 GB/s NVLink, 3.2 Tb/s IB |
| ND GB200 v6 | Grace plus Blackwell | rack-scale node | 192 GB HBM3e | 5th-gen NVLink, 400 GB/s IB per GPU |
| ND MI300X v5 | 8x AMD MI300X | 1.5 TB HBM total | 192 GB HBM | 5.3 TB/s aggregate HBM |
H200, GB200, or MI300X?
The honest answer is that most teams pick the SKU their quota approval and their software stack allow, not the one that looks best on paper. Even so, the memory number drives the first cut, because a model has to fit before anything else matters. Look at HBM capacity per accelerator. That single figure decides how big a model one chip can hold and how much room is left for the KV cache that grows with every token of context.
If your stack is CUDA and you want the newest silicon, GB200 wins on raw bandwidth and is built for the largest jobs, but supply is tight and a rack-scale unit is overkill for a 13B model. H200 is the safe default: broad availability, a mature CUDA path, and 141 GB per GPU is enough for most open models with tensor parallelism. MI300X gives you the most memory per VM and often a better price for memory-bound inference, at the cost of living in ROCm. Every hyperscaler now builds its own silicon for the same reasons; AWS makes Trainium and Inferentia, which I covered in the AWS series.
In practice
I reach for H200 first, MI300X when the model is large and the team is comfortable outside CUDA, and GB200 only when a training run genuinely needs rack-scale NVLink. Picking GB200 because it is newest is how teams end up waiting weeks on capacity for a job an H200 cluster would have finished.
Can you rent a Maia chip?
No, and understanding why tells you something about Azure’s direction. Maia is Microsoft’s own accelerator, designed to run inference for Microsoft’s own services at a cost and power profile Microsoft controls end to end.
Maia 100 came first. It is a large chip on TSMC’s N5 process, around 820 mm2, with 64 GB of HBM2E, 1.8 TB/s of memory bandwidth, and a custom Ethernet-based fabric giving 4.8 Tb/s per accelerator. It supports low-precision formats like MX. Maia 100 went into Microsoft datacenters to carry internal load, Copilot and Azure OpenAI among the tenants.
Maia 200 was announced in January 2026, built for inference specifically. It moves to TSMC’s 3nm node with 140 billion transistors, 216 GB of HBM3e at 7 TB/s, and 272 MB of on-chip SRAM. Microsoft quotes over 10 petaFLOPS at FP4 and over 5 at FP8, native FP8 and FP4 tensor cores, and Ethernet scaling to clusters of up to 6,144 accelerators. Microsoft claims about 30 percent better performance per dollar than the systems it replaces. It is going into the US Central region near Des Moines, Iowa first, with US West 3 near Phoenix, Arizona next.
| Spec | Maia 100 | Maia 200 |
|---|---|---|
| Process | TSMC N5 | TSMC 3nm |
| HBM | 64 GB HBM2E | 216 GB HBM3e |
| Memory bandwidth | 1.8 TB/s | 7 TB/s |
| On-chip SRAM | large SRAM | 272 MB |
| Scale-out fabric | custom Ethernet, 4.8 Tb/s per chip | Ethernet, up to 6,144-chip clusters |
| Rentable as a VM? | No | No |
What does that mean for you, the person building on Azure? Directly, nothing you configure. Indirectly, a lot. Every efficiency Microsoft wins on Maia is what lets it hold Azure OpenAI prices and keep PTU capacity available. You get the benefit through the endpoint, not through a VM.
Size the GPU to the model
Here is the calculation that decides whether one GPU is enough, and it is the one teams skip. Model weights in FP16 take two bytes per parameter. A 70 billion parameter model is therefore about 140 GB just for weights. A single H200 has 141 GB. It looks like it fits. It does not.
You also need memory for the KV cache, which holds the attention state for every token in the context window and grows with batch size and sequence length, plus activations and framework overhead. On a real serving setup that easily adds 30 to 60 GB. So a 70B model in FP16 needs two H200 GPUs at least, and the model splits across them with tensor parallelism. Drop to FP8 or a 4-bit quantization and the weights halve or quarter, so one GPU can hold the model, at some quality cost you have to measure rather than assume.
Worked example
Serving a 70B model in FP16. Weights: 70e9 params times 2 bytes equals 140 GB. KV cache and overhead at a 32k context with a modest batch: budget another 40 to 45 GB, call it 185 GB total. One H200 gives 141 GB, so a single card cannot hold it. Two H200 give 282 GB, which fits with headroom for larger batches. Verdict: plan two GPUs minimum for this model in FP16, or quantize to FP8 and fit it on one. Numbers to validate for your own case: exact KV cache size at your context length, and the quality loss from quantization.
This is also why MI300X and its 192 GB per GPU is attractive for large models. Fewer GPUs to hold the same weights means less cross-GPU traffic and a simpler deployment, and sometimes it means one card where NVIDIA would need two.
Check your quota before you promise a date
ND-series capacity is not sitting idle waiting for you. GPU quota on a new subscription is often zero, and a quota increase for H200 or GB200 can take days and sometimes a capacity conversation with your account team. Check before you commit to a delivery date, not after. Two read-only Azure CLI commands answer the question.
# 1) Is the H200 SKU offerable in your region, and are you restricted?
az vm list-skus
--location eastus2
--size Standard_ND96isr_H200
--all
--output table
# 2) How much ND-family GPU quota do you already have?
az vm list-usage
--location eastus2
--query "[?contains(localName, 'ND')]"
--output tableExpected: command 1 returns a row for the size with an empty Restrictions column when it is available in that region, or NotAvailableForSubscription when it is not. Command 2 lists CurrentValue against Limit for each GPU family. Failure mode: skip these, go straight to an az vm create, and you hit a SkuNotAvailable or a quota error at deploy time, after the sprint is already planned around GPUs you do not have.
Where the self-host math usually falls apart
Most teams who ask me about the ND-series do not need it. They are trying to cut the token bill on Azure OpenAI, and renting eight H200 GPUs to self-host an open model rarely comes out cheaper once you count the idle GPU hours between traffic peaks, the ops load, and the engineer who now owns driver upgrades and dead nodes. A managed endpoint scales to zero attention from you. A rented node bills whether or not a single request arrives.
My take
Rent metal when you own the model, not when you want a discount. Fine-tuning at scale, a proprietary open-weight model the catalog will not host, or a research run that needs the raw node are the cases that justify an ND VM. A cheaper way to call the same public model almost never is. [AUTHOR: add a real before-and-after cost anecdote from a self-host project.]
Rent a GPU only when you own the model weights
So here is the decision, stated plainly. If you are consuming a model through an Azure OpenAI or Foundry endpoint, you are already on this hardware and you should stay there. The managed layer runs on the same H200, MI300X, GB200, and Maia silicon, and it hides quota, drivers, and dead nodes. Reach for the ND-series only when you cross into ground the managed endpoint cannot cover: fine-tuning or continued pre-training on your own data at scale, hosting an open-weight model the catalog does not offer, or a research workload that needs the raw node. When you do cross that line, start with ND H200 v5 for its availability and CUDA maturity, size the GPU count to the model with the memory math above, and confirm quota in your target region before anyone writes a date on a plan. Maia you will never rent, and that is fine, because you get its efficiency every time you call the endpoint. Next I take on regions, quotas, and data zones, where the capacity you just checked becomes a map of where your model can legally and physically run. If you take one action from this part, open the portal and check your GPU quota in the region you actually deploy to.
References
- ND-H200-v5 size series, Microsoft Learn
- General availability of ND GB200 v6, Azure HPC blog
- Maia 200, the AI accelerator built for inference, Microsoft blog
- ND-MI300X-v5 size series, Microsoft Learn
About The Author
DrJha