Picture your company payroll system going dark at 2am because one server quietly lost a power supply. Nobody is awake. Nobody notices until morning, and by then a hundred people are asking where their salary is. That single fear is why vSphere HA exists. It is the feature that notices a host has died and brings its virtual machines back to life on other hardware, without waiting for a human to wake up.
In Part 9 you met vMotion, which moves a running VM from one host to another with no downtime. HA is its blunter cousin. HA does not move anything gently. It waits for a host to fail, then restarts the VMs that were on it somewhere else. There is downtime, the length of a reboot, but the VMs come back on their own. People mix these two up constantly, so hold on to the difference: vMotion is planned and graceful, HA is unplanned and a reboot.
What HA actually promises (and what it does not)
A cluster is just a group of ESXi hosts that vCenter manages as one pool of CPU and memory. HA is a setting you turn on at the cluster level, not on individual VMs. Once it is on, every host keeps a quiet watch on every other host. The moment one stops responding, the survivors restart its VMs using the spare capacity that sits idle for exactly this reason.
Think of a hospital backup generator. When mains power cuts out, the generator starts within seconds. The lights blink off, then come back. Patients on life support keep breathing. HA is that generator for your virtual machines. The blink is the reboot your VMs go through. The fact that they come back without anyone running to the basement is the whole point.
The one hard requirement: shared storage
HA has one non-negotiable requirement that surprises freshers: the VMs must live on shared storage every host can reach, the datastores you met in Part 6. A VM is really a set of files on a datastore. If host A dies, host B can only power that VM on if host B can also see those files. If the VM sat on local disk inside host A, those files die with the host and HA can do nothing. This is why production clusters use shared storage such as SAN, NFS or vSAN, and why your home lab needs at least a shared NFS share to see HA work.
What HA does not do
HA does not protect against a crash inside the guest operating system. If Windows blue-screens but the host is fine, plain HA will not react, because the host is still healthy. There is a setting called VM Monitoring that watches VMware Tools heartbeats from inside the guest and reboots a hung VM, but it is off by default. HA also does not remove downtime. Your VMs reboot. Applications that cannot survive a reboot need clustering inside the guest, or Fault Tolerance, which is a heavier feature we compare below.
How HA decides a host is really dead
This is the part worth slowing down on, because it is where HA earns its reputation for being subtle. Deciding a host is dead sounds easy. It is not. A host that went silent might be truly dead, or it might have lost its network cable while still happily running your VMs. Restarting VMs that are actually still running would be a disaster: two copies of the same VM writing to the same disk. So HA is deliberately careful.
Master and slaves
When you enable HA, an agent called FDM (Fault Domain Manager) is pushed to every host. The hosts hold an election and one becomes the master; the rest are slaves. The election is quick, around 15 seconds, and the host that can see the most datastores tends to win. The master is the brain. It tracks which VMs are protected, watches the slaves, and orchestrates restarts when a slave goes down. If the master itself dies, the survivors simply elect a new one.
Two heartbeats, not one
Slaves and the master exchange a network heartbeat once per second over the management network. That alone is not enough to declare a host dead, because the management network is exactly the thing most likely to break. So HA adds a second channel: the datastore heartbeat. Each host also writes a heartbeat to shared datastores. When the master stops hearing a slave on the network, it does not panic. It checks the datastore. If the silent host is still updating its datastore heartbeat, the host is alive but isolated or partitioned, a network problem, not a dead host. If both the network and datastore heartbeats are gone, the host is genuinely dead and its VMs get restarted elsewhere.
The detail you only learn by doing it
A host that stops hearing the master pings an isolation address to work out whether it is the one cut off. By default that address is the management network gateway. In plenty of real environments the gateway does not answer ping, so the host wrongly concludes it is isolated and may shut down its own VMs depending on the isolation response. The known fix is to set the advanced option das.isolationaddress to an IP that actually replies, and to set the host isolation response sensibly. On modern shared storage, power off is the common choice so the VMs can be restarted safely elsewhere. This one misconfiguration causes more surprise outages than almost anything else in HA.
Admission control: the part everyone wants to switch off
Restarting VMs after a failure only works if there is somewhere to restart them. If every host in your cluster is already running at 95 percent memory, a host failure has nowhere to send the orphaned VMs and HA fails at the worst possible moment. Admission control is the safety net that stops you filling the cluster so full that failover becomes impossible. When a colleague says I cannot power on this VM, admission control is very often the reason.
The three policies
You choose how HA reserves the safety margin. There are three policies, and the cluster resource percentage one is the default on modern vSphere.
| Policy | How it reserves capacity | Good when |
|---|---|---|
| Cluster resource percentage | Keeps a set percentage of CPU and memory free | Most clusters; easiest to reason about |
| Slot policy (host failures to tolerate) | Reserves whole slots sized to the largest VM | Uniform VM sizes |
| Dedicated failover hosts | Keeps named hosts empty as standby | You can afford idle hardware |
The slot size trap
The slot policy hides a classic gotcha. A slot is a unit of CPU and memory big enough for any single powered-on VM. If most of your VMs are small but one VM has a giant 64 GB memory reservation, HA sizes every slot to fit that giant. Suddenly your cluster has far fewer slots than you expected, and admission control blocks new VMs even though there is plenty of real free memory. The cure is either to fix the oversized reservation or to cap the slot size with an advanced option. Interviewers love this one, because it shows whether you understand the mechanism or just the menu.
Worked example: the failover math
Say you run 10 application VMs, each needing 16 GB of RAM, so 160 GB in use. You want to survive any single host failure, often written N+1. With three hosts of 128 GB each you have 384 GB total. After one host dies you have 256 GB left, comfortably more than the 160 GB your VMs need, so all 10 restart. Set admission control to reserve roughly one host worth of capacity, about 33 percent here, and the cluster refuses to let you pile on so many VMs that a failure can no longer fit. If instead you packed VMs to use 250 GB, one host loss leaves only 256 GB and you are one large VM away from a failed restart.
Restart order, and when storage misbehaves
Restart priority
When several VMs need to come back at once, HA restarts them in priority order: Highest, High, Medium, Low, Lowest. You set this so the things that matter most return first. The classic example is making your domain controllers, and in a self-managed setup vCenter itself, a high priority, because so much else depends on them. Restart order is one of those settings nobody configures until the first real failure brings the reporting server up before the database it depends on.
VMCP, APD and PDL
HA also handles a nastier failure than a dead host: a host that is alive but has lost its storage. VM Component Protection (VMCP) watches for two storage conditions. PDL (Permanent Device Loss) is when the array tells the host a device is gone for good. APD (All Paths Down) is when the host simply cannot reach the storage and does not know whether it will return. With VMCP enabled, HA can power off the affected VMs and restart them on a host that still has working storage. APD offers a conservative and an aggressive setting: conservative only restarts when HA is confident there is a healthy host to move to.
Real interview question
What is the difference between vSphere HA and Fault Tolerance?
A weak answer says both keep VMs running. A strong answer: HA restarts a failed VM on another host, so there is a short outage the length of a reboot, and it protects many VMs cheaply. Fault Tolerance runs a second, shadow copy of the VM in lockstep on another host, so if the original host fails there is zero downtime and no reboot, but it is expensive and limited to smaller VMs, so you use it only for the few workloads that truly cannot reboot. Add that HA needs shared storage and you have answered better than most.
Try it yourself
In a nested home lab (VMware Workstation or nested ESXi), build a tiny cluster of two ESXi hosts in vCenter and put their VMs on a shared NFS datastore. Turn on HA at the cluster level. Now power off one host hard, pull its virtual power rather than shutting it down cleanly. Watch vCenter: within a few minutes the VMs from the dead host should restart on the survivor. You got it right if those VMs reappear as powered on under the second host and the cluster summary shows a host failure was handled. If nothing restarts, check that the VMs were on the shared NFS and not on local disk. That is the number one reason home lab HA does nothing.
FAQ
Does HA cause downtime?
Yes. The VMs reboot. HA restarts them automatically, but the applications are unavailable for the length of that reboot, typically a few minutes. If you need zero downtime, that is Fault Tolerance or in-guest clustering.
How long does HA take to restart VMs?
It varies. The master needs a short time to confirm the host is truly dead using the network and datastore heartbeats, then the VMs power on and boot. In practice expect a few minutes from failure to VMs being back, longer for VMs with slow-booting applications.
Do I need vCenter running for HA to work?
You need vCenter to configure HA, but once it is set up the FDM agents on the hosts handle failover on their own. HA will still restart VMs even if vCenter is down, which is why vCenter is often given a high restart priority.
What is the difference between HA and DRS?
HA restarts VMs after a failure. DRS, which is Part 11, balances VMs across hosts for performance during normal running. They work together: HA gets the VMs back, then DRS rebalances where they landed.
Should I ever disable admission control?
In a lab, sure, to experiment. In production, no. Disabling it removes the guarantee that failed VMs can restart. If it is blocking you, that is a capacity signal, not a nuisance to switch off.
The takeaway
HA is the quiet feature that turns a dead server from a 2am emergency into a few minutes of reboot. Remember the three ideas that matter: VMs must be on shared storage, HA uses two heartbeats so it does not restart a host that is merely isolated, and admission control keeps the spare capacity that makes failover possible. Leave admission control on.
Set up a two-host nested lab this week and force a host failure. Watching HA restart a VM on its own teaches you more than any diagram. Next we look at DRS, the feature that decides which host each VM should run on in the first place.
References
- vSphere HA Admission Control — Broadcom TechDocs
- Design decision: set vCenter restart priority to high — Broadcom
- What is VM Component Protection (VMCP) — Settlersoman
About The Author
DrJha