By Dr. Pranay Jha, infrastructure architect and long-time vExpert, who designs and tests DR for production VCF estates.
One checkbox in a guest customization spec was all it took. It quietly deleted the network config on every interface that used DHCP, so when the recovery plan powered on all 180 machines at the second site, half of them came up reachable by nothing. The console reported success. The order queue reported the truth. That one setting had passed every design review and every quarterly test, because neither of them ever exercised the path a real failover takes.
Design mistakes in disaster recovery do not announce themselves during the build. They pass every review, they survive the first test, and they wait for the one morning when the primary site is gone and there is no second attempt. This part walks through the failures that stay hidden until a real outage on VMware Cloud Foundation 9 running VMware Live Recovery, why each one hides, and what to change before it costs you the recovery.
Why a green test proves less than you think
A non-disruptive test in VMware Live Site Recovery brings your virtual machines up on an isolated test network at the recovery site, checks that they boot and that the recovery plan steps complete, and then cleans up. That is genuinely useful, and Part 8 in this series covers how to run it well. What it does not do is exercise the parts of a real event that only exist during a real event. Failback never happens in a test. Reprotect never happens. Guest reachability across production routing never happens, because the test network is deliberately walled off. The steps that pass in test are not the steps that fail at 3 a.m.
So the honest way to read a green test is narrow. It tells you the recovery plan mechanics work and the replicated data is bootable. It says nothing about whether the application answers, whether the return trip works, or whether a storage rebalance last Tuesday quietly moved a protected VM outside the replication boundary. The mistakes below all live in that blind spot.
Mistake 1: trusting IP customization for mixed Linux NICs
Re-IP at the recovery site is convenient. You define IP subnet mapping rules, the recovery plan rewrites addresses as each VM comes up, and you avoid hand-editing anything. It works cleanly for Windows and for single-NIC Linux. It has a sharp edge for multi-NIC Linux guests that mix DHCP and static addressing on the same VM.
What actually happens
Broadcom documents this plainly in the Live Site Recovery known issues. When a Linux VM has several NICs and only some of them match an IP subnet mapping rule, Site Recovery Manager customizes the ones it has a rule for and can clear configuration on the others. On Red Hat Enterprise Linux and CentOS 6.x and 7.x, the customization deletes the /etc/sysconfig/network-scripts/ifcfg-ethX files for NICs configured with DHCP. The VM boots, the managed interface has the right address, and the DHCP interface comes up with nothing. If that interface was your application or storage network, the service is unreachable while every dashboard says the VM recovered.
The reason this hides is that a non-disruptive test rarely checks in-guest reachability across every interface. The VM powers on, the plan goes green, and nobody logs in to confirm that eth1 still has a config file. The gap only reveals itself when real users depend on the interface that got wiped.
The fix
For these guests, use the Manual IP Customization option in Site Recovery Manager rather than relying on subnet mapping rules to cover a partially matched set of NICs. Give every interface that matters an explicit customization entry so nothing is left to be cleared. If you can, standardize protected Linux VMs on static addressing for the interfaces that carry service traffic, which removes the mixed DHCP and static condition that triggers the behavior. And validate reachability from inside the guest during a test, not just power state from the console.
Mistake 2: a datastore cluster that quietly breaks consistency
This is the one I have seen strand the most VMs during a real failover, and it leaves almost no trail beforehand. You build a datastore cluster, turn on Storage DRS in automatic mode, and let it balance capacity and load. Somewhere in that cluster one datastore is not replicated, or two datastores sit in different array consistency groups. Storage DRS does its job and migrates a protected VM onto the datastore that keeps the cluster balanced. It has no idea that datastore is outside your replication boundary.
Here is the part that makes it dangerous. Broadcom notes that a datastore cluster made up of datastores that are not all in the same consistency group, or that are not all replicated, should raise a Site Recovery Manager warning, and it does not. There is no red banner. The VM keeps replicating from wherever it started, or stops being covered at all, and you learn the boundary moved only when the recovery plan reaches that VM and finds nothing to recover.
What to change
My opinion here runs against the convenience default. For any datastore cluster that holds protected VMs, do not run Storage DRS in fully automatic mode. Set it to manual, or at minimum partially automated so a human approves storage migrations, and keep every datastore in the cluster inside the same consistency group and the same replication scope. If a datastore is not replicated, it does not belong in a cluster with protected workloads. This costs you some automated balancing. It buys you a boundary that a background process cannot silently redraw.
Mistake 3: turning on vMotion at failover without reading the fine print
Recent Live Recovery versions can vMotion eligible VMs during a recovery, and the option to enable it looks like a free upgrade. On stretched storage it can reduce disruption. It also carries two failure modes that do not appear until the plan is already running, and both come straight from the documented known issues.
The first is array specific. With the Dell PowerStore storage replication adapter at version 2.2.0.447, using vMotion during a planned migration can leave the device in a permanent state mismatch. Broadcom is blunt about the remedy: uncheck the option to enable vMotion of eligible VMs when you run recovery. The second is worse because it can strand the plan. If a failover with vMotion is interrupted during the vMotion step, the plan can drop into a Recovery interrupted state, then every re-run can fail at the Change recovery site storage to writable step. The failover actually completes, but the plan stays in Recovery incomplete and will not return to Ready. To fail those VMs back you have to recreate the protection groups and the recovery plan.
My take is to leave vMotion on failover off unless you have a specific stretched-storage reason to use it and you have tested the exact SRA version in your environment. The default recovery path without it is well understood and does not have a state that requires rebuilding your plan to escape.
Mistake 4: treating failback as failover in reverse
Most DR plans are written for the trip out and go quiet about the trip home. Failback runs a reprotect to reverse replication, then a recovery in the other direction, and it has its own costs that never appear in a one-way test. Part 9 covers the workflow. What matters for design is that the return is often slower than the departure, and a specific, measurable delay shows up when NSX-T is in the picture.
Broadcom documents that for a VM on a vSAN datastore replicated by vSphere Replication with NSX-T present, migrating back to the protected site takes an additional 60 seconds per VM. NSX-T stores port configuration inside the VM directory, vSphere Replication does not replicate that directory content, so on the way back the port configuration is invalid, gets removed, and has to be resolved during registration in vCenter. Sixty seconds sounds trivial for one VM. It is not trivial for a recovery plan.
| VMs in the recovery plan | Seconds added at 60 s each | Added failback time |
|---|---|---|
| 25 | 1,500 | 25 min |
| 50 | 3,000 | 50 min |
| 100 | 6,000 | 1 h 40 min |
| 200 | 12,000 | 3 h 20 min |
| 400 | 24,000 | 6 h 40 min |
The fix for this one is concrete. Broadcom notes it is resolved by updating target recovery hosts to ESXi 8.0.2 or later, so the design lesson is to keep the recovery site hosts patched to a version that removes the penalty, and to size your failback window on measured behavior rather than assuming it mirrors failover. While you are there, remember that reprotect at large scale can itself fail under temporary storage or network pressure, so a failback plan should expect to retry reprotect on a subset of VMs rather than treat it as a single clean step.
Two smaller traps that still cost you
Not every design mistake is dramatic. Some just waste the one resource you cannot make more of during an outage, which is time. Two are worth knowing before they surprise you.
The boot timeout that ignores you
Slow-booting VMs, big databases and appliances, sometimes need a longer power-on timeout so the recovery plan does not mark them failed while they are still coming up. You raise recovery.powerOnTimeout, and nothing changes. The documented behavior is that this setting does not take effect on its own. You first have to set replication.archiveRecoverySettingsLifetime to 0 on both sites, then change powerOnTimeout, then repeat on the other site. Miss that order and your carefully raised timeout is quietly ignored, and a slow VM gets marked as a failure it did not have.
Storage that cannot be protected
Some configurations are simply out of scope, and it is cheaper to find that at design time. Live Site Recovery 9.0.5 does not protect virtual machines that have persistent memory devices or disks, and it does not support NFSv4.1 datastores for array-based replication. Virtual Volumes is deprecated starting with VCF 9.0 and Broadcom recommends migrating those replications to array-based replication. If any protected workload sits on one of these, the plan you think you have does not cover it. Catch that on paper, not during the recovery.
Myth versus reality
| The myth | What actually happens |
|---|---|
| A green DR test proves the plan works. | A non-disruptive test skips failback, reprotect and in-guest reachability. Those are exactly where these mistakes live, so a green test clears them without touching them. |
| Site Recovery warns me if a protected VM lands on the wrong datastore. | A datastore cluster mixing consistency groups or non-replicated datastores raises no warning. Storage DRS can move a VM outside protection silently. |
| vMotion on failover is a free improvement. | With some SRAs it causes a permanent device state mismatch, and an interrupted vMotion step can strand the plan in Recovery incomplete until you rebuild it. |
| Failback is just failover run backward. | Reprotect and re-registration add time, and unreplicated NSX-T port state adds about 60 seconds per VM, which is hours across a large plan. |
| If I raise the boot timeout it just applies. | recovery.powerOnTimeout is ignored unless you first set replication.archiveRecoverySettingsLifetime to 0 on both sites. |
Close the gaps before the outage picks them for you
Every mistake in this part shares one trait. It is invisible in the calm and expensive in the crisis, and the audit that would have caught it is one you can run any afternoon. Take the validation list above into your next DR review, work down it against your real environment, and turn the quiet failures into known ones. Part 14 closes the series with the runbook that ties this together, the one you want to trust at 3 a.m. A DR plan earns your trust the day it survives the test you least wanted to run.
« Previous: Part 12 | VMware Live Recovery Complete Guide | Next: Part 14 »
References
- VMware Live Site Recovery 9.0.5 Release Notes, caveats and known issues (Broadcom TechDocs)
- Detailed Design for Site Protection and Disaster Recovery for VMware Cloud Foundation (Broadcom TechDocs)
- VMware Live Recovery Frequently Asked Questions, January 2026 (VMware)
About The Author
DrJha