Dr. Pranay Jha

Up to now this series has been about building. From here it is about running, and running a software-defined network well comes down to one thing: can you see what it is doing? The old complaint about virtual networking was that it turned the network into a black box, traffic vanished into the overlay and you had no idea where it went. NSX answers that complaint with a genuinely good toolset, and the difference between an operator who fixes problems in minutes and one who escalates for hours is almost entirely whether they know which tool to reach for. So this part is a tour of that toolbox, starting with the one I reach for first on almost every incident.

Traceflow: see the path

Traceflow is the tool I open first when someone says “A cannot reach B.” It injects a synthetic packet from a source and follows it through the data plane hop by hop, the source vNIC, the Distributed Firewall, the segment, the Tier-1, the Tier-0, and tells you at each step whether the packet was forwarded or dropped, and if dropped, by what. When a DFW rule is silently blocking traffic, Traceflow shows you the exact rule and the exact hop, which turns a vague “the firewall might be doing it” into a precise answer in under a minute. Because the packet is synthetic, you can test connectivity that does not exist yet, before a workload is even sending traffic, which makes it as useful for validating a new design as for diagnosing a broken one.

The rest of the toolbox

Traceflow uses a synthetic packet, which is perfect for a clean yes-or-no path test but does not show you what real traffic is doing. For that, NSX gives you more tools, each suited to a different question. Live Traffic Analysis traces actual packets in flight, so you can study a real flow rather than a manufactured one. Port mirroring copies traffic from a source to an analyzer or capture tool, the virtual equivalent of a SPAN port, for when you need to see the bytes themselves. IPFIX exports flow records, who talked to whom, on what ports, how much, to a collector, which is the raw material for flow analysis and for building micro-segmentation policy from observed behaviour. Knowing which of these answers your question is most of the skill.

One VCF 9 detail worth knowing: some IPFIX latency tracking that previously needed the vDefend Firewall license is now built into VCF, and a new ConnectionTrack module enriches the exported flows with session flags, retransmission counts, flow direction, and average latency. That richer data is what makes flow-based troubleshooting and planning genuinely useful rather than just a list of conversations.

Alarms and System Health

Troubleshooting tools answer a question you already know to ask. Alarms and health monitoring tell you there is a question before a user does. NSX raises alarms against a defined set of events, a degraded cluster, a failing tunnel, a capacity threshold, and the System Health Monitoring view centralizes the picture: overall status, active alarms, resource utilization, API usage, and the reachability of the compute managers NSX depends on. The discipline that separates good operations from firefighting is to wire these alarms into wherever your team actually looks, your monitoring platform or alerting tool, rather than expecting someone to keep the NSX UI open. An alarm nobody sees is the same as no alarm at all.

Operations for Networks: the wider view

NSX’s built-in tools are excellent for a specific flow or a specific component. For the estate-wide picture, VCF Operations for Networks, the product formerly called Aria Operations for Networks and vRealize Network Insight before that, consumes the flow data and turns it into something broader: a map of how applications actually communicate, the evidence base for designing micro-segmentation from observed behaviour rather than guesswork, and assurance that verifies the network is doing what you intended. It is the difference between troubleshooting one problem and understanding the whole system. When a security team asks “what talks to this application, really?”, this is the tool that answers, and the answer it gives is what you feed back into the groups and rules from Parts 12 and 13.

A worked troubleshooting flow

Here is how the tools chain together on a real ticket, the sequence I run almost on autopilot. The report comes in: the application server cannot reach the database. First, Traceflow from the app VM to the database VM. The result says dropped at the DFW, and names the rule. That is the moment most of the work is already done, because I now know it is a policy problem, not routing, not the overlay, not the database. Next, I open the named rule and the group it uses, and check effective members from Part 13, the usual culprit is that the database VM was never tagged and so is not in the group the allow rule applies to. I fix the tag, Traceflow again to confirm the packet now reaches the destination, and close the ticket. Three steps, a few minutes, no escalation, because each tool answered exactly one question and pointed at the next.

Contrast that with the version where the operator does not reach for Traceflow: hours auditing firewall rules by eye, pulling in the database team, second-guessing the routing, all to eventually discover a missing tag that Traceflow would have pointed at in sixty seconds. The tools are not just convenient; they change the entire shape of an incident. The skill is knowing the sequence, and the sequence almost always starts with Traceflow telling you which layer the problem lives in.

What I’d Do

Make Traceflow the first move on any connectivity ticket, because it converts a guess into an answer faster than anything else, and use Live Traffic Analysis and port mirroring when you need to see real packets rather than a synthetic one. Wire NSX alarms into the monitoring system your team actually watches, so problems find you instead of waiting to be discovered. Turn on IPFIX and feed Operations for Networks, both to understand how your applications really communicate and to design micro-segmentation from evidence rather than assumption, and remember that in VCF 9 some of that flow and latency data is now built in. The goal is simple: never operate the network blind. NSX gives you the visibility; the discipline is in actually using it. Next up is Part 19: backup, restore, and config protection, the safety net under everything you have built. When something breaks tomorrow, is Traceflow your first instinct, or your last resort?

Alarms tell you what; Traceflow tells you where

NSX alarms surface component health and events, a degraded manager node, a failed service, a tunnel that went down, and the discipline that separates calm operations from firefighting is working those alarms proactively rather than waiting for users to notice. An unactioned alarm is just a future incident with an early timestamp you chose to ignore. The alarm framework in NSX 9 is considerably richer than the NSX-T days, but richer only helps if someone is watching, triaging and clearing them, so build the habit of treating the alarm list as a work queue rather than wallpaper.

Traceflow answers the complementary question: not what is unhealthy, but where a packet is dying. It injects a synthetic packet and renders every component it traverses, pinpointing the exact hop where a flow is dropped, which collapses hours of guesswork into a single picture. The prerequisite people forget is that the segment must be connected to a gateway with gateway connectivity enabled, or Traceflow fails for a reason that has nothing to do with your actual problem and sends you chasing a phantom. Used together, alarms point you at what is wrong and Traceflow shows you where in the path the symptom actually lives.

Operations for Networks closes the flow-visibility gap

Alarms and Traceflow are point tools, excellent for health and for single-flow tracing, but they do not give you the continuous picture of who is talking to whom across the whole estate. That is what Operations for Networks provides: flow-level visibility, traffic analytics, assurance and anomaly detection across the environment over time. It is the tool that turns the question which workloads actually communicate, and how much, from a guess into a dataset, and that dataset is the foundation the entire micro-segmentation methodology depends on.

The posture that distinguishes mature teams is proactive rather than reactive. Watch the alarms, baseline the flows, and use the assurance and verification features to catch drift and anomalies before they become user-visible, instead of reaching for Traceflow only after the phone rings. Monitoring you look at exclusively during incidents is monitoring you paid for and never used. Build the operational rhythm around the continuous view, keep the point tools sharp for when you need them, and most problems become things you notice and fix on your own schedule rather than emergencies someone else discovers for you.

Baseline the environment before you need to

The single most undervalued monitoring activity is capturing a baseline while everything is healthy. Flow volumes, component metrics, the normal shape of traffic between tiers, all of it is far more useful as a comparison point than as a thing you scramble to characterize in the middle of an incident. Troubleshooting without a baseline is troubleshooting by guesswork, because you have no idea whether the number in front of you is alarming or completely ordinary for this environment at this hour.

So invest in the baseline before the bad day. Let the flow visibility and the metrics accumulate a picture of normal, and revisit that picture as the environment grows so it stays current. When something does break, the question becomes what changed relative to normal, which is a question you can actually answer, rather than is this value bad, which you cannot answer in isolation. The teams that resolve incidents fastest are almost always the ones that knew what healthy looked like before the incident started, because half of diagnosis is just recognizing the deviation.

References

• What’s New in NSX for VCF 9.0 (Broadcom TechDocs)
• NSX 9.0 Administration Guide: monitoring and troubleshooting (Broadcom TechDocs)
• NSX 9 Distributed Firewall Fundamentals (NSX Series, Part 12)

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts