A freshly installed OpenShift cluster enforces nothing between workloads. Every pod reaches every other pod in every namespace, on every port, and the cluster considers that correct behaviour. That is the flat network your NSX-T distributed firewall spent three years carving up on the TKGI side, and on day one of the target platform none of that segmentation exists.

Distributed firewall, usually shortened to DFW, is the NSX-T feature that enforces rules at each virtual machine network interface rather than at a perimeter appliance. On TKGI it did two jobs at once. It translated Kubernetes NetworkPolicy objects into firewall rules through the NSX Container Plugin (NCP), and it carried a second body of rules that your network team wrote by hand in NSX Manager against NSGroups. Only the first body of rules is visible to kubectl. Miss the second and the migrated estate is quieter than it should be, in a way nobody notices until an audit.

Part 15 gave people a way to log in and a place to pull images from. This part decides what those workloads are allowed to talk to once they arrive. It has to land before wave one moves, because retrofitting segmentation onto running production is how you end up with a change freeze.

Preflight and DFW Rule Inventory

Three things have to be true before you write a single policy. Your cluster network type must actually be OVN-Kubernetes, the AdminNetworkPolicy custom resource definitions must be present, and you need a rule count from the NSX side that you can defend in a meeting. That last one is the step teams skip, and it is the one that decides whether this migration reproduces your security posture or quietly relaxes it.

Three hundred and sixty five enforced rules against one hundred and sixty eight Kubernetes objects. That gap is the whole point of this section. Forty seven of the remainder belong to infrastructure and the default section, which leaves one hundred and fifty tenant rules that exist only in NSX Manager, written by people who were never going to file a pull request against an application repository. Every migration guide that says export your NetworkPolicy YAML and reapply it is describing forty six percent of your firewall.

Getting those hand written rules out of NSX Manager is tedious rather than hard. Walk the security policies endpoint, expand each rule, and pull the source groups, destination groups, services and action into a spreadsheet. What you are building is not a translation, it is a reconciliation list, and every row ends in one of exactly two states. Either it becomes a Kubernetes object, or somebody signs off in writing that the rule is retired. A third category, meaning rules you meant to look at later, is how estates arrive in production with a segmentation story nobody can defend. Budget half a day per cluster for this and do it before you write any policy, because the inventory changes which objects you reach for.

Mapping NSX-T DFW Constructs to OVN-Kubernetes Objects

Once you have both inventories, you need a translation. Some DFW concepts map cleanly, some map with a caveat, and a few have no target at all. Keep the table below beside you while you write policy, because arguing about it in the middle of a cutover window is expensive.

Two rows deserve more than a table cell. Reject is the one that generates support tickets, and I cover it in the field note below. Layer 7 context profiles are the one that generates escalations, because somebody in security signed off on a rule that inspects HTTP method and there is no way to reproduce that with a NetworkPolicy. If your DFW ruleset leans on layer 7, that requirement moves to Red Hat OpenShift Service Mesh or it goes away, and that decision belongs in the design review, not in the cutover.

One difference sits underneath the whole table and it is a mental model rather than a mapping. NSX-T DFW evaluates rules in sequence and the first match wins, so a Drop at position 1010 genuinely stops everything below it from being considered. Kubernetes NetworkPolicy does not work that way at all. Policies are additive. Once any policy selects a pod, that pod is deny by default for the listed directions, and every other policy selecting the same pod contributes its allows to a union. There is no ordering, no first match, and critically no way for one NetworkPolicy to subtract permission that another one granted. Engineers who spent years thinking in rule sequence write a policy expecting it to override an earlier one, and instead they widen the opening. AdminNetworkPolicy reintroduces ordering through priority values, which is the reason it exists, and it is also the reason it deserves the restraint I argue for in the next section.

Step by Step, Namespace Baseline and Tenant Policies

Work namespace by namespace. Four objects per namespace give you a posture equivalent to a well built DFW section, and they go in this order every time.

Step 1. Label the namespace so cluster scope policy can find it later. Step 2. Apply a default deny for both directions. Step 3. Immediately add the DNS egress exception, in the same oc apply, not as a follow up. Step 4. Add the ingress exceptions for the router and for monitoring, then the application specific rules translated from your DFW inventory.

Apply the deny without the DNS rule, even for ninety seconds, and this is what your application logs produce. I include it because it is the single most common failure in this whole part, and because the error names a timeout rather than a firewall, which sends people looking in the wrong place.

Cluster Scope Guardrails with AdminNetworkPolicy

AdminNetworkPolicy is the object that looks most like a DFW section, and that resemblance is a trap. It is cluster scoped, priority ordered and owned by the platform team, exactly like the section your network engineers used to edit. So the instinct is to lift the whole DFW section into one big ANP and be done. Resist it. An ANP Deny is non overridable by design, which means when it blocks something legitimate the namespace owner cannot unblock it, cannot see why, and files a ticket that lands on you at 22:00.

Use ANP for the handful of rules that genuinely must never be overridden, and use a BaselineAdminNetworkPolicy as the catch all underneath. BANP is a singleton, it must be named default, and its rules are overridable by tenant NetworkPolicy objects. That combination gives you a deny by default cluster where a team can still ship without a platform ticket.

Pass is the action with no DFW ancestor and it is worth understanding properly, because it is what makes the three tier model usable. Allow and Deny both settle the question immediately and skip everything below. Pass explicitly declines to settle it and hands evaluation down to tier 2, where namespace NetworkPolicy objects get their say, and then to tier 3 if nothing selected the pod. In practice that gives you a way to say something a DFW section could never express, which is that platform engineering has an opinion about this traffic class but delegates the final call to the team that owns the workload. Monitoring scrape traffic is the obvious candidate. Some teams must be scraped, some must never be, and most should choose. Pass at tier 1 with a Deny in the BANP underneath gets all three outcomes from one policy pair.

flowchart TD
  P[Packet on OVN logical switch] --> T1[Tier 1, AdminNetworkPolicy]
  T1 -->|Allow| A[Delivered]
  T1 -->|Deny| D[Dropped, no ICMP]
  T1 -->|Pass| T2[Tier 2, NetworkPolicy]
  T1 -->|No ANP matches| T2
  T2 -->|A rule allows it| A
  T2 -->|Pod selected, no rule matches| D
  T2 -->|No policy selects the pod| T3[Tier 3, BaselineAdminNetworkPolicy]
  T3 -->|Allow| A
  T3 -->|Deny| D
  T3 -->|No BANP present| A

Here is the pair I run in the reference estate. One ANP holds the three rules nobody may break, and the BANP closes everything else. Note the priority values, and note the failure underneath, which is the mistake every NSX engineer makes on their first try because DFW sequence numbers start at 1000 and climb.

Now the failure. Set priority: 600 because that is what the equivalent DFW rule was numbered, and the object is accepted by the API server but never programmed. Nothing warns you at apply time. You only find out when you check status, which is why the verification section below is not optional.

Verification, Audit Logging and Rollback

Four checks prove the chain. Each fails loudly rather than quietly, which is why I picked them over a connectivity smoke test that only tells you the happy path works.

Rollback is one command per namespace and it is fast, which is the good news about doing segmentation with Kubernetes objects rather than a firewall appliance. Delete the policies and the namespace returns to open within a second or two, because OVN-Kubernetes removes the access control lists as soon as the object goes.

Policy Failures and Remediation

Field Note from a Deny That Broke Name Resolution

Pilot window, a Tuesday afternoon. I applied the default deny to wave1-web and wave1-data at 14:10, planning to add exceptions rule by rule and watch what broke. Textbook approach, and it is wrong. Two minutes later the storefront started returning 500s while every pod stayed Running and Ready, because the readiness probe was an HTTP GET on the root path that never opened a database connection. It took 26 minutes to find, and 24 of those went into checking the Route, the Service endpoints and the ingress controller, because a lookup timeout reads like a DNS outage and not like a firewall.

Cluster DNS is egress traffic. That sentence is obvious in hindsight and invisible at 14:12. On TKGI it never came up, because the NCP generated section carried an infrastructure allow that predated any of us.

Audit logging would have answered it in ninety seconds, and I had not turned it on, because I planned to enable it once the policies were stable. Wrong order. Annotate the namespace before the first deny lands, watch acl-audit-log.log while you apply, and the verdict lines name the policy that dropped the packet. Logging costs disk and some throughput on busy nodes, which is a fair reason to turn it off later, and no reason at all to leave it off during a migration wave.

Second surprise the same afternoon, and this one cost more. Our NSX-T tenant boundary used Reject, so a blocked connection came back as a refusal in single digit milliseconds and the application failed fast and retried. OVN-Kubernetes drops. Same policy intent, and now the client sat on an open socket for its full 30 second default until the pool of 20 connections was gone and the pod was effectively dead while reporting itself healthy. I reversed my plan after that. Segmentation went in with the DNS and router exceptions preloaded, audit logging on before the deny rather than after, and a pull request against three application repositories setting an explicit five second connect timeout. That last change had nothing to do with OpenShift and everything to do with the fact that Reject had been papering over sloppy client configuration for years.

Namespace Baselines First, Cluster Guardrails Second

A clean result looks like this. Every migrated namespace carries a default deny with DNS, router and monitoring exceptions applied in a single manifest. One BANP named default closes the gaps underneath. One ANP holds three rules at priority 20, all reporting SetupSucceeded in every zone. Your NSX rule inventory reconciles to a Kubernetes object or to a written decision that the rule is retired, with no third category. Audit logging is on for the namespaces still in a migration wave and off everywhere else.

Monday morning, run the two inventory commands from the preflight section against one TKGI cluster and subtract. Whatever number falls out is the part of your security posture that no export will carry across, and it is better to know it now than during the pilot. If you are still weighing whether OpenShift is the right landing place at all, the TKGI to VKS guide covers the VMware side of that fork. Otherwise the next part starts moving actual workloads, with Velero on TKGI and OADP on OpenShift, and it assumes the segmentation in this part is already standing. Worth rereading Part 8 on the NSX-T and NCP assessment alongside it, because the egress identity decisions you made there constrain what you can write here.

References

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts