oc get svc printed <pending> under EXTERNAL-IP for eleven minutes before I accepted that nothing was coming. On Tanzu Kubernetes Grid Integrated (TKGI) that column filled itself in, because the NSX Container Plugin (NCP) watched for services of type LoadBalancer and asked NSX-T for a virtual server. OpenShift Container Platform 4 (OCP) on vSphere ships no such watcher. The field stays empty, no event is raised, and nothing in the console tells you why.
Part 12 installed the cluster. This Part gives it a data path: the objects that carry HTTP into it, the shards that keep internal traffic away from external, the load balancer TKGI used to hand you at no effort, and the source address your database team allowlisted three years ago and has forgotten how to change.
Preflight, What the Cluster Must Already Do
Four things have to be true before any of this is worth typing. Wildcard DNS for the apps domain resolves to the ingress virtual IP address (VIP) chosen at install. The ingress, dns and network cluster operators all report Available and not Degraded. The cluster network maximum transmission unit (MTU) is what you expect it to be. And you know which endpoint publishing strategy the default IngressController is using, because that single field decides whether you can add a second shard without extra hardware.
Everything below was run against OCP 4.18.12 with the matching oc client, MetalLB Operator from the redhat-operators catalog on the stable channel, against a source estate of TKGI 1.18 on NSX-T 4.1 with NCP. I use oc rather than kubectl throughout, and only once will I say why: oc knows about Route, Project, SecurityContextConstraints and the other OpenShift API groups, and it carries the login and project context that kubectl has no concept of. Everything oc does, it does through the same API server.
Two of those output lines matter more than they look. 1400 is the cluster network MTU on a 1500 byte host network, because Geneve encapsulation, the tunnelling protocol OVN-Kubernetes uses between nodes, takes the difference. If your vSphere port groups run 9000 byte jumbo frames and you assumed the pod network inherited them, it did not. HostNetwork is the endpoint publishing strategy the installer picks on vSphere, meaning the default router pods bind ports 80 and 443 directly on the worker nodes and a keepalived managed VIP floats between them. That is why a second shard cannot simply be created the same way.
| What you had on TKGI | OpenShift equivalent | Who creates it | Watch out for |
|---|---|---|---|
| NCP creating an NSX-T virtual server per LoadBalancer service | MetalLB IPAddressPool plus L2Advertisement | You, once, as an Operator install | In layer 2 mode the pool must sit inside the node subnet |
| Kubernetes Ingress object handled by NCP | Route object handled by the Ingress Operator | You, per application | Ingress still works but generates unselectable Routes |
| One NSX-T load balancer per TKGI cluster | IngressController shards inside one cluster | You, per traffic class | Default shard keeps serving everything unless you exclude |
| NSX-T SNAT giving a namespace a predictable source IP | EgressIP custom resource on OVN-Kubernetes | You, per namespace | Silently unassigned until a node carries the egress assignable label |
| NCP annotations tuning per service load balancing | Route annotations on haproxy, or a shard with its own tuning | You, per route | Annotation names do not carry over, nothing is translated |
That table is the artifact to keep from this Part. Print it, put it beside the migration wave plan from Part 6, and use it to answer the question every application team asks in week one, which is what happened to the load balancer they never had to think about.
Step by Step, Routes, Shards and Load Balancer Addresses
Step 1, expose the first migrated service
Start with the stateless storefront from wave one. Two ways exist to publish it. oc expose generates a Route with a derived hostname and no TLS. oc create route edge gives you the hostname you want and terminates TLS at the router with the cluster wildcard certificate. Use the second one in anything you intend to keep.
A 503 from the router is almost never a routing problem. It means haproxy has a Route, resolved a Service, and found no endpoints behind it. Here the pods never started, because the image runs as user 0 and restricted-v2, the default Security Context Constraint (SCC), refuses it. That is the admission gap from Part 7 arriving at the exact moment you thought you were doing networking. Fix the image or bind the service account to nonroot-v2, then re-check endpoints before touching anything in the router.
Step 2, decide what happens to your Ingress objects
You will arrive with a pile of them. Our reference estate had 61 Ingress objects across three TKGI clusters. Apply one unchanged to OpenShift and it works, which is exactly why teams stop thinking about it.
Look at the generated name. checkout-xr7k2 carries a random suffix that changes if the Ingress is recreated, and the only label on it is one the Operator wrote. Anything that selects routes by name breaks on the next apply, and ingress sharding by route label cannot see it at all, because you cannot add your own labels to an object you did not create. Then look at the last two commands. An Ingress naming an ingressClassName that OpenShift does not own produces no Route, no warning and no event. It is simply ignored, and someone finds out during the cutover window.
Step 3, install MetalLB before you build the second shard
Order matters here and most walkthroughs get it backwards. Because the default IngressController already holds ports 80 and 443 on every worker through HostNetwork, a second shard needs an address of its own, and on vSphere that address has to come from somewhere. MetalLB is that somewhere. Install the Operator into metallb-system from OperatorHub, then create the three objects below.
Layer 2 mode answers ARP for the pool addresses from whichever node currently owns them, which gives failover but not load spreading. Border Gateway Protocol (BGP) mode spreads sessions across nodes and needs a peering session with your top of rack switches, plus a network team willing to grant an autonomous system number. Start on layer 2. Move to BGP when a measurement, not an opinion, says one node is the bottleneck.
Step 4, create the internal shard and exclude it from the default
One shard per traffic class is the pattern that survives. In the reference estate that means two: the default one serving customer facing routes on the apps wildcard, and an internal one serving back office routes on a separate wildcard that only resolves inside the corporate network. The second object below is not optional, and skipping it is the mistake in the field note further down.
Had you set endpointPublishingStrategy to HostNetwork on that shard, copying what the default controller does, the router pods would never have scheduled. Worth seeing the real message once, because it reads like a capacity problem and is not.
Step 5, give the namespace a predictable source address
This is the step nobody plans for and everybody needs. On TKGI, NSX-T applied source network address translation (SNAT) per namespace, so an application talking to an Oracle database left the cluster wearing one predictable address, and a firewall rule was written against it years ago. Move the workload to OpenShift without an EgressIP and it leaves wearing the node address, which changes whenever the pod reschedules, and the database connection is refused by a rule nobody remembers writing.
The EgressIP object itself selects namespaces by label and lists addresses that must be routable in the node subnet but outside any DHCP scope and outside the MetalLB pool. Overlap those ranges and you get a duplicate address on the wire, which vSphere will report as a MAC flap and your network team will report as your fault.
flowchart LR C1[External client] --> D1[Wildcard DNS apps domain] D1 --> V1[Ingress VIP on keepalived] V1 --> R1[Default router shard] C2[Internal client] --> D2[Wildcard DNS int domain] D2 --> V2[MetalLB layer 2 address] V2 --> R2[Internal router shard] R1 --> S[Service ClusterIP] R2 --> S S --> P[Pod on Geneve overlay]
Verification and Rollback
Four checks prove the whole chain, and each of them fails loudly rather than quietly, which is the point of choosing them.
A clean result looks like this. Every Route shows ADMITTED True against exactly one routerName, never two. External hostnames answer 200 with the wildcard certificate your browser already trusts. router-internal has an address from the MetalLB pool rather than pending. Outbound traffic from the migrated namespace presents the EgressIP, not a node address. And oc get clusteroperator ingress still shows Degraded False, because a shard that cannot schedule will degrade the whole ingress operator, not just itself.
Rollback is unusually cheap at this stage, which is the argument for doing all of it before any production DNS changes. Remove the routeSelector from the default IngressController and it re-admits every route within seconds, so a broken shard never means a dark application. Delete the shard IngressController and its Service and router pods go with it. For MetalLB, delete the L2Advertisement first and the IPAddressPool second, otherwise addresses stay advertised on the wire for a few seconds longer than you expect. Remove an EgressIP by deleting the custom resource; traffic reverts to node source addresses immediately, so re-open the firewall rule before you do it, not after. Keep the TKGI NSX-T virtual servers live and serving throughout. Nothing in this Part requires them to be switched off, and Part 20 handles the actual DNS cutover.
Networking Failures and Remediation
| What you see | Actual cause | Fix |
|---|---|---|
| 503 Application is not available from the router | Service has no endpoints, usually pods blocked by restricted-v2 | oc get endpoints first; fix the image or bind nonroot-v2 before touching ingress |
| EXTERNAL-IP stuck at pending on a LoadBalancer service | No load balancer provider on vSphere, and no error is raised | Install MetalLB, create an IPAddressPool and an L2Advertisement |
| Router pods Pending with no free ports for the requested pod ports | Second shard set to HostNetwork while the default already binds 80 and 443 | Switch the shard to LoadBalancerService, or pin it to separate infra nodes |
| One Route admitted by two routerName values | Default IngressController still matches the sharded label | Patch the default with a NotIn matchExpression for the shard label |
| MetalLB address assigned but nothing answers ARP for it | Pool sits outside the node subnet, which layer 2 mode requires | Move the pool into the node subnet or switch that pool to BGP mode |
| EgressIP created but ASSIGNED NODE stays blank | No node carries the k8s.ovn.org/egress-assignable label | Label at least two workers so the address survives a node reboot |
| An applied Ingress produces no Route and no event | ingressClassName names a controller OpenShift does not own | Drop the field or convert the object to a Route with your own labels |
Field Note from a Wildcard Certificate That Followed the Wrong Router
Saturday, 02:10. We had built the internal shard exactly as above, minus one command. I created the internal IngressController with its routeSelector and never patched the default controller to exclude the same label, because the documentation example only ever shows the new object and my reading was that a labelled route belongs to whoever claims it. It does not work that way. The default controller keeps serving every route it is not explicitly told to ignore, so six back office applications were admitted by both routers at once.
Nothing failed. That was the problem. Both routers answered, the internal wildcard DNS record happened to resolve to an address that reached the default router first, and every request landed on a router holding the apps wildcard certificate rather than the int one. Browsers threw a certificate name mismatch, curl threw an SSL error, and the back office team logged a P2 saying the platform had broken TLS. It took 40 minutes to stop looking at certificates and run the one command that shows routerName per route, at which point two entries in the BY column made the cause obvious. One patch, and the second entry disappeared inside 20 seconds.
Questions from the network review
Can we keep NSX-T doing the load balancing and point it at OpenShift. Yes, as an external load balancer in front of the ingress VIP, and plenty of estates run exactly that during transition. What you cannot do is have NSX-T create per service virtual servers automatically, because that was NCP, and NCP has no OpenShift equivalent.
Do we need MetalLB if everything is HTTP. No. Routes handle HTTP and HTTPS through the router shards and MetalLB adds nothing there. You need it for a second shard, and for non HTTP TCP or UDP services such as message brokers and database front ends that used to sit behind an NSX-T virtual server.
What replaces the NSX-T distributed firewall rules between namespaces. NetworkPolicy and AdminNetworkPolicy on OVN-Kubernetes, which is Part 16 and deserves its own runbook rather than a paragraph here. If the landing place you are weighing is VMware vSphere Kubernetes Service rather than OpenShift, the NSX-T posture carries over far more directly, and the TKGI to VKS guide covers that path.
Two Shards, One MetalLB Pool, and Route Objects Everywhere
My recommendation for a TKGI estate landing on OpenShift is narrow on purpose. Build exactly two IngressController shards, external and internal, and resist the pull to recreate one shard per old TKGI cluster; that habit is what Part 2 argued against and it costs you router pods and certificates for no isolation you actually enforce. Install MetalLB once, in layer 2 mode, with a single pool sized for the shard plus a handful of non HTTP services, and leave BGP for the day a measurement demands it. Convert every Ingress object to a Route as it migrates, with your own name and your own shard label, so the object your pipeline updates is the object the router reads.
On Monday, run the route admission check across all namespaces on whatever cluster you have, even an empty one, and confirm the BY column holds exactly one routerName per route. It takes ten seconds and it is the check I did not have on that Saturday. Part 14 takes this networked cluster and turns to projects, quotas, MachineSets and node scaling, where the TKGI plan model finally gets replaced by something you declare rather than order.
References
Red Hat, OVN-Kubernetes network plugin, OpenShift Container Platform 4.19
Red Hat, MetalLB Operator, Networking Operators, OpenShift Container Platform 4.18
Red Hat, Ingress sharding in OpenShift Container Platform, Networking
About The Author
DrJha