, ,

vSphere Namespace Ingress and Egress Design Options (VCAP-VKS Exam Series, Part 13)

Objective 3.3 is an address design question, not a wizard question. How ingress and egress differ across the three Supervisor networking modes, which pool runs out first on a real estate, and what you can still change once the Supervisor is enabled.

VCAP-VKS Exam Series · Part 13 of 34

Ingress and egress are not one decision with two names. They are two address planes, owned by different objects, failing in different ways, and audited by different teams, and Objective 3.3 separates them on purpose. A candidate who has only ever ticked boxes in the namespace creation wizard will lose points here, because almost all of the evaluation happened at Supervisor enablement, weeks before any namespace existed.

Who this is for: You run a Supervisor with a dozen or so vSphere Namespaces on it, you can explain what a LoadBalancer Service does, and nobody has yet made you defend an address plan in front of a network architect. This Part covers Objective 3.3, published wording Evaluate the architecture of vSphere namespaces with ingress and egress options. Terms on first use: a vSphere Namespace is the resource and policy boundary a Supervisor hands to a platform team; SNAT is source network address translation; NCP is NSX Container Plugin, the component that turns Supervisor objects into NSX objects; AKO is the Avi Kubernetes Operator; a VPC is an NSX Virtual Private Cloud, an independent routing domain inside an NSX project; TGW is a transit gateway. Reference estate is the one carried since Part 3: VCF 9.0, one workload domain, three vSphere Zones, NSX with VPC networking available, Avi in the load balancing path, VKS 3.3.x supplying the Kubernetes releases.
Key takeaways: Objective 3.3, published wording Evaluate the architecture of vSphere namespaces with ingress and egress options. Ingress is a pool of virtual IP addresses consumed per exposed Service; egress is a pool of source addresses consumed per namespace, and it is the smaller pool that caps your namespace count. On NSX segment networking both pools are chosen at Supervisor enablement and cannot be widened afterwards. On NSX VPC networking a namespace gets three subnet access modes, Private, Public and Private Transit Gateway, and only Public spends an address from the project external IP block. Broadcom documents that VM Service virtual machines land on the Private subnet by default and that the default VM SubnetSet and VM interface network cannot be changed. Read the exposure matrix below as the artifact.

Ingress and egress as two separate address planes

Most people learn ingress first because it is visible. Someone deploys a Service of type LoadBalancer, an address appears in the EXTERNAL-IP column, a browser reaches the application, and that feels like the whole networking story. Egress is invisible by design. A pod opens a connection to an Oracle listener or a corporate proxy, something rewrites the source address on the way out, and nothing in the pod ever knows. Both planes draw from address ranges you sized once, but they scale on completely different multipliers.

Ingress scales with exposed Services. Twelve namespaces running two exposed Services each consume twenty four addresses, and a team that discovers Ingress objects and starts fronting six applications behind one controller drops that number back to twelve. Ingress consumption is elastic and it responds to application design. Egress scales with namespaces, and on NSX segment networking the default is one SNAT address per namespace whether that namespace runs one pod or four hundred. Egress consumption is rigid and it responds only to how many tenants you onboard.

That asymmetry is the whole objective in one sentence. Study material tends to treat the ingress range as the important one because it is the one people run out of in a lab where every demo is a LoadBalancer Service. On a production estate with tenant onboarding as the growth driver, egress is what pins you, and egress is also what your security team keeps in a firewall rule, which means changing it is a change ticket rather than a config edit. Evaluate the two planes with different questions: for ingress, ask how many applications will be exposed and whether anyone will consolidate behind Ingress objects; for egress, ask how many tenants you will onboard in three years and whether each one truly needs a distinct source address.

One naming note that will follow you through Broadcom documentation and CLI output. VKS is the product formerly called TKG Service or TKGS, and the rename is not finished in the plumbing. Namespace and cluster objects, API groups and log paths still carry tkg in many places, so a resource name with tkg in it is not evidence that you are looking at an old product.

flowchart LR
  subgraph IN [Ingress plane, scales with exposed Services]
    A[Client on enterprise network] --> B[Virtual IP from ingress pool]
    B --> C[LoadBalancer Service or Ingress object]
    C --> D[Pod or node endpoint]
  end
  subgraph OUT [Egress plane, scales with namespaces]
    E[Pod in vSphere Namespace] --> F[SNAT rule on the namespace gateway]
    F --> G[Source address from egress pool]
    G --> H[Database or proxy outside the Supervisor]
  end
Two planes, two multipliers. Ingress grows with applications, egress grows with tenants.

Namespace exposure paths by networking mode

Part 12 established that a namespace inherits its network model from the Supervisor rather than choosing one. This table is the consequence of that inheritance, expressed as exposure paths instead of models. Keep it next to your design document, because every row is a question a reviewer will eventually ask you, and the last column is the one that decides whether an answer costs a config change or a rebuild.

Exposure needvSphere networking with Avi or Foundation LBNSX segment networkingNSX VPC networking
Inbound to a LoadBalancer ServiceVirtual IP from the Supervisor wide VIP range, allocated by AKOVirtual IP from the ingress CIDR, allocated by NCP on the namespace tier 1Address from the project external IP block via the VPC connectivity profile
Inbound HTTP routing behind one addressAvi acts as the Ingress data path, one VIP fronts many hostsSame ingress CIDR, one VIP fronts many hostsExternal subnet, one VIP fronts many hosts
Outbound to enterprise networksWorkload network routing, no Supervisor owned translationSNAT on the namespace tier 1, source from the egress CIDRSNAT at the VPC, source from external or private transit block by access mode
Distinct source identity per namespaceNot available, all namespaces share the workload subnetDefault behaviour, one SNAT address per namespacePer VPC, and NCP creates one VPC per namespace when none is supplied
Namespace to namespace inside the SupervisorFlat, same subnet, policy onlyRouted through tier 1 and tier 0, no address spendPrivate Transit Gateway subnet, reachable across VPCs in the same project
VM Service virtual machine placementWorkload network portgroupNamespace segmentPrivate subnet by default, and the default VM SubnetSet cannot be changed
Editable after Supervisor enablementVIP range is an Avi or FLB setting, not a namespace settingNo, ingress and egress CIDRs are fixed at enablementConnectivity profile and IP blocks are project level and editable

Two rows in that table carry most of the design weight. Row four is why people choose NSX in the first place, because a distinct source address per namespace is what makes a firewall rule per tenant possible at all. Row seven is why people regret their first Supervisor, because on segment networking the ranges that feed rows one and three are frozen the moment enablement completes. If you want the mechanics of tier 0, tier 1 and project scoping rather than the Supervisor view of them, the NSX series owns that ground and this series does not repeat it.

Egress identity and SNAT address sharing

Egress identity means one thing to a platform engineer and something else to a security engineer, and Objective 3.3 sits exactly on that seam. To a platform engineer it means outbound connectivity works. To a security engineer it means a log line at the perimeter can be attributed to one tenant without asking anyone. NSX segment networking gives you the second interpretation for free by handing every namespace its own SNAT address, and that generosity is precisely what burns through an egress CIDR.

NSX VPC networking changes the shape of that trade. Broadcom documents three subnet access modes inside a VPC: Private subnets, which are the default and are reachable only inside the VPC; External subnets, which draw public or externally routed addresses from the project external IP block and serve LoadBalancer Services, Ingress and the egress of the NSX project; and Private Transit Gateway subnets, which are reachable by workloads in other VPCs inside the same project and which back SNAT on egress of the VPC and for vSphere Pods. NCP builds a VPC for system namespaces in the NSX system project, a separate VPC for Supervisor Services, and an individual VPC per vSphere Namespace whenever a VPC was not supplied at workload domain creation. Read that carefully, because it means the per namespace identity you wanted on segment networking arrives on VPC networking as a routing domain rather than as an address, and a routing domain costs nothing from your routable block.

Before you evaluate any of this on your own estate, pin down what you are actually running. Design questions on this exam are answered against a stated configuration, and the same is true of a real design review.

# Tested against VCF 9.0, vSphere Supervisor 9.0, VKS 3.3.1 $ kubectl version –client Client Version: v1.33.1 Kustomize Version: v5.6.0 # Which NSX custom resources this Supervisor actually exposes. # Take the group and version from your own output, not from a blog post. $ kubectl api-resources | grep nsx # Password comes from the environment, never from the command line. $ kubectl vsphere login –server $SUPERVISOR_VIP –vsphere-username $VSPHERE_USER –insecure-skip-tls-verify

Now the failure that teaches the objective. On a lab Supervisor built with NSX segment networking and a deliberately mean ingress CIDR, I created namespaces until the pool was gone and then asked for one more exposed Service.

$ kubectl -n ns-payments get svc web-lb NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE web-lb LoadBalancer 10.96.114.22 <pending> 80:31284/TCP 9m14s $ kubectl -n ns-payments describe svc web-lb … Events: Type Reason Age From Message —- —— —- —- ——- Warning SyncLoadBalancerFailed 9m service-controller Error syncing load balancer: failed to ensure load balancer # Fix: free an address or widen the pool. On segment networking the # ingress CIDR is not widenable, so the only real fix is to reclaim # addresses by consolidating Services behind one Ingress object.

Two things about that output matter for the exam. First, EXTERNAL-IP sitting at pending forever is an allocation failure, not a health failure, and it never resolves on its own. Second, the Reason string SyncLoadBalancerFailed comes from upstream Kubernetes, while the tail of the Message is written by NCP or AKO and its exact wording moves between builds, so memorise the Reason and treat the Message as a hint. Trace it back the way this series always does: the object that failed is a Service in a namespace, but the object that caused the failure is the Supervisor, because the ingress CIDR was set at enablement and no namespace can widen it.

Production gotcha: One load balancer per Supervisor, and a VPC must use the same load balancer the Supervisor uses. A Supervisor configured with the NSX load balancer cannot consume a VPC that was built for Avi. That constraint is easy to miss during a paper design and expensive to discover after an NSX project has been handed to a tenant team.

Address budget arithmetic on the reference estate

Design arguments about ingress and egress collapse into agreement the moment somebody counts. So I counted. Same reference estate, same twelve namespaces, two LoadBalancer Services in each, rebuilt four ways, and I recorded only one number: how many externally routable addresses the estate consumed once everything was green.

Routable addresses consumed by twelve namespaces Two LoadBalancer Services per namespace. Reference estate rebuilt four ways on VCF 9.0 with VKS 3.3.1. vDS with Avi 24 addresses NSX segment 36 addresses NSX VPC default 36 addresses NSX VPC plus TGW 24 addresses Ingress accounts for 24 in every row. Only the egress plane moves, and it moves by 12, one address per namespace.

Ingress is a constant twenty four in all four rows because two exposed Services per namespace is two exposed Services per namespace regardless of who allocates the address. Everything that moves in that chart is egress, and it moves in units of one per namespace. Put that against a realistic range and the arithmetic gets uncomfortable fast. A slash 27 egress CIDR carries thirty two addresses, of which roughly thirty are usable once the range is realised, so a slash 27 caps you near thirty namespaces on segment networking no matter how generous your ingress range is. A slash 26 doubles that to about sixty two. Nobody thinks of the smaller field as the limiting one, and that is exactly why it limits people.

Zonal deployments add a second kind of arithmetic that candidates confuse with this one. Broadcom documents that namespace resources on a three zone Supervisor are drawn from all three underlying vSphere clusters in equal parts, so dedicating 300 MHz of CPU takes 100 MHz from each cluster. That splitting applies to compute, not to addresses. A namespace on a three zone Supervisor still consumes one egress address, not three. Exam items like putting those two facts next to each other and seeing whether you multiply the wrong one.

Field note: A customer estate I reviewed last year had a slash 27 egress CIDR and twenty six live namespaces on NSX segment networking. A new business unit arrived wanting fourteen more. Four addresses were left. Because the egress CIDR is fixed at Supervisor enablement, the only honest options were a second Supervisor or a rebuild of the first, and the rebuild carried a six hour outage window plus three weeks of change approval because two of the existing egress addresses were named in perimeter firewall rules owned by another team. Total elapsed cost of a field nobody discussed for more than ninety seconds during the original design: about a month. Sizing that field takes one meeting. Repairing it takes a quarter.

Here is where the popular advice is wrong. Nearly every walkthrough tells you to give each namespace its own routable egress address so security can attribute outbound traffic, and that advice made sense when segment networking was the only NSX option. On VPC networking it is the expensive default. Put namespace egress on a Private Transit Gateway subnet, let the VPC boundary carry tenant identity, and do attribution at the centralised transit gateway where you have the VPC context anyway. Same audit answer, twelve fewer routable addresses in my test, and an address block you can hand back to the network team.

Exam focus for objective 3.3

Objective 3.3, Evaluate the architecture of vSphere namespaces with ingress and egress options: What this objective expects you to be able to do is look at a stated topology and say which exposure paths exist, which address pool feeds each one, and which of them is fixed. Because it lives in the Plan and Design section, it favours item types that present a scenario and ask you to choose or arrange rather than to recall: multiple selection multiple choice where several exposure statements are true and several are true only on one networking mode, matching between a networking mode and the object that allocates its addresses, and drag and drop or build list where you order design decisions and the wrong order puts a namespace decision before the Supervisor decision that constrains it. The trap that catches experienced admins is the assumption that anything visible in the namespace creation wizard is a per namespace choice. Ingress and egress ranges on segment networking are Supervisor properties surfaced at namespace scope, and a question that offers you a namespace level remedy for an exhausted pool is offering you the wrong answer on purpose. Second trap, smaller but common: assuming NSX VPC networking always costs more routable addresses than segment networking. It costs the same by default and less when egress sits on a Private Transit Gateway subnet.

Objective checkpoint

Three original questions from the published objective wording:

1. A Supervisor on NSX segment networking has an ingress CIDR of slash 27 and an egress CIDR of slash 27. Forty vSphere Namespaces are planned, each exposing one Service. Which pool is exhausted first, and can it be widened without redeploying the Supervisor?
Answer: Both pools are consumed at roughly the same rate in this specific case because one Service per namespace makes ingress and egress scale together, and neither CIDR can be widened after enablement. Reasoning: The rate depends entirely on Services per namespace, so the question is testing whether you check that ratio before answering, and both ranges are enablement time properties on segment networking.

2. A tenant needs workloads in namespace A to reach workloads in namespace B, where both namespaces sit in separate VPCs inside one NSX project, and neither workload should consume an address from the project external IP block. Which subnet access mode satisfies this?
Answer: Private Transit Gateway. Reasoning: Private subnets are reachable only inside their own VPC and External subnets draw from the external IP block, so only the Private Transit Gateway mode crosses VPCs inside the project without spending an externally routable address.

3. A design calls for a Supervisor using the NSX load balancer, and a tenant team has already built a VPC configured for Avi Load Balancer. What is the outcome?
Answer: That VPC cannot be used by this Supervisor. Reasoning: Only one load balancer is supported per Supervisor and the VPC must be configured with the same load balancer the Supervisor uses, so the VPC has to be rebuilt or the Supervisor design changed.

Ingress and egress call for this estate

My pick for the reference estate is NSX VPC networking, with External subnets used only where something is genuinely published to the enterprise and Private Transit Gateway subnets carrying namespace to namespace traffic and VPC egress. Tenant identity comes from the VPC boundary rather than from a routable address, which is the same audit story at lower address cost, and the connectivity profile and IP blocks stay editable at project level instead of freezing at Supervisor enablement. What I would avoid on a greenfield build is NSX segment networking chosen out of familiarity, because it hands you a distinct egress address per namespace that feels like a feature right up to the day the CIDR runs dry and the repair is a rebuild. If you are on vSphere networking with Avi and cannot move yet, accept that you have no per namespace source identity and solve attribution with policy and logging instead of pretending the address plane will give it to you.

Tonight, in your own lab, do one thing: count. Write down your ingress range, your egress range, your current namespace count and your current count of exposed Services, then divide. If either quotient says you have fewer than two years of headroom at your real onboarding rate, you have found the design decision this objective is actually about, and you have found it while it is still a spreadsheet rather than a rebuild. Readers who want the product level walkthrough underneath all of this rather than the exam view of it should sit the VKS complete guide alongside this series, and anyone who arrived here from a Tanzu Kubernetes Grid Integrated estate should read the TKGI to VKS guide first, because migration reshapes an address plan in ways an exam never will. Part 14 walks the enablement process itself, which is where every range discussed here gets committed.

VCAP-VKS Exam Series · Part 13 of 34
« Previous: Part 12  |  Guide  |  Next: Part 14 »

References

About The Author


Discover more from Journal of Intelligent Infrastructure

Subscribe to get the latest posts sent to your email.

Leave a Reply

Your email address will not be published. Required fields are marked *

Architect’s Toolkit

About the Author

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

Discover more from Journal of Intelligent Infrastructure

Subscribe now to keep reading and get access to the full archive.

Continue reading