A candidate in one of my study groups asked whether Objective 3.6 wanted him to learn Istio properly or just recognise the name on an architecture diagram. Published wording is Create an implementation of service mesh, and the verb is carrying most of the weight in that sentence, because an implementation is a set of decisions you can defend under questioning, not a package you can spell.

Mesh decisions that belong to the platform team

Three decisions sit under this objective and only one of them involves typing. Does this estate need a mesh at all. Sidecar or ambient. One mesh per workload cluster or one mesh spanning several. Everything downstream, the values file, the gateway manifest, the injection label, falls out of those three answers.

Start with whether, because popular study advice gets this wrong in a specific and expensive way. You will read that you should install a mesh so that traffic between services is encrypted. On a VKS cluster that argument is weak. Antrea NetworkPolicy already restricts pod to pod traffic inside the cluster, and the NSX distributed firewall already restricts it at the hypervisor, and neither of those puts a proxy in the request path. A mesh earns its keep when you need one of four things that policy cannot give you: authorisation decisions based on workload identity rather than source address, weighted traffic splitting for a canary release, retry and timeout behaviour applied uniformly without touching application code, and per hop telemetry that survives a service being rewritten. If you cannot name which of those four you are buying, you are buying an Envoy process per pod and paying for it every day after.

Second, understand where Istio physically lands, because this is the single most common misreading I see in study notes. Istio is a VKS Standard Package. Standard Packages are Carvel style bundles pulled from a Broadcom repository and driven by the VCF CLI, and they install on the workload cluster. Harbor, which this series reaches in Part 19, is a Supervisor Service and shows up in the vSphere Client under Supervisor Services. Istio does not, and never will. If a question puts a mesh in the vSphere Client, that question is testing whether you know the difference.

That does not let the Supervisor off the hook. Which CNI the cluster runs was fixed in the cluster manifest against a ClusterClass the Supervisor published. Whether an ingress gateway can get an address depends on the load balancer wired in at enablement, sized back in Part 11. Whether nodes can pull Istio images depends on the registry trust set in Part 10. A mesh that refuses to come up is nearly always a Supervisor level object declining to cooperate, and the fastest diagnosis is to stop reading Istio logs and go and look at the namespace.

One naming note, because it will bite you in the CLI. VKS is the product formerly called TKG Service or TKGS, and the rename has not reached the plumbing. Package repositories live in the tkg-system namespace and ClusterClass names still carry tkg strings, so a tkg prefix is not evidence that you are on an old product.

flowchart TD
  A[Do you need L7 policy, canary split or per hop telemetry] -->|no| B[Stay with Antrea NetworkPolicy and NSX DFW]
  A -->|yes| C{Which CNI does the cluster run}
  C -->|Cilium| D[Istio add on will not install, rebuild the cluster on Antrea]
  C -->|Antrea| E{What is the latency budget per hop}
  E -->|tight, L4 only| F[Ambient mode, ztunnel per node]
  E -->|room for L7| G[Ambient mode plus waypoint per namespace]
  E -->|per pod control required| H[Sidecar injection per namespace]

Discovery before design

Two commands tell you whether a mesh is even reachable on this cluster. Run them against the workload cluster context, not the Supervisor. Versions in the lab for everything below:

If that last command returns cilium, stop. Broadcom lists the Istio add-on as incompatible with Cilium clusters in the VKS Standard Packages release notes, and the install does not degrade gracefully, it fails. There is no supported flag that works around it. You rebuild the cluster on Antrea or you go without a mesh, and since CNI is chosen at cluster creation and not changeable afterwards, that is a rebuild and a workload migration, not an edit.

Sidecar mode and ambient mode compared

Sidecar mode is what most people picture. Label a namespace, restart the workloads, and every pod comes back with a second container running Envoy. All traffic in and out of the pod goes through it. Ambient mode splits that job in two. A ztunnel DaemonSet on each node carries encrypted layer 4 traffic between pods, and a waypoint proxy, deployed only for the namespaces that need it, handles layer 7 work such as header routing and weighted splits. Pods themselves stay untouched, which means enrolling an application into an ambient mesh does not require restarting it.

Published Istio benchmark figures make the cost difference concrete rather than theoretical. At 1,000 HTTP requests per second with a 1 KB payload, a single sidecar proxy with two worker threads consumes about 0.20 vCPU and 60 MB of memory, a waypoint proxy about 0.25 vCPU and 60 MB, and a ztunnel proxy about 0.06 vCPU and 12 MB. Those are per proxy numbers at load, and the multiplier is what matters: sidecars multiply by pod count, ztunnel multiplies by node count.

Apply that to the reference estate. My application cluster runs 3 worker nodes carrying 62 pods across the meshed namespaces. Sidecar mode at the benchmark rate would be 62 proxies, so 12.4 vCPU and about 3.7 GB of memory reserved for plumbing, on nodes sized with 4 vCPU each. Ambient layer 4 on the same cluster is 3 ztunnels, so 0.18 vCPU and 36 MB. Real traffic is nowhere near 1,000 requests per second per pod, and my measured idle sidecar footprint was closer to 45 MB, but the ratio holds and the ratio is the design input. Ambient wins on arithmetic until you need layer 7 on many namespaces at once, at which point waypoints start reintroducing the cost you avoided.

Reference artifact: sidecar versus ambient decision matrix for a VKS workload cluster.

Costs a mesh imposes on a Supervisor estate

CPU is the cost people budget for. Addresses are the cost that actually stops a rollout. An Istio ingress gateway is a Service of type LoadBalancer, which means it consumes exactly one VIP from the pool wired at Supervisor enablement, allocated by AKO or by NCP depending on the networking mode you chose. That is not a problem for one cluster. Put a mesh on six workload clusters and you have added six VIPs to a pool that was sized in Part 13 before anyone mentioned a mesh. Hostname separation is free, because one gateway VIP fronts as many hostnames as you like through Gateway and VirtualService objects, so the honest number is one VIP per mesh, not one per application.

Certificates are the second hidden cost. Istio runs its own certificate authority by default and issues short lived workload certificates that have nothing to do with the trusted CA bundle you configured on the Supervisor. Broadcom supports bringing your own CA to the Istio package, and on any estate with a security team that audits certificate issuance you will be made to use it. Decide that before the mesh is carrying production traffic, because swapping the root afterwards means every workload certificate reissues at once.

Third cost, and the one that catches vSphere administrators specifically: an Istio namespace is not a vSphere Namespace. Sidecar injection is controlled by a label on a Kubernetes namespace inside the workload cluster. A vSphere Namespace lives on the Supervisor and governs resource quota, storage policy and VM classes. Labelling the wrong one does nothing at all, silently, which is the worst kind of nothing.

Failure worth rehearsing

Here is what a mesh install looks like when the design homework was skipped. Cluster built on Cilium because somebody read a blog post about eBPF, then asked for a mesh three weeks later:

Fix is not in that log. Fix is a new cluster on Antrea and a workload migration, roughly two days of work for us including the change record. Check the CNI before you promise anyone a mesh.

Field note, and what I would do differently

My worst mesh day was not an install. It was a PeerAuthentication object. Wanting a clean security posture, I applied strict mTLS at mesh scope, in istio-system, on a Friday afternoon, which tells you something about my judgement that year. Within about ninety seconds Prometheus dashboards went blank and half the alerting fired. Cause was not the meshed applications, they were fine. Prometheus was scraping targets that had no proxy, the mesh now demanded a client certificate on those connections, and every scrape failed. Forty minutes of a very bright red wall later I reverted to permissive at mesh scope and reapplied strict namespace by namespace, checking scrape targets after each one. That took a further two days and was completely uneventful, which is the point. Set mesh wide policy to permissive, tighten per namespace, and keep monitoring components either fully in the mesh or explicitly excluded, never straddling the boundary.

Exam focus for objective 3.6

Objective checkpoint

Service mesh call for this estate

My recommendation for the reference estate, stated plainly. Run one mesh, on one cluster, in ambient mode, on Antrea, with waypoints deployed only for the two namespaces that actually need weighted routing. Keep the PostgreSQL StatefulSet outside it entirely, because a database gains nothing from layer 7 features and loses something real to an extra hop on every query. Set PeerAuthentication to permissive at mesh scope and strict per namespace. Budget one VIP for the gateway and add it to the address plan before you install, not after. And move the estate to VKS 3.4 or later first, so that the mesh you are running is supported while it runs and not merely while you install it.

What to avoid: sidecar injection turned on across every namespace because it was one label and felt tidy. That decision costs vCPU per pod forever, adds a restart to every future workload change, and buys layer 7 capability for services that will never use it.

Tonight, in your own lab: run vcf package available list -n tkg-system against a workload cluster, find the Istio version your VKr supports, then read the cluster manifest and confirm which CNI it was built with. Write both down. Those two facts decide the entire implementation, and if you can produce them from memory for your own estate you are already answering this objective better than most candidates. Next Part starts Section 4 and gets hands on with Supervisor creation across NSX VPC, NSX segment, vDS and Avi. If you arrived here from a TKGI platform, the migration path is covered separately in the TKGI to VKS Series, and the product level walkthrough underneath this whole series lives in the VKS Series.

References

• VKS Standard Packages Release Notes, Broadcom TechDocs, for Istio package versions, VKr compatibility and the Cilium known issue.
• Introducing VMware vSphere Kubernetes Service 3.4, VMware Cloud Foundation Blog, for the introduction of Istio as a Standard Package with runtime support.
• Istio Performance and Scalability, for the sidecar, waypoint and ztunnel resource figures quoted above.
• VMware Cloud Foundation VKS Administrator Exam Guide, 3V0-24.25, for the published objective wording.

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts