Our LDAP identity provider had been live for about twenty minutes. People were signing in with corporate credentials, the console rendered, and oc whoami returned the right username every time. Then I checked group membership, because all the RoleBindings I had applied that morning pointed at groups.

Nine groups existed in the directory. Every one of them was already named as a subject in a RoleBinding on the new cluster. OpenShift knew about none of them, and nothing in the console or the API said so. Authentication was working perfectly and authorization was silently empty, which is a much worse failure mode than a login error, because a login error tells you something is broken.

Authorization Models Compared, UAA Scopes Against OpenShift RBAC

TKGI splits authorization across two systems that barely talk to each other. UAA (User Account and Authentication, the OAuth2 identity service that ships inside Tanzu Operations Manager) decides who may create, resize, delete or fetch credentials for a cluster. Kubernetes RBAC (role based access control, the native permission model) inside each provisioned cluster decides what a person may do once they hold a kubeconfig. A user with the pks.clusters.manage scope can stand up a cluster and, by way of the credentials that command hands back, becomes an administrator inside it. A user with pks.clusters.admin can do that to every cluster in the estate. Those two scopes are mapped to LDAP groups with uaac group map, and that mapping lives in UAA, not in any cluster.

OpenShift collapses both layers into one. An OAuth server runs inside the cluster, identity providers are declared in a single cluster scoped OAuth custom resource, and everything downstream is ordinary Kubernetes RBAC plus a handful of OpenShift specific cluster roles. There is no outer control plane that decides who gets a cluster, because in the model established back in Part 2 you are not handing out clusters any more. Admission adds a third gate that TKGI mostly did not enforce, Security Context Constraints, and that gate is covered on its own in Part 7.

Before mapping anything, count what you actually have. This is the same discovery pass as Part 5, narrowed to identity. Note that these commands use kubectl, not oc, because the source side is upstream Kubernetes with no OpenShift client available. From Part 12 onward everything on the target uses oc, which is the same binary plus the OpenShift API groups, project switching and the oc adm subcommands you will need for group sync and policy.

Nineteen RoleBindings on a single namespace is not a permission model. It is sediment. Someone needed access in 2022, a binding was written, the person changed teams, and nobody revoked it because revoking access on a cluster that already works is a task with no deadline. Across the three clusters that number came to 214. When we rebuilt the same access on OpenShift from the actual team rosters rather than from the inherited objects, it came to 31.

Below is the mapping table this part exists to produce. Print it, argue about the middle column with your security team, and use it as the checklist when you configure the target. Every row is a construct that exists on TKGI today and has to land somewhere on OpenShift, including the two rows where the honest answer is that it lands nowhere.

Mapping TKGI Plans onto OpenShift Projects

A TKGI Plan is a template for a whole cluster. It fixes control plane and worker counts, VM sizing, whether privileged containers are permitted, which add-ons get installed, and which availability zones the nodes land in. Tenants pick a Plan, run tkgi create-cluster, and receive an entire Kubernetes control plane of their own. That is the unit of tenancy, and it is why a mid sized estate ends up with a dozen clusters nobody can patch on schedule.

OpenShift has no object that plays this role, and looking for one is the most common way teams lose a fortnight. What a Plan encodes decomposes cleanly into four separate OpenShift concerns. Compute shape becomes a MachineSet, optionally tainted so only one tenant schedules there. Capacity becomes a ResourceQuota and a LimitRange attached to the Project. Security posture becomes an SCC binding. Network isolation becomes NetworkPolicy, which Part 8 covered when it traded the NSX-T distributed firewall for OVN-Kubernetes. Write those four down per former Plan and you have your target tenancy design.

Deciding how much isolation each tenant genuinely needs is the hard part, because tenants will always ask for more than they can justify and the request costs them nothing. Push the question toward evidence: is there a regulator, an auditor or a contractual boundary that requires separation, or is this a preference. Three tiers cover almost every case I have met.

LDAP Wiring, OAuth Identity Provider and Group Sync

Wiring authentication is a short job. OpenShift reads identity providers from a single cluster scoped OAuth resource named cluster, and the bind password and certificate authority live as a Secret and a ConfigMap in the openshift-config namespace. Read the bind password from an environment variable so it never reaches your shell history or a Git commit.

Two details in that manifest decide how painful the next year is. Setting preferredUsername to uid rather than mail fixes what usernames look like forever, because mappingMethod: claim creates the User object on first login and that name is then baked into every RoleBinding, audit record and Project annotation. Changing it later means rewriting every binding. Pick the same attribute your UAA configuration already used, so that inherited documentation and runbooks still read correctly. Setting insecure: false with a real certificate authority is not optional in production, and it is the one line people copy from a lab and forget.

Now the part that catches almost everyone. An identity provider authenticates. It does not enumerate groups, and it never creates Group objects. Group membership is a separate synchronisation job that you run yourself with oc adm groups sync against an LDAPSyncConfig file, and until it runs, every group subject in every RoleBinding matches nobody. Red Hat documents this clearly, but it sits in a different chapter from identity provider configuration, and nothing in the login path warns you.

That first error is worth pausing on, because it is the normal state of a corporate directory rather than an exception. A contractor left, the account was removed from the people tree, and the group entry still carries a member attribute pointing at a distinguished name that no longer resolves. Setting tolerateMemberNotFoundErrors to true makes the sync succeed and is what most teams do. It also means directory rot is now invisible to you permanently. My preference is to leave it false, let the sync fail loudly the first time, and use that failure to get the directory cleaned, then decide.

Whichever you choose, this sync has to be scheduled, and a CronJob in a dedicated namespace running the oc image every thirty minutes is enough. Give its ServiceAccount permission to manage Group objects and nothing else, mount the sync config and bind password from a Secret, and pair it with oc adm prune groups so that groups deleted in the directory eventually disappear from the cluster. Read the pruning caveat in the field note below before you enable that second job.

Role Bindings per Tenant, and What to Take Away from Developers

OpenShift ships default cluster roles that already match how most teams actually work, so resist writing custom roles until you have proved the built in ones do not fit. Bind admin to the group that owns a Project and they can manage everything inside it, including RoleBindings for their own people. Bind edit to people who deploy but should not hand out access. Bind view to auditors, support and anyone who only reads. Those three cover the overwhelming majority of what those 214 inherited bindings were trying to express.

Bind to groups, always. A RoleBinding whose subject is a named user is a permission that will outlive that person’s time on the team, and it is exactly how a TKGI estate accumulates two hundred bindings nobody can explain. If the group does not exist in the directory, the correct response is to create it in the directory rather than to bind the individual as a stopgap, because a stopgap in access control has a half life measured in years.

Then there is self-provisioner. By default every authenticated user holds it through a cluster role binding on system:authenticated:oauth, meaning anyone who can log in can create Projects. Standard hardening advice is to remove it on day one with oc adm policy remove-cluster-role-from-group, and there is a real trap in doing so: unless you also set the rbac.authorization.kubernetes.io/autoupdate annotation on the binding to false, the cluster resets it to default on the next restart or upgrade and your change quietly evaporates.

Before you disable self provisioning, shape what a new Project looks like. Generate the default template with oc adm create-bootstrap-project-template, add a ResourceQuota, a LimitRange, a default deny NetworkPolicy and the tenant label the next section depends on, load it into openshift-config, and point the cluster Project configuration at it. Every Project created from that moment arrives governed instead of empty, which is the single most valuable hour of configuration in this whole part.

Quota and Isolation Across a Tenant Estate

A TKGI tenant with a large Plan had a hard ceiling: the worker node count in that Plan. Nothing they deployed could exceed it, because the cluster physically was the quota. On OpenShift, a tenant that holds four Projects has four independent ResourceQuotas, and four times the ceiling you thought you granted. This is the arithmetic error that turns a carefully sized cluster into an oversubscribed one within a quarter.

OpenShift solves it with an object Kubernetes does not have. ClusterResourceQuota, in the quota.openshift.io/v1 API group, selects Projects by label or by the annotation recording who requested them, aggregates consumption across all of them, and enforces one ceiling over the set. It is the closest thing OpenShift offers to a Plan sized allocation, and it is the object I would reach for first when translating Plan tiers.

Notice what that selector implies. A Project without the tenant: payments label is not covered by this quota, and it does not error, warn or appear anywhere in the describe output. It simply consumes cluster capacity outside the ceiling you believe you set. We found one such Project six weeks in, holding a forgotten load test at 14 CPU. Putting the tenant label into the project request template is the fix, and pairing it with a scheduled report that lists Projects carrying no tenant label is the belt and braces.

Add a LimitRange alongside every quota. A ResourceQuota on requests.cpu rejects any pod that does not declare a CPU request at all, which produces a confusing admission error on workloads that ran happily on TKGI without resource declarations. LimitRange supplies defaults so those pods are admitted with sensible values instead of bounced. That combination is what keeps a wave one migration from generating fifty support tickets on its first afternoon.

Field Note from a Three Cluster Identity Cutover

Forty minutes is what the empty group table cost me, and it was avoidable. I had budgeted a two hour window for identity on the target cluster, wired the OAuth resource in about fifteen minutes, applied the RoleBindings I had prepared the week before, and then spent the rest of that window convincing myself the LDAP filter was wrong. It was not. Login worked. Group sync had simply never been run, because I had read the identity provider chapter and not the group sync chapter, and those two chapters are far apart in the documentation. Once I found oc adm groups sync, the fix took four minutes.

Worse came a fortnight later, from the prune job I had been so pleased to automate. Our directory team renamed an Active Directory group during a routine tidy up, from reporting-analysts to bi-analysts, and told nobody outside their own change record. To the sync job, a rename looks exactly like a deletion followed by a creation. Our scheduled prune removed the old Group object at 03:00. A RoleBinding on the reporting Project still named reporting-analysts, still existed, still validated, and now matched an empty set. At 09:00 on Monday five analysts had a console that rendered perfectly and showed them zero Projects. Diagnosing that took most of an hour, largely because everything looked healthy.

Two changes came out of that morning and both have earned their keep. Group names used in RoleBindings are now covered by the same change control as a DNS record, agreed in writing with the directory team, and a rename requires a ticket that names the affected bindings. And the prune job writes a diff of removed groups to a channel the platform team actually reads, rather than silently succeeding. Neither is clever. Both would have turned a one hour outage into a message nobody had to act on.

Verdict on the rebuild question, since it is the one people put off. Rebuild access from team rosters. Do not migrate RoleBindings, and do not let a Velero backup that includes rbac.authorization.k8s.io restore them for you on the target. Going from 214 bindings to 31 took two afternoons with the four team leads in a room and a shared screen. Doing that same review on a running TKGI cluster had been on our backlog for three years and had never once reached the top.

Sync Groups Before Binding Anything

If you take one sequencing rule from this part, take that one. Configure the identity provider, run the group sync, confirm oc get groups returns real membership, and only then write a single RoleBinding. Doing it in that order removes an entire category of confusing failure, because from that point on a permission problem is a permission problem rather than a phantom subject.

Everything else in this part follows from treating the Project, not the cluster, as your unit of tenancy. Plans become quota plus node placement plus an SCC decision. Cluster lifecycle rights stop being something you hand to application teams. Multi Project tenants get one ceiling through ClusterResourceQuota instead of one per namespace. If a tenant genuinely needs a whole cluster, that is a legitimate answer, and the tier table above is how you make them prove it rather than simply asserting it. Readers weighing VMware vSphere Kubernetes Service instead of OpenShift as the landing place will find the same tenancy questions answered differently in the TKGI to VKS guide.

Your action for Monday: run the RoleBinding count loop from the first code block against every cluster in your estate and write the total on a whiteboard. Then book two hours with your team leads and rebuild that list from who actually needs access today. Whatever the gap turns out to be, it is the clearest argument you will ever have for why this migration is worth doing properly. Part 11 turns all of this into a target reference architecture for OpenShift 4 on vSphere.

References

• Syncing LDAP groups, Authentication and authorization, OpenShift Container Platform, Red Hat Documentation
• Projects, project request templates and self-provisioning, Building applications, OpenShift Container Platform, Red Hat Documentation
• ClusterResourceQuota [quota.openshift.io/v1], Schedule and quota APIs, OpenShift Container Platform, Red Hat Documentation
• Managing Cluster Access and Permissions, Tanzu Kubernetes Grid Integrated Edition 1.18, Broadcom TechDocs

About The Author

Dr. Pranay Jha

Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.

See author's posts