NVIDIA AI factory stack · Overview
In the last few weeks NVIDIA has published four open-source projects that look unrelated at first: a Kubernetes management plane (NVKE), a cluster recipe tool (AICR), a security runtime for AI agents (OpenShell) and a semantic layer for enterprise data (Auto Ontology). Read together, they describe one stack, from bare-metal GPUs up to an agent that gives a trusted answer. This post explains what each project does, where it sits in that stack, how mature it is, and which one to look at first for your role.
The big pictureWhat problem do these four projects solve together?
Running AI in production is no longer one problem. A platform team has to turn racks of GPUs into Kubernetes clusters that customers or internal teams can use. Each cluster then needs the right drivers, operators and inference software, in versions that actually work together. Once agents run on that platform, someone has to decide what each agent is allowed to touch. And when an agent answers a business question, the number has to be right, not just fast.
Each of the four projects takes one of those jobs. Two belong to the platform team, two to the teams building agents:
Figure 1. Where each project sits. The two red layers are infrastructure the platform team runs; the two grey layers govern what agents do and what they know.
You do not need all four. AICR works on clusters you already have, OpenShell does not need Kubernetes at all, and Auto Ontology runs on any Postgres-capable environment. The stack view is useful because it shows which problem each one is not trying to solve.
Layer 1 · ClustersWhat is NVKE?
NVKE is a management plane for running Kubernetes clusters on NVIDIA GPU infrastructure, usually bare metal. The people who install it are platform teams that want to offer managed Kubernetes to their own customers, internal or external. Each customer gets a workload cluster with its own control plane, etcd and node pools, and uses it through the normal Kubernetes API and kubectl.
It is not another Kubernetes distribution. NVKE works like a factory that builds itself:
- A temporary seed cluster bootstraps the first, permanent cluster.
- The management cluster runs the NVKE management plane and infrastructure controllers. The
nvkectltool creates it. - Workload clusters are created on demand for customers. The management plane asks NICo for machines through a Cluster API provider (CAPNICo), and a node agent (
nvke-agent) runs on every GPU node.
Figure 2. How NVKE builds clusters. A temporary seed cluster creates the management cluster; the management plane then asks NICo for machines and creates an isolated workload cluster for each customer. Each GPU node proves its identity with its TPM before it joins.
Two details matter for anyone running shared GPU infrastructure. Nodes prove their identity with SPIFFE/SPIRE and their TPM before they receive credentials to join a cluster, and etcd encryption can be switched on per cluster. Workload clusters can optionally run AICR as their accelerated runtime, which is the link to the next layer.
If you know VKS on VMware Cloud Foundation, the idea will feel familiar: a supervisor-style cluster that creates tenant clusters. The difference is the target. NVKE is aimed at GPU cloud and “GPU-as-a-service” providers running NVIDIA hardware directly, not at a virtualised private cloud.
Layer 2 · RuntimeWhat is AICR?
AI Cluster Runtime (AICR) answers a question every GPU Kubernetes team has asked: which driver, GPU Operator, network operator, kernel setting and inference stack versions actually work together on this cloud, with this GPU and this OS? AICR ships the answer as a recipe, a version-locked configuration that has been tested for a specific environment.
AICR does not create clusters and does not run a service you depend on. You bring the cluster (EKS, AKS, GKE, OKE, LKE, OpenShift, BCM, Metal3 or Kind) and your deployment tool, and AICR generates the configuration your tool deploys. The workflow, step by step:
Figure 3. AICR’s flow. The recipe is built up from layered overlays for your exact cloud, GPU, OS and workload, rendered for your deployment tool, then checked on the live cluster in three validation phases.
Configure. aicr recipe takes your target: cloud service, GPU (A100 to GB300), OS, workload intent (training or inference) and optionally a platform such as Dynamo, NIM, Kubeflow, Run:ai or Slurm. It resolves a recipe from layered overlays.
Deploy. aicr bundle renders that recipe for Helm, Helmfile, Argo CD or Flux, one folder per component, ready for GitOps.
Validate. aicr validate checks the live cluster against the recipe in phases: deployment, performance (training and inference) and conformance.
Two more commands are worth knowing early. aicr snapshot runs a Kubernetes Job that records the cluster’s real state (GPUs, drivers, kernel, operators), and aicr diff compares two snapshots to find drift between clusters or over time.
# install the CLI
brew tap NVIDIA/aicr && brew install aicr
# or
curl -sfL https://raw.githubusercontent.com/NVIDIA/aicr/main/install | bashThe Dynamo connection is where AICR becomes practical for inference teams. NVIDIA’s own walkthrough packages Dynamo and everything under it into one AICR recipe, then validates the runtime. It covers both routing options: Dynamo’s own Frontend router, and the Kubernetes Gateway API Inference Extension with Dynamo’s Endpoint Picker. If you have read the llm-d post, this is the NVIDIA-side equivalent of that “what has to be installed underneath” problem.
Supply chain is treated seriously. Recipes are signed by NVIDIA CI so their origin can be checked offline, images carry SLSA Build Level 3 provenance and signed SBOMs, and aicr verify checks a bundle before you deploy it.
Layer 3 · Agent securityWhat is OpenShell?
The first two layers are about infrastructure. OpenShell is about the agents that run on it. Its starting point is simple: an agent cannot be trusted to govern itself. Agents change their behaviour as they work, so the rules about what they can reach (files, internet services, internal systems, even physical devices) have to be enforced by the infrastructure around them, not by the agent’s own prompt or code.
OpenShell was first announced at GTC in March 2026 as part of NVIDIA’s Agent Toolkit. The newer OpenShell Reference Architecture shows how to run it at scale. It has four parts:
| Component | What it does |
|---|---|
| Gateway | Control plane. Creates and removes sandboxes and holds the policies that apply to them. |
| Supervisor | Runs outside the sandbox. Checks every outbound request against policy and controls access to credentials, so the agent never holds secrets directly. |
| Sandbox | Isolates the agent’s workload and limits what files and processes it can touch. |
| Policy prover | Uses formal verification to prove a policy cannot grant more than an organisation-wide limit allows. |
Figure 4. OpenShell’s enforcement path. The agent never talks to the outside world directly: the supervisor checks each request against policy, lets the approved call through with the right credentials, and blocks the rest. Policy comes from the gateway and is checked by the policy prover.
The model is zero trust: every agent is treated as untrusted for its whole run and gets access only at the moment a task needs it. Supervisor middleware can inspect, block or change network traffic, and gateway interceptors let you add your own governance without changing OpenShell itself. Those are also the points where your existing identity, authorisation, data-protection and audit systems plug in.
The reference design runs OpenShell on NVIDIA Vera CPUs, with models on the Vera Rubin platform. An optional hardware layer, NVIDIA Sentry on BlueField-4 DPUs, watches requests, network and file activity from outside the host and can quarantine a misbehaving agent within milliseconds. NVIDIA and CoreWeave report large gains in sandbox start time and agent density on Vera; treat those as vendor figures until you have measured your own.
Layer 4 · Data contextWhat is Auto Ontology?
Ask an agent “what was revenue for product X in Karnataka last March?” and it can write valid SQL in seconds. The risk is that the SQL is valid and the number is wrong: the wrong revenue column, a missing filter on returns, a join that double-counts. Auto Ontology tackles that problem. It is an Apache 2.0 Python library that builds a semantic layer over enterprise data so agents use certified business definitions instead of guessing from column names. NVIDIA announced it in October 2026 as part of a wider set of agent data tools that also includes Nemotron Retriever and Kumo models.
The ontology holds three kinds of definition:
- Terms: business concepts such as Customer or Invoice Line, with descriptions, synonyms and the columns they map to.
- SQL attributes: reviewed SQL expressions, such as margin or a rolling total, so every answer calculates a measure the same way.
- Relationships: joins between tables, including bridge tables, inferred from keys and data.
It is built in three steps. Generate: an ingestion service reads schemas, profiles columns and drafts terms and joins into Postgres with pgvector. Certify: people test the draft with real questions and approve the definitions that are right. Answer: for each question the agent finds matching terms, writes SQL, checks it, runs it and returns the answer together with the SQL and the steps it took. If the ontology cannot support a question, for example return on ad spend when there is no advertising data, it says so instead of inventing a query.
Figure 5. How Auto Ontology answers. The agent matches the question to certified terms, attributes and joins, writes and checks SQL against them, and returns the answer with the SQL and steps, so a person can verify it. Values are illustrative.
Agents reach it through an MCP server with read-only tools such as search_terms, check_answerable and ask_question, each running with the signed-in user’s permissions, or through a CLI or REST endpoint. It connects to ten database engines including Snowflake, Databricks, PostgreSQL and DuckDB, and its models default to NVIDIA NIM but can point to any OpenAI-compatible endpoint.
Side by sideHow do the four compare?
| Project | Layer | Main users | Runs on | Maturity |
|---|---|---|---|---|
| NVKE | Cluster lifecycle | GPU cloud and platform teams | Bare metal through NICo (default provider) | Preview, not for production |
| AICR | GPU runtime configuration | Platform and MLOps teams | Any supported Kubernetes (EKS, AKS, GKE, OCP and more) | Usable today |
| OpenShell | Agent sandbox and policy | Security and agent platform teams | Runtime; reference design on Vera, optional BlueField-4 | Released, reference architecture new |
| Auto Ontology | Semantic layer for data agents | Data and analytics agent builders | Docker Compose or Kubernetes, Postgres + pgvector | New, no outside contributions yet |
Where to startWhich one should you look at first?
- You run GPU Kubernetes clusters today (EKS, OpenShift, VKS-style setups): start with AICR. It works on the clusters you already have, and
aicr snapshotplusaicr validatewill tell you quickly whether your current stack matches a known-good recipe. - You are building a GPU cloud or GPU-as-a-service offer: evaluate NVKE on a single machine now, and track it until upgrades and backup/restore are documented.
- You are putting agents near real systems: read the OpenShell design even if you never use Vera hardware. The split between gateway, supervisor and sandbox is a good model for any agent platform.
- You are building “ask your data” agents: try Auto Ontology against a copy of one warehouse and see how many questions it can answer correctly after certification.
Quick answers
Do I need NVKE to use AICR?
No. AICR is bring-your-own-cluster and supports the major managed Kubernetes services, OpenShift and bare-metal installs. NVKE can use AICR in its workload clusters, but AICR does not depend on NVKE.
Is NVKE a replacement for VKS or OpenShift?
Not today. It targets providers running NVIDIA GPU hardware directly through NICo, and it is in preview. VKS and OpenShift remain the platforms for most enterprise private clouds.
Does OpenShell need NVIDIA Vera or BlueField hardware?
No. Those are part of the reference architecture for running many agents at high density. The runtime is open source, and the BlueField-4 Sentry layer is optional.
Does Auto Ontology replace my Databricks or Snowflake semantic layer?
No. It is designed to work alongside existing semantic models and ontologies, connect to them and reconcile conflicting definitions, then serve that context to any agent through MCP.
- Decide which layer your actual problem is in: clusters, runtime, agent control or data meaning.
- For NVKE, use the no-GPU evaluation path first, and do not expose its API before authentication is configured.
- For AICR, take a snapshot of an existing cluster and compare it with the matching recipe before changing anything.
- For OpenShell, write down what each agent should be allowed to reach before you write a policy.
- For Auto Ontology, keep the backend and ingestion services on a private network and budget time for human certification.
- Treat all vendor performance numbers as a starting point and measure on your own workload.
Sources: NVKE README (github.com/dsx-ai-factory/nvke); AICR README (github.com/NVIDIA/aicr) and NVIDIA Technical Blog, “Build a Validated Kubernetes Runtime for NVIDIA Dynamo”; NVIDIA Technical Blog, “How NVIDIA OpenShell Reference Architecture Secures and Governs AI Agent Workloads at Scale”; NVIDIA GTC 2026 Agent Toolkit announcement; Auto Ontology README (github.com/NVIDIA/auto-ontology); The New Stack coverage of NVIDIA’s agent data tools, October 2026. Performance figures for Vera and OpenShell are as reported by NVIDIA and CoreWeave and have not been independently verified. Project status reflects the repositories as of November 2026.
About The Author
DrJha