NVIDIA AI factory stack · Bonus: the infrastructure view

Everyone is explaining what NVIDIA Auto Ontology does. This post is about what it takes to run it. The data team chooses the tool; the platform team gets the ticket that says “deploy Auto Ontology, connect it to the warehouse, and make it available to our agents”. That ticket involves more than helm install: GPU or API decisions for four models, a database that becomes business-critical, network paths into your data platforms, and a security boundary that the default chart does not draw for you. Here is the whole job, from an infrastructure architect’s point of view.

If you have not read Part 4, the short version is: Auto Ontology builds a graph of certified business definitions (terms, reviewed SQL measures and joins) over your SQL databases, and agents use it to write correct queries. This post does not re-explain that. It is about the platform underneath.

InventoryWhat exactly are you deploying?

NVIDIA publishes three images and a Helm chart on NGC, all versioned together (1.2.0 at the time of writing). Here is what the chart creates in your namespace, and what sits around it:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia && helm repo update
helm search repo nvidia/auto-ontology --versions

# secrets in a file (chmod 600, not in git): defaultModelsApiKey, postgresPassword,
# adminEmail, adminPassword; self-service sign-up is disabled, so the admin is required
helm install auto-ontology nvidia/auto-ontology --version 1.2.0 
  -f auto-ontology-secrets.yaml -f production.yaml

HardeningWhich defaults must you change?

The chart is built so that one person can try the product quickly. That is reasonable, but it means the defaults are the opposite of what a shared platform needs. These are the changes I would put in production.yaml or around the chart before anyone outside the platform team can reach it:

SecurityWhere is the security boundary?

This is the section your security review will focus on, so it is worth getting the model right. The project documents two facts that define the boundary: the backend trusts the user identity the frontend forwards in a header, and the ingestion service’s ingest, delete, reset and compile endpoints have no authentication of their own. Inside the namespace, anything that can reach those ports is effectively an administrator.

A NetworkPolicy along these lines closes the private zone. It uses the chart’s real component labels; check which components call each other in your version before applying it, and add the MCP server’s label if you deploy it in the same namespace:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: auto-ontology-private-zone
spec:
  podSelector:
    matchExpressions:
      - key: app.kubernetes.io/component
        operator: In
        values: [backend, ingestion, postgres]
  policyTypes: [Ingress]
  ingress:
    - from:
        - podSelector:
            matchExpressions:
              - key: app.kubernetes.io/component
                operator: In
                values: [frontend, backend, ingestion, mcp]

Two more boundary decisions belong in the design document. First, what leaves the building: building the ontology sends schemas, column profiles and sample values to the reasoning model, and every question goes to a model too. If your data classification forbids that for a hosted API, self-host the models. Second, source access: give Auto Ontology read-only database accounts scoped to the schemas you actually want agents to see. Every chat and MCP call runs with the signed-in user’s permissions inside Auto Ontology, but the database account it uses decides the outer limit.

GPUsHosted models or self-hosted NIM?

Auto Ontology needs four model roles, each configured with its own endpoint, key and model name, all falling back to one default:

For self-hosting, size the reasoning model first; the others are small next to it. A 30B mixture-of-experts model holds all 30 billion parameters in GPU memory even though only about 3 billion are active per token, so memory, not compute, decides the GPU. In FP8 the weights take roughly 30 GB, which leaves room for KV cache on a single 80 GB-class GPU. Treat that as a starting estimate and run it through the GPU Sizing Calculator with your expected number of analysts and question length; the mixture-of-experts post explains why total, not active, parameters set the memory. On VMware Private AI, the NIM Operator reference architecture is a ready-made way to run all four endpoints declaratively.

DataWhy Postgres becomes business-critical

The Postgres instance looks like an implementation detail, but it holds the most valuable output of the whole exercise: the certified definitions your analysts spent weeks reviewing, plus users, roles, API tokens and the embeddings index. Losing it means losing the certification work, not just a cache. Treat it like any other system of record:

• Backups: scheduled, tested restores. The chart creates none.
• Exports as a second copy: the ontology can be exported as Apache Ossie YAML (POST /api/model/export). Keep exports in Git; they double as a human-readable change history of your business definitions.
• Storage: a replicated StorageClass (vSphere CSI on VKS, for example), sized well above the 5 Gi default once embeddings for a large catalog accumulate.
• Upgrades: each chart release runs schema migration Jobs. Take a backup before every upgrade and pin the chart version.
• Embedding consistency: the embedding model must stay the same between ingestion and queries. Changing it means re-indexing, so treat it like a database schema change, not a config tweak.

The whole pictureHow does it fit with the rest of the stack?

Seen from the platform, Auto Ontology is one tenant of the AI factory this series has described. Each earlier part has a job in running it:

Splitting it this way also simplifies ownership. The application tier is an ordinary stateful web application, which any Kubernetes platform team already knows how to run. The GPU tier is a shared inference service that other applications can use too. Auto Ontology is just one consumer of it.

Quick answers

Does Auto Ontology itself need GPUs?

No. The five services are CPU workloads. Only the model endpoints need GPUs, and only if you self-host them.

Can it run on VMware VKS?

It is a standard Helm chart, so any conformant Kubernetes cluster with a default StorageClass will do, VKS included. Put the NIM endpoints on a GPU-enabled cluster, such as one built with Private AI Foundation, and point the four model roles at them.

Can it run disconnected?

In principle, yes: mirror the three images from nvcr.io and the chart, self-host all four models, and leave KumoRFM off. Test the full compile and answer flow in the disconnected environment before relying on it.

How many replicas does it need?

Start with two each for frontend and backend for availability, then size from load tests. Most of the latency in an answer is model calls, so scale the model tier before the application tier.

Sources: Auto Ontology repository (github.com/NVIDIA/auto-ontology) at commit c447003: README, DEPLOYMENT.md, docs/authentication.md, docs/connections.md and the Helm chart helm/auto-ontology version 1.2.0 (values.yaml and templates). Default values and component lists may change between releases; check the chart you install. The NetworkPolicy is an example written for this post, not part of the project. GPU memory figures are estimates; size with your own load test.