Tag: nim
-
NVIDIA Blueprints and Agentic AI: AI-Q and the NeMo Agent Toolkit (NVIDIA AI Series, Part 28)
NVIDIA Blueprints, the AI-Q enterprise research agent, and the framework-agnostic NeMo Agent Toolkit: how to build agents you can profile, afford, and trust in production.
-
NVIDIA NeMo Retriever: RAG with Embeddings, Reranking and Guardrails (NVIDIA AI Series, Part 27)
How NVIDIA’s NeMo Retriever builds enterprise RAG: extraction, embedding and reranking NIMs, the open Nemotron Retriever models, and NeMo Guardrails, plus the retrieval failures they quietly fix.
-
Inference Economics: Throughput, Latency, Batching and Cost Per Token (NVIDIA AI Series, Part 21)
TTFT, ITL, continuous batching, KV cache pressure, FP8 quantization — this is how you compute and actually drive down $/1M tokens on NVIDIA H100, H200, and B200 GPUs without breaking your latency SLOs.
-
Triton Inference Server vs NIM: When to Use Which (NVIDIA AI Series, Part 19)
Triton Inference Server and NVIDIA NIM solve different problems. This guide breaks down when to use each — and when to run both — covering backends, ensembles, dynamic batching, and the NIM packaged-microservice approach for LLM serving.
-
NVIDIA Dynamo Disaggregated Inference: Prefill, Decode, and KV-Aware Routing at Scale (NVIDIA AI Series, Part 20)
NVIDIA Dynamo separates prefill and decode onto independent GPU pools, routing requests via a KV-aware smart router and transferring KV cache blocks via NIXL. Here is when disaggregation wins, when it does not, and what to validate before committing to the architecture.
-
Deploying and Autoscaling NIM in Production on Kubernetes (NVIDIA AI Series, Part 17)
How to deploy NVIDIA NIM in production using the NIM Operator and Helm, wire autoscaling on the right GPU and KV-cache signals instead of CPU, handle cold-start model load, and run blue-green rollouts without dropping throughput.
-
NVIDIA NIM Inference Microservices: What a NIM Is and How It Serves a Model (NVIDIA AI Series, Part 16)
NVIDIA NIM packages a model, an optimized inference engine, and an OpenAI-compatible API into a single container. Pull it, pass your NGC API key, and you have a production inference endpoint on your own GPU infrastructure in minutes.
-
Troubleshooting VMware Private AI Foundation: 7 Failures That Actually Bite (Private AI Series, Part 23)
The seven failures I hit most often on VMware Private AI Foundation with NVIDIA, from a dark GPU on the ESXi host to a NIM pod crashing on CUDA out of memory, with the real error strings and the checks that isolate each layer.
-
How to Benchmark LLM Inference on VMware Private AI with genai-perf (Private AI Series, Part 21)
A practical runbook for benchmarking NIM inference on VMware Private AI Foundation: the metrics that matter, the concurrency sweep that exposes the real latency-throughput curve, and how to pick an operating point you can defend.
-
Air-Gapped VMware Private AI Foundation: Mirroring, AMT and the Bootstrap Problem (Private AI Series, Part 19)
Deploying VMware Private AI Foundation in a fully disconnected enclave: what to mirror, how the artifact mirroring tool (AMT) fits, the Harbor bootstrap problem, and how to validate offline NIM and GPU before handover.
-
Building a RAG Pipeline on VMware Private AI: 7 Failures That Quietly Break Retrieval (Private AI Series, Part 14)
Most RAG failures on VMware Private AI Foundation are not the LLM. Here are the seven pipeline failures that quietly wreck retrieval quality on PAIF 9, and how I fix each one in the field.
-
NVIDIA NIM Microservices on VMware Private AI: The Model-Serving Layer Explained (Private AI Series, Part 11)
NVIDIA NIM is the model-serving layer of VMware Private AI. A reference-architecture look at the NIM Operator, NIMCache and NIMService, GPU placement, and the design choices that decide whether your endpoints survive production.
Architect’s Toolkit
PJ’s Tools
VMware Cloud Foundation
- VCF Documentation
- VCF 9 Planning & Preparation Workbook
- VCF Bill of Materials (BoM)
- VMware Compatibility Guide
- VMware Interoperability Matrix
- VMware Configuration Maximums
- VMware Ports & Protocols
- VMware Hands-on Labs
- RVTools Download
Nutanix
AI & Cloud-Native Platform
- NVIDIA Build (Model Catalog)
- NVIDIA AI Enterprise Reference Architecture
- NVIDIA NIM Performance Benchmarking
- NVIDIA NGC Catalog
- NeMo Microservices Helm Chart
- Helm Charts Repository
- Hugging Face Models
Architecture & Design
About the Author
Dr Pranay Jha
Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.
You May Have Missed
DrJha