Tag: nvidia
-
Hugging Face Air-Gapped: Enterprise Hub, Offline Mirroring, and the On-Prem Build (Hugging Face Series, Part 16)
How to run Hugging Face models on a segmented or air-gapped network: mirror the artifacts to local storage, force offline mode at runtime, and use the Enterprise Hub for identity, governance, and the rate limits a proxy needs.
-
Running NVIDIA AI On-Prem and on VCF: Cost, Trade-offs and the Verdict (NVIDIA AI Series, Part 30)
The finale: running the NVIDIA AI stack on bare metal, on VMware Cloud Foundation, or in the cloud; the real total cost of an AI factory; and the verdict on when to build versus rent.
-
GPU Observability and Multi-Tenancy: DCGM, Honest Utilization, and Sharing (NVIDIA AI Series, Part 29)
Why GPU utilization lies, the DCGM profiling fields that tell the truth (SM and Tensor activity), dcgm-exporter into Prometheus, and choosing MIG vs time-slicing for multi-tenancy.
-
NVIDIA Blueprints and Agentic AI: AI-Q and the NeMo Agent Toolkit (NVIDIA AI Series, Part 28)
NVIDIA Blueprints, the AI-Q enterprise research agent, and the framework-agnostic NeMo Agent Toolkit: how to build agents you can profile, afford, and trust in production.
-
NVIDIA NeMo Framework: Training and Fine-Tuning at Scale (NVIDIA AI Series, Part 22)
What the NVIDIA NeMo framework is: Megatron-Core parallelism, NeMo 2.0 Python recipes and NeMo-Run, Megatron Bridge for Hugging Face interop, and when to fine-tune instead of pretrain.
-
NVIDIA NeMo Retriever: RAG with Embeddings, Reranking and Guardrails (NVIDIA AI Series, Part 27)
How NVIDIA’s NeMo Retriever builds enterprise RAG: extraction, embedding and reranking NIMs, the open Nemotron Retriever models, and NeMo Guardrails, plus the retrieval failures they quietly fix.
-
NVIDIA Nemotron Foundation Models: Open Weights from Nano to Ultra (NVIDIA AI Series, Part 26)
NVIDIA’s Nemotron family explained: genuinely open weights, data and recipes; the hybrid Mamba-Transformer MoE architecture; Nano, Super and Ultra; and when to self-host open models instead of calling a proprietary API.
-
Multi-Node LLM Training: Scheduling, Checkpointing and Fault Tolerance (NVIDIA AI Series, Part 25)
At thousands of GPUs, failures are routine. This part covers gang scheduling (Slurm vs Kubernetes vs NVIDIA Run:ai), async distributed checkpointing with NeMo, and the NVIDIA Resiliency Extension stack for fault tolerance, straggler detection, and elastic restart.
-
Inference Economics: Throughput, Latency, Batching and Cost Per Token (NVIDIA AI Series, Part 21)
TTFT, ITL, continuous batching, KV cache pressure, FP8 quantization — this is how you compute and actually drive down $/1M tokens on NVIDIA H100, H200, and B200 GPUs without breaking your latency SLOs.
-
NeMo Customization: LoRA, SFT, and RLHF on NVIDIA NeMo (NVIDIA AI Series, Part 23)
A practical decision guide for AI infrastructure architects on the full NeMo customization spectrum: when to use LoRA, full SFT, DPO, or GRPO, what data and GPU budget each method needs, and how the NeMo Customizer microservice ties it all together.
-
Data Preparation at Scale with NeMo Curator (NVIDIA AI Series, Part 24)
NeMo Curator is NVIDIA’s GPU-accelerated data curation toolkit that runs exact dedup, fuzzy dedup, semantic dedup, heuristic filtering, classifier-based quality filters, and PII redaction at trillion-token scale using RAPIDS cuDF and Dask. Learn why investing in data curation beats buying more GPUs.
-
Triton Inference Server vs NIM: When to Use Which (NVIDIA AI Series, Part 19)
Triton Inference Server and NVIDIA NIM solve different problems. This guide breaks down when to use each — and when to run both — covering backends, ensembles, dynamic batching, and the NIM packaged-microservice approach for LLM serving.
Architect’s Toolkit
PJ’s Tools
VMware Cloud Foundation
- VCF Documentation
- VCF 9 Planning & Preparation Workbook
- VCF Bill of Materials (BoM)
- VMware Compatibility Guide
- VMware Interoperability Matrix
- VMware Configuration Maximums
- VMware Ports & Protocols
- VMware Hands-on Labs
- RVTools Download
Nutanix
AI & Cloud-Native Platform
- NVIDIA Build (Model Catalog)
- NVIDIA AI Enterprise Reference Architecture
- NVIDIA NIM Performance Benchmarking
- NVIDIA NGC Catalog
- NeMo Microservices Helm Chart
- Helm Charts Repository
- Hugging Face Models
Architecture & Design
About the Author
Dr Pranay Jha
Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.
You May Have Missed
DrJha