Category: AI Stack
-
What a GPU cluster really costs, and how to cut it
Cost per useful GPU-hour, chargeback by allocation or by use, four savings levers and the buy-versus-rent break-even, worked through for 16 H100s shared by three teams. With a live calculator.
-
How to autoscale LLM inference on Kubernetes GPUs
The default autoscaler watches CPU, which barely moves when a GPU is full, and a new GPU replica can take ten minutes to start. Which vLLM metric to scale on, how to handle cold starts, and when scale-to-zero makes sense. With a live demo.
-
How to network GPU nodes for multi-node training
Multi-node training swaps gradients after every step, so the slowest link sets the pace for every GPU. InfiniBand vs RoCE vs TCP, rail-optimised topology, GPUDirect RDMA, the Kubernetes pieces and how to prove it with nccl-tests. With a live demo.
-
How to plan storage for AI training and inference
A training checkpoint is about 14 bytes per parameter, nearly 1 TB for a 70B model, and large jobs crash every day or two. How big checkpoints are, how often to save them, which storage keeps up and how fast models load for inference. With a live calculator.
-
How to run GPUs on VMware Cloud Foundation
Passthrough, time-sliced vGPU or MIG-backed vGPU: the first GPU decision on VMware Cloud Foundation decides density, vMotion and HA. The same nine workloads tested all three ways, plus the setup from ESX host to VKS GPU pod. With a live demo.
-
The GPU platform readiness checklist
The summary of a ten-part series on running GPUs on Kubernetes and VMware Cloud Foundation: one lesson from each part, the order to tackle them in, and a 61-point scorecard that gives you your next three fixes.
-
How to manage the GPU node lifecycle in Kubernetes
How a GPU node joins a Kubernetes cluster, gets upgraded, gets repaired after a hardware fault and is retired, without killing long training jobs. With a live demo of every step.
-
How to read GPU logs in Kubernetes
Metrics tell you a GPU failed; logs tell you why. Where GPU logs live on a Kubernetes node, how to ship them with Fluent Bit to Loki, and the six log lines that explain most GPU failures, with a live demo.
-
How to monitor GPUs in Kubernetes
Kubernetes knows a pod asked for a GPU, not whether that GPU is busy, hot or dead. How DCGM Exporter, Prometheus and four alert rules fill the gap, with a live demo cluster to break.
-
100 hands-on lab tasks for AI infrastructure
A 100-task lab workbook for AI and GPU infrastructure roles, in learning order: Linux on GPU hosts, drivers and DCGM, Slurm, Kubernetes with the GPU Operator, Prometheus and Loki, InfiniBand and RoCE, NCCL validation, Terraform and Ansible, BCM, storage and serving, runbooks and timed scenarios.
-
Attention Is All You Need: the 2017 paper that changed AI strategy and the IT industry
How the 2017 Transformer paper made AI parallel, turned AI budgets into GPU, memory, network and power decisions, and changed research, business, jobs and the wider IT industry.
-
AI & GenAI — Complete Guides & Series
Deep-dive series across the AI and generative-AI stack — self-hosted, private cloud and public cloud, explained part by part. Pick one below to read the full series. Red Hat Gen AI SeriesSelf hosting generative AI on Red Hat: RHEL AI, InstructLab, Granite, OpenShift AI, KServe, vLLM, llm-d, guardrails and GPU cost, in 30 parts.Read →Infra…
Architect’s Toolkit
PJ’s Tools
- Infra 360 Hub – All Series
- VCF Exam Practice Labs
- VCF 9 Interactive Walkthroughs
- VCF Design Cheatsheet
- VCF Upgrade Planner
- VCF 9 Series Hub
- VCF Deployment Hub
- AI Stack Hub
- AI Infra Sizing & Cost Calculator
- LLM & RAG Cost Calculator
- DrJhaGPT – Ask Pranay
VMware Cloud Foundation
- VCF Documentation
- VCF 9 Planning & Preparation Workbook
- VCF Bill of Materials (BoM)
- VMware Compatibility Guide
- VMware Interoperability Matrix
- VMware Configuration Maximums
- VMware Ports & Protocols
- VMware Hands-on Labs
- RVTools Download
Nutanix
AI & Cloud-Native Platform
- NVIDIA Build (Model Catalog)
- NVIDIA AI Enterprise Reference Architecture
- NVIDIA NIM Performance Benchmarking
- NVIDIA NGC Catalog
- NeMo Microservices Helm Chart
- Helm Charts Repository
- Hugging Face Models
Architecture & Design
About the Author
Dr Pranay Jha
Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.
You May Have Missed
DrJha