Category: AI Stack
-
Scaling Inference: The Latency vs Throughput Trade-Off (GenAI Series, Part 25)
Scaling AI inference means choosing a point on the latency-versus-throughput curve. How batching, tensor and pipeline parallelism, and autoscaling on the right signal work.
-
vLLM vs TensorRT-LLM vs SGLang: Which Inference Engine, and When (GenAI Series, Part 24)
The inference engine decides whether a GPU serves five users or fifty. How continuous batching and paged attention work, and when to choose vLLM, TensorRT-LLM, SGLang or NIM.
-
Why GenAI Runs on GPUs, and the Memory Wall That Limits It (GenAI Series, Part 23)
Models run on GPUs for parallel matrix math, but generating text is limited by memory, not compute. Why bandwidth caps speed, VRAM caps what runs, and the KV cache fills the gap.
-
Where the Money Actually Goes in Generative AI (GenAI Series, Part 22)
Almost every dollar in generative AI is GPU time, metered as tokens. The real cost drivers, why output tokens cost more than input, and the build-versus-buy decision.
-
Guardrails and Responsible AI: What They Catch, and What They Miss (GenAI Series, Part 21)
Guardrails screen what goes into and out of an AI model. What they catch, harmful content, jailbreaks, prompt injection, data leaks, and why safety must be layered, not a single filter.
-
Quantization: Running Big Models on Smaller GPUs (GenAI Series, Part 20)
Quantization stores a model at lower precision so it needs far less memory. How FP16, INT8 and INT4 trade a little quality for big savings, plus distillation and pruning.
-
Why Data, Not Model Size, Usually Decides Quality (GenAI Series, Part 19)
A smaller model trained on more, cleaner data often beats a bigger one. Why parameter count is overrated, what the Chinchilla result showed, and how data curation decides quality.
-
“Looks Good” Isn’t Enough: Evaluating GenAI Output (GenAI Series, Part 18)
Fluent is not the same as correct. How to evaluate GenAI output properly: build a golden set, choose human, automatic or model-graded scoring, and run it as a harness.
-
AI Agents: What Actually Works, and What’s Hype (GenAI Series, Part 16)
An AI agent is a model in a loop that plans, calls tools, and observes results. What agents genuinely do well today, and why reliability, not intelligence, is the real bottleneck.
-
Fine-Tuning vs RAG vs Prompting: Which One, and When (GenAI Series, Part 15)
Prompting steers, RAG adds facts, fine-tuning changes behaviour. The one question that decides which to use, a side-by-side comparison, and why to escalate in order of cost.
Architect’s Toolkit
PJ’s Tools
VMware Cloud Foundation
- VCF Documentation
- VCF 9 Planning & Preparation Workbook
- VCF Bill of Materials (BoM)
- VMware Compatibility Guide
- VMware Interoperability Matrix
- VMware Configuration Maximums
- VMware Ports & Protocols
- VMware Hands-on Labs
- RVTools Download
Nutanix
AI & Cloud-Native Platform
- NVIDIA Build (Model Catalog)
- NVIDIA AI Enterprise Reference Architecture
- NVIDIA NIM Performance Benchmarking
- NVIDIA NGC Catalog
- NeMo Microservices Helm Chart
- Helm Charts Repository
- Hugging Face Models
Architecture & Design
About the Author
Dr Pranay Jha
Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.
You May Have Missed
