-
Why GenAI Runs on GPUs, and the Memory Wall That Limits It (GenAI Series, Part 23)
Models run on GPUs for parallel matrix math, but generating text is limited by memory, not compute. Why bandwidth caps speed, VRAM caps what runs, and the KV cache fills the gap.
-
Where the Money Actually Goes in Generative AI (GenAI Series, Part 22)
Almost every dollar in generative AI is GPU time, metered as tokens. The real cost drivers, why output tokens cost more than input, and the build-versus-buy decision.
-
Guardrails and Responsible AI: What They Catch, and What They Miss (GenAI Series, Part 21)
Guardrails screen what goes into and out of an AI model. What they catch, harmful content, jailbreaks, prompt injection, data leaks, and why safety must be layered, not a single filter.
-
Quantization: Running Big Models on Smaller GPUs (GenAI Series, Part 20)
Quantization stores a model at lower precision so it needs far less memory. How FP16, INT8 and INT4 trade a little quality for big savings, plus distillation and pruning.
-
Why Data, Not Model Size, Usually Decides Quality (GenAI Series, Part 19)
A smaller model trained on more, cleaner data often beats a bigger one. Why parameter count is overrated, what the Chinchilla result showed, and how data curation decides quality.
-
“Looks Good” Isn’t Enough: Evaluating GenAI Output (GenAI Series, Part 18)
Fluent is not the same as correct. How to evaluate GenAI output properly: build a golden set, choose human, automatic or model-graded scoring, and run it as a harness.
-
AI Agents: What Actually Works, and What’s Hype (GenAI Series, Part 16)
An AI agent is a model in a loop that plans, calls tools, and observes results. What agents genuinely do well today, and why reliability, not intelligence, is the real bottleneck.
-
Fine-Tuning vs RAG vs Prompting: Which One, and When (GenAI Series, Part 15)
Prompting steers, RAG adds facts, fine-tuning changes behaviour. The one question that decides which to use, a side-by-side comparison, and why to escalate in order of cost.
-
RAG: How to Stop Your AI Making Things Up (GenAI Series, Part 13)
Retrieval-augmented generation lets a model answer from your own documents by fetching the relevant passages at question time. How RAG works, and why it beats fine-tuning for facts.
-
Prompt Engineering That Actually Works (GenAI Series, Part 12)
Prompt engineering is not secret incantations, it is clear communication. The four moves that do most of the work, system vs user prompts, and the anti-patterns that waste tokens.
Architect’s Toolkit
PJ’s Tools
VMware Cloud Foundation
- VCF Documentation
- VCF 9 Planning & Preparation Workbook
- VCF Bill of Materials (BoM)
- VMware Compatibility Guide
- VMware Interoperability Matrix
- VMware Configuration Maximums
- VMware Ports & Protocols
- VMware Hands-on Labs
- RVTools Download
Nutanix
AI & Cloud-Native Platform
- NVIDIA Build (Model Catalog)
- NVIDIA AI Enterprise Reference Architecture
- NVIDIA NIM Performance Benchmarking
- NVIDIA NGC Catalog
- NeMo Microservices Helm Chart
- Helm Charts Repository
- Hugging Face Models
Architecture & Design
About the Author
Dr Pranay Jha
Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.
Recommendations & Endorsements
Pranay has very deep technical knowledge and exceptional capabilities. He has strong ability to well-manage complex situations and deliver the customer expectations with high standards of satisfaction. We worked together in one of most complex migration projects and Pranay delivered flawlessly.
Pranay was my VMware Architect on one of our financial accounts at IBM. He was instrumental in documenting and recommending our reference architecture. As a Chief Architect, you are only as good as your team that supports you. Pranay was top notch.
Pranay is proactive, result oriented, responsible and technically sound. Always ready to put all his energy to get the job done. Exceptional troubleshooting skills in Windows Server, virtualization and VMware — a great asset to any company.
Pranay is one of the most talented and dedicated persons I have ever worked with. His vast knowledge from different domains enables him to discuss and help on almost every topic in IT. Needless to say, he is expert of experts in VMware.
DrJha