Category: Career Aspirants
-
Cost Control and Model Routing for LLM Applications (AI Engineering Series, Part 27)
Model routing saved our documentation assistant 22 percent. Trimming retrieval saved more, in one line of config. Here is the cost model, the routing decision table, and the measured numbers behind both.
-
Caching, Batching and Latency Engineering for LLM Applications (AI Engineering Series, Part 26)
Prompt caching, batch processing and streaming, measured on a production docs assistant. Where the breakpoint really goes, why the obvious block to cache is the wrong one, and the cost arithmetic that follows.
-
Observability for LLM Applications: Tracing and Debugging a Non Deterministic System (AI Engineering Series, Part 25)
A request level trace is the only artifact that explains why an LLM answer was wrong, slow or expensive. Here is how I instrument one with OpenTelemetry GenAI conventions and Langfuse, which attributes actually earn their storage, and what tracing costs once traffic is real.
-
Prompt Injection and the Security Model of an LLM Application (AI Engineering Series, Part 24)
Prompt injection is not a filtering problem. Here is how an instruction hidden in a support ticket made my assistant fire 63 escalations, and the tool permission model that took attack success from 71.7 percent down to 3.3 percent.
-
Guardrails for LLM Applications: Input Filtering, PII and Output Validation (AI Engineering Series, Part 23)
Where each guardrail belongs in an LLM request path, what it costs in milliseconds, and why redacting PII in your corpus at ingest time is a mistake you cannot undo. With runnable Presidio and Pydantic code.
-
Regression Testing Prompts and Model Upgrades Without Shipping a Silent Downgrade (AI Engineering Series, Part 22)
A prompt edit is a code change with no compiler behind it. Here is the pytest gate, the baseline record and the model migration runbook I use so an upgrade cannot quietly make the assistant worse.
-
Automated Evaluation With LLM as Judge, and Where It Lies to You (AI Engineering Series, Part 21)
An LLM judge that agrees with your humans 71 percent of the time can still be worthless, because a judge that passes everything scores 60. Here is how I calibrate one with Cohen kappa, and the four biases that make it lie.
-
Building an Eval Set Before You Build the Feature (AI Engineering Series, Part 20)
An eval set written before the feature is the only thing that tells you whether a prompt change helped. Here is the case schema, the coverage measurement, and a pytest runner, with the real failures they produce.
-
Human in the Loop and Approval Gates for LLM Agents (AI Engineering Series, Part 19)
Pausing an agent for approval is the easy half. This part builds an approval gate in the OpenAI Agents SDK and in LangGraph, then explains why the gate has to be enforced at the service that moves the money rather than inside the agent process.
-
Multi Agent Patterns and Their Coordination Cost (AI Engineering Series, Part 18)
Subagents, handoffs, router and skills compared on measured token and model call counts, plus the coordination ratio I use to decide whether a split is worth paying for. Includes the handoff bug that returns a 400 and how to avoid it.
-
Multi Step Workflows, State and Memory in LLM Applications (AI Engineering Series, Part 17)
A checkpointer is a crash recovery log, not memory, and confusing the two is how a support assistant ends up carrying 47,000 tokens of unrelated history into every request. Threads, checkpoints, stores and compaction, with runnable LangGraph code.
-
Model Context Protocol and Standardising Tool Access, With a Working Python Server (AI Engineering Series, Part 16)
Part 14 gave the support assistant a working tool loop. This part turns those hand wired functions into an MCP server any client can discover, with the transport choice, the failure modes and the context cost measured rather than assumed.
Architect’s Toolkit
PJ’s Tools
- Infra 360 Hub – All Series
- VCF 9 Interactive Walkthroughs
- VCF Design Cheatsheet
- VCF Upgrade Planner
- VCF 9 Series Hub
- VCF Deployment Hub
- AI Stack Hub
- AI Infra Sizing & Cost Calculator
- LLM & RAG Cost Calculator
- DrJhaGPT – Ask Pranay
VMware Cloud Foundation
- VCF Documentation
- VCF 9 Planning & Preparation Workbook
- VCF Bill of Materials (BoM)
- VMware Compatibility Guide
- VMware Interoperability Matrix
- VMware Configuration Maximums
- VMware Ports & Protocols
- VMware Hands-on Labs
- RVTools Download
Nutanix
AI & Cloud-Native Platform
- NVIDIA Build (Model Catalog)
- NVIDIA AI Enterprise Reference Architecture
- NVIDIA NIM Performance Benchmarking
- NVIDIA NGC Catalog
- NeMo Microservices Helm Chart
- Helm Charts Repository
- Hugging Face Models
Architecture & Design
About the Author
Dr Pranay Jha
Dr. Pranay Jha is a Cloud and AI Consultant with 18+ years of experience in hybrid cloud, virtualization, and enterprise infrastructure transformation. He specializes in VMware technologies, multi-cloud strategy, and Generative AI solutions. He holds a PhD in Computer Applications with research focused on Cloud and AI, has published multiple research papers, and has been a VMware vExpert since 2016 and a VMUG Community Leader.
You May Have Missed
DrJha