Calculators & Estimators · Solution Architecture
LLM Platform Sizing Worksheet for Solution Architects
One Excel workbook to take into a customer meeting: ask the right discovery questions, size the GPUs from weights and KV cache, produce a bill of materials, compare three years of self-hosting with three years of API, and decide. Pre-filled with a worked example; overwrite the yellow cells.
The method the sheet walks, left to right:
What is inside
| Tab | What you do there |
|---|---|
| README | How to run the meeting with it, colour legend (blue on yellow = you type; black = formula; green = pulled from another tab), and what the sheet deliberately does not do. |
| Settings | Pick the working currency from a drop-down. Indicative prices are stored in INR and converted through an editable FX table; every price line also has a "Your quote" cell in the chosen currency that overrides the converted figure. Headers, totals, the TCO and the proposal sentence all follow the chosen currency. |
| 1 Discovery | 22 questions to ask the customer. Each has why it matters, an answer cell, a notes column for the customer's own words, and the sizing step it drives. Numeric answers flow straight into sizing. Model and precision are drop-downs. |
| 2 Sizing | Tokens per day → weights with runtime overhead → KV cache per token (looked up from the model's layers, KV heads and head dimension) → KV cache at peak concurrency → VRAM needed → cards, rounded to tensor-parallel sizes → how many concurrent conversations the design actually holds → a plain verdict. Then the throughput check against a measured tokens-per-second figure you enter, GPUs and servers to quote with an N+1 toggle, and storage, RAM, NVMe, power and rack units. At the bottom, a comparison table runs the same workload against every card in the library. |
| 3 BOM | Quantities pulled from sizing, indicative Indian list prices to replace with quotes, one-time total, and recurring options (NVIDIA AI Enterprise, commercial Kubernetes, support) with include toggles. |
| 4 TCO | Self-hosted versus API over the horizon you choose: hardware, power, recurring, people, token cost. The cost ratio, the break-even in tokens per day, and a formula-written sentence for the proposal. |
| 5 Decision | The five self-host questions with a counted verdict, and a twelve-item checklist to complete before any purchase order. |
| Model library | Architecture facts for 11 open models (parameters, layers, KV heads, head dim → KV KB per token). Add rows; verify against the model card. |
| GPU library | 10 cards with VRAM, TDP and indicative price. Add rows for whatever you are quoting. |
vllm bench serve with the customer's token profile, and type the result in. And it does not price licences: every yellow price is an order-of-magnitude list figure to be replaced by a quote.Where it comes from
The worksheet is the calculation half of the post I run a 70B model for free and own no GPU. So where does the enterprise AI stack come in?, which walks one AI application end to end and explains every term with a plain example or a VMware equivalent. Question 8 of that post is this sheet, in prose.
For the product-level design (which Kubernetes, which storage, which vector database, which observability) use the Enterprise AI Stack Builder; it draws the layers and produces the "what". This worksheet produces the "how many". Together they are a solution document.
Worked example baked in
- 50-person training team, 20 concurrent at peak, 2,000 queries a day, 3,000 tokens in and 500 out, 16k longest context, 30-second answers, Llama-3.3-70B at INT4.
- Result: 7 M tokens a day; 38.5 GB of weights; 320 KB per token of KV cache, 21 GB at peak; 62 GB VRAM needed; two L40S in tensor-parallel; 51 concurrent conversations held; 1.2 kW per server.
- Three-year TCO: the API is about ten times cheaper at this volume; self-hosting is justified by the "documents may not leave the premises" answer alone.