TL;DR
There are three ways to run a model from the Hub. Inference Providers (the serverless proxy that used to be called the Inference API) is a single HF token in front of Groq, Together, Fireworks and others; you pay per token and own nothing. Inference Endpoints gives you a dedicated, autoscaling instance HF manages for you, billed by the minute at a fixed hourly rate whether or not traffic arrives. Self-host with Text Generation Inference puts the model on GPUs you own and operate. The decision is build-vs-buy plus one question most teams forget to ask: where does the request data go. My default: prototype on Providers, ship steady production on a dedicated Endpoint or self-hosted TGI, and self-host the moment data residency or sustained GPU load makes the math obvious.
A model that runs on your laptop is not a service. The moment something else has to call it ten thousand times a day, you are no longer doing machine learning, you are doing capacity planning, cost control and access management. That is your job already. The only new vocabulary is the three doors Hugging Face gives you to walk through, and each door hands the GPU bill, the uptime and the data path to a different owner.
Think of it the way you already think about compute. You can rent a function that runs on someone else’s pool and bills per call. You can rent a dedicated VM that bills whether or not it is busy. Or you can buy the hardware and run it in your own rack. Inference Providers, Inference Endpoints and self-hosting are those same three choices, applied to a model instead of a web app. Get the analogy right and the rest is arithmetic.
Three ways to serve, side by side
Before the detail, here is the whole decision on one screen. Read it as an infra person reads an instance-type table: who owns the box, how billing behaves when idle, and where the data goes.
Door one: Inference Providers, the serverless proxy
If you remember the old ‘Inference API’, this is its replacement and it is a bigger idea. Inference Providers is a router. You send one request with one Hugging Face token, and Hugging Face forwards it to a real inference provider behind the scenes: Groq, Together, Fireworks, Cerebras, SambaNova, HF Inference and more. You do not pick a server, a region or a GPU. You pick a model, and optionally a policy for how the router chooses among providers.
For an infra reader the useful framing is an API gateway with upstream failover. The router gives you unified authentication and one bill across many upstreams, and with automatic selection it reroutes to another provider when the primary is flagged unavailable. The endpoint is OpenAI-compatible, so existing client code mostly just changes its base URL. You can steer the routing with a suffix on the model id: append :fastest for highest throughput (the default), :cheapest for lowest price per output token, or :preferred to follow your own order.
Worked example: one call, two ways
Install the client, log in once, and hit a model. The same token works for every provider behind the router.
pip install huggingface_hub
hf auth login # paste a token from hf.co/settings/tokens
# python
from huggingface_hub import InferenceClient
client = InferenceClient()
out = client.chat.completions.create(
model='openai/gpt-oss-120b:cheapest',
messages=[{'role':'user','content':'Ping'}],
)
print(out.choices[0].message.content)Or skip the SDK and use the OpenAI-compatible route directly:
curl https://router.huggingface.co/v1/chat/completions
-H 'Authorization: Bearer '$HF_TOKEN
-H 'Content-Type: application/json'
-d '{"model":"openai/gpt-oss-120b:fastest","messages":[{"role":"user","content":"Ping"}]}'Expected: a JSON chat completion with a short reply. Failure mode: a 401 means the token is missing or lacks the ‘Make calls to Inference Providers’ permission; once your free allowance is spent, requests stop until you are on a paid plan with credit on file. Fix the token scope first, then the plan.
Door two: Inference Endpoints, dedicated but still managed
Inference Endpoints is the rented dedicated VM of the three. You pick a model, a cloud, a region and an instance type, and Hugging Face stands up a dedicated deployment that only serves you. It autoscales between a minimum and maximum replica count, can scale to zero when idle, and runs proven engines under the hood: TGI, vLLM, SGLang and Text Embeddings Inference. You get logs, analytics, and on the enterprise tier private networking such as AWS PrivateLink. You do not manage the host, the driver stack or the serving container.
The billing is the part an infra owner has to internalize, because it does not work like Providers. You pay the instance hourly rate, by the minute, for every replica that is up, whether or not a single request arrives. An always-on T4 is not free between requests. The published formula makes it concrete:
cost = instance hourly rate * ((hours * min replicas) + (scale-up hrs * extra replicas))Worked example: the idle tax
Take an always-on T4 at $0.50/hr with one replica and an hourly traffic spike that adds two replicas for 15 minutes. The base is 730 hrs × 1 replica; the spikes add 182.5 hrs × 2 replicas. That is $0.50 × (730 + 365) = roughly $547.50/month. Notice that more than half the bill is the base replica that runs all night serving nobody. That single fact decides whether scale-to-zero or self-hosting is the smarter move for your traffic shape.
Door three: self-host with Text Generation Inference
The third door is the one infra teams underrate because it sounds like the most work. It is also the one that gives you everything the other two take away: the hardware, the network, the data path and the unit economics. Text Generation Inference is the open-source serving container Hugging Face itself runs inside Endpoints. You can pull the same image and run it on a GPU you already own, on-prem or in your own cloud account. It is a container behind a port. You have deployed a hundred of those.
Worked example: TGI on your own GPU
One container, one model, an OpenAI-compatible endpoint on localhost:
docker run --gpus all --shm-size 1g -p 8080:80
-v $PWD/data:/data
ghcr.io/huggingface/text-generation-inference:latest
--model-id mistralai/Mistral-7B-Instruct-v0.3
# then call it
curl http://localhost:8080/v1/chat/completions
-H 'Content-Type: application/json'
-d '{"model":"tgi","messages":[{"role":"user","content":"Ping"}]}'Expected: after the weights download to the mounted /data volume, a chat completion comes back on port 8080. Failure mode: a CUDA out-of-memory error means the model does not fit the card; quantize it or move to a bigger GPU. NCCL or shared-memory errors on multi-GPU usually mean --shm-size is too small. Pin the image to a tested version tag in production rather than tracking latest.
Self-hosting is where this series connects to the rest of your stack. Fitting the model on the GPUs you have is a quantization problem, covered next in Part 13. The cost-per-token math that decides whether self-hosting beats renting is the same arithmetic the NVIDIA series works through in Inference Economics. And if the driver is data residency rather than cost, running TGI inside your own VMware estate is exactly the path the Private AI series lays out. The HF model becomes the artifact; your platform becomes the runtime.
How to choose
Two questions settle most decisions before cost ever comes up. First: can the request data leave your boundary? If a contract, a regulator or a data-residency rule says no, doors one and two are off the table and you self-host. That is not a preference, it is a constraint, and it is the same constraint that governs where you allow any other workload to run. Second: do you have anyone to operate a GPU service? If the honest answer is no, a managed Endpoint buys you an ops team you do not have to hire. Only after those two do you reach for the cost crossover.
This is the same Hub-as-registry mental model the series opened with. The model is an artifact you pulled and vetted, the way you would push and version it on the Hub in Part 10. Serving is just deciding which runtime that artifact lands on. For the underlying concepts behind the model itself, the vendor-neutral GenAI series covers them; here we are only deciding where it runs.
Where I land
I do not treat these as three equal options to weigh fresh each time. I treat them as a ramp. Start on Inference Providers because it costs you nothing to stand up and proves the model does the job; it is the right answer for prototypes, internal tools and any workload too spiky to justify a standing GPU. It is the wrong answer the moment per-token cost crosses what a dedicated instance would cost flat, or the moment your data cannot leave your boundary. Validate that crossover with your real token volume before you assume it.
Move to a dedicated Inference Endpoint when traffic is steady, latency matters and you have no one to run GPUs. It is the wrong answer when traffic is bursty enough that you pay mostly for idle replicas, or when you do have a platform team, because at that point you are renting at a markup something you could run yourself. And self-host with TGI when scale, unit economics or data residency make it the obvious call. It is the wrong answer if you have no GPU capacity and no one to operate it, in which case you are inventing an ops burden to save money you have not yet proven you are spending.
Run the cost crossover for your own traffic this week. Take your real or projected token volume, price it on Providers, then compare it to a flat T4 or A10G Endpoint and to TGI on a GPU you already own. The number will tell you which door you are standing in front of, and it is almost never the one you assumed when you started.
References
- Hugging Face: Inference Providers documentation
- Hugging Face: Inference Endpoints pricing
- Hugging Face: Text Generation Inference (TGI)
About The Author
DrJha