vLLM vs Text Generation Inference (TGI): which to use and when
High-throughput inference server built around PagedAttention. Hugging Face's production inference server.
At a glance
| Capability | vLLM | Text Generation Inference (TGI) |
|---|---|---|
| Licence | Apache 2.0 | Apache 2.0 |
| Implementation | Python + CUDA | Rust + Python |
| Runs on | GPU only | GPU only |
| Continuous batching | Yes | Yes |
| Paged attention | Yes | Yes |
| Tensor parallelism | Yes | Yes |
| Quantisation | AWQ, GPTQ, FP8, INT8 | AWQ, GPTQ, EETQ, bitsandbytes |
| OpenAI-compatible API | Yes | Yes |
How to choose
vLLM — Production serving of open models where throughput per GPU is the metric that matters, and for multi-GPU deployment of models too large for one card.
Text Generation Inference (TGI) — Teams already standardised on the Hugging Face stack who want a supported serving path with tight Hub integration.
Both expose an OpenAI-compatible HTTP API, so this is not a one-way door: switching is a base-URL change, and running one locally while serving on the other is a common and sensible split.
Frequently asked
- Should I use vLLM or Text Generation Inference (TGI)?
- Production serving of open models where throughput per GPU is the metric that matters, and for multi-GPU deployment of models too large for one card. By contrast, text generation inference (tgi) is the better answer when: teams already standardised on the Hugging Face stack who want a supported serving path with tight Hub integration.
- Can I use both?
- Yes, and most teams do. Because both expose an OpenAI-compatible HTTP API, moving a workload between them is a base-URL change. A common pattern is developing against the lighter runtime locally and serving production traffic on the higher-throughput one.
Project home: https://docs.vllm.ai