vLLM
High-throughput LLM inference engine with PagedAttention for efficient GPU memory usage
vLLM is an open-source inference and serving engine for Large Language Models, originally developed at UC Berkeley. It uses PagedAttention to manage GPU memory efficiently, achieving up to 24x higher throughput compared to Hugging Face Transformers. It supports most popular open-source models including Llama, Mixtral, DeepSeek, and multimodal models like LLaVA. vLLM includes both a fast inference engine and a production-ready OpenAI-compatible serving server, making it a popular choice for self-hosted LLM deployments.
Pricing: Free
vLLM is an inference and serving engine built around PagedAttention, which applies operating-system paging to the KV cache. Conventional serving reserves contiguous memory for each sequence, and because the cache is large and its final size unknown in advance, vLLM's authors measured 60 to 80 percent of that memory lost to fragmentation and over-reservation. PagedAttention splits the cache into fixed-size blocks and maps logical blocks to non-contiguous physical ones through a block table, allocating on demand, which cuts the waste to under 4 percent and leaves far more room for concurrent sequences. Parallel sampling can share blocks between completions.
It has been a PyTorch Foundation hosted project since May 2025, contributed by UC Berkeley's Sky Computing Lab, so governance sits with a neutral foundation rather than a vendor. The licence is a clean, unmodified Apache-2.0. Hardware coverage is broad: NVIDIA CUDA through Blackwell, AMD ROCm, Intel XPU, CPU and Gaudi, Google TPU, AWS Neuron, ARM and Huawei Ascend. Apple Silicon is served by a separate vLLM-Metal project rather than the main distribution, and Windows is not listed as supported.
The documented limitations matter for anyone serving production traffic. vLLM does not guarantee stable log probabilities across runs, and because batching changes with concurrency, the same prompt can produce different output depending on server load; using float32 or fixing a seed reduces this without eliminating it. One server serves one model, so running several means separate instances plus your own routing. Embeddings extracted from generation models underperform purpose-trained embedding models.
vLLM Alternatives
Explore 100 products in the Inference APIs category. View all vLLM alternatives.
Project Zero
CPU-only LLM inference engine in C with no runtime dependencies
Baseten
AI inference platform for deploying and serving ML models with autoscaling and optimized infrastructure
Lambda
GPU cloud for AI training and inference with on-demand and cluster options
CoreWeave
GPU cloud infrastructure built for large-scale AI training and inference workloads
Also listed in
Work on vLLM? Feature it at the top of Inference APIs.
Is your product missing?