Xinference
Open-source inference engine for 300+ open models, plus a hosted API and managed GPUs served from Australia
Xinference is an Apache-2.0 inference engine for serving open models behind an OpenAI-compatible API, developed in the open at xorbitsai/inference (about 9,600 GitHub stars). The Community edition runs on NVIDIA GPUs in a single cluster.
Xinference Holdings, a Singapore company, also sells a hosted Model API and dedicated H100, H200, B200 and A100 instances served from Australia. An Enterprise edition adds AMD, Intel and CPU support plus multi-cluster scaling.
The engine is free; hosted and Enterprise pricing goes through sales.
Pricing: Free (open source); hosted and Enterprise via sales
Xinference Alternatives
Explore 147 products in the Inference APIs category. View all Xinference alternatives.
vLLM
High-throughput LLM inference engine with PagedAttention for efficient GPU memory usage
Project Zero
CPU-only LLM inference engine in C with no runtime dependencies
OpenAI Decisions API
Typed answers with probabilities for yes/no checks, fixed choices and rubric scores
Microsoft-Decision-1
Microsoft's decision model on Foundry: typed scores and yes/no answers with probabilities instead of text
Also listed in
Work on Xinference? Feature it at the top of Inference APIs.
Is your product missing?