LiteLLM Alternatives

Direct alternatives to LiteLLM

If you came here from "litellm alternatives", you probably want what LiteLLM does: a self-hosted, OpenAI-compatible proxy (plus Python SDK) that gives one interface to 100+ LLM providers, with cost tracking, load balancing, and fallbacks. The closest direct replacements:

• Portkey: open-source AI gateway routing to 1,600+ LLMs across 200+ providers, with retries, fallbacks, and load balancing, plus observability, guardrails, and prompt management on top.
• OpenRouter: a hosted aggregator instead of a proxy you run. One API and one bill across providers, no infrastructure to operate, but your requests flow through a third party.
• Cloudflare AI Gateway: managed proxy at Cloudflare's edge with caching, rate limiting, retries, and cost analytics. No self-hosting, and pairs naturally with an existing Cloudflare stack.
• Vercel AI Gateway: routing layer for hundreds of models with built-in rate limiting and key management, tightly integrated with the Vercel AI SDK.
• TrueFoundry: enterprise-leaning gateway routing across 250+ LLMs with latency-based routing, guardrails, RBAC, and rate limiting. Self-hostable or managed.
• Orq.ai: gateway plus LLMOps platform, combining model routing with prompt management, evals, and governance. Deployable in your own environment if data residency drives the decision.
• Helicone: proxy-style integration focused on observability: logging, caching, and rate limiting across providers. A common pick when monitoring is the main need alongside routing.
• NanoGPT: hosted rather than self-hosted, but worth knowing if the reason you are running a proxy is cost control. Text is billed at each provider's list price with no percentage added.

One distinction worth keeping in mind: LiteLLM is open source and runs in your own infrastructure. Portkey's gateway is also open source and self-hostable, and TrueFoundry and Orq.ai both offer self-hosted deployments; OpenRouter, NanoGPT, Vercel AI Gateway, and Cloudflare AI Gateway are managed services. If self-hosting is a hard requirement, that narrows the list quickly.