TrueFoundry

TrueFoundry is an AI gateway that routes requests to LLMs across providers (OpenAI, Anthropic, Gemini, Mistral, Groq, and others) through a unified API. It adds latency-based routing, weighted load balancing, automatic fallbacks, and per-user or per-team rate limits and token or cost quotas.

The platform includes observability for token usage, latency, and error rates, built-in guardrails (PII filtering, toxicity detection), and governance via role-based access control with per-team API keys. It can run in a VPC, on-prem, or air-gapped, with no request data leaving the customer environment.