LiteLLM
LiteLLM is a self-hosted AI gateway and LLM proxy that sits between your apps and model providers. You point clients at one OpenAI-compatible endpoint, and the gateway handles routing, logging, and spend controls across 140+ backends. Teams at Netflix, NVIDIA, and Okta use it to standardize model access without rebuilding every service when a provider changes.
Where hosted proxies bill per token and keep keys on their cloud, LiteLLM runs in your VPC or an air-gapped cluster under an MIT license. Budgets and RPM/TPM caps are hard stops, not alerts, so a leaked key or runaway job cannot keep spending after it hits the limit.
Platform and ML infrastructure teams reach for it when they need one front door for humans, agents, and batch jobs. The same gateway covers LLM calls, MCP servers, and agent workflows, with enterprise SSO, audit logs, and SLAs when rollout goes company-wide.
One OpenAI-compatible API routes requests to 140+ providers and 1,892 models
Hard budgets plus RPM and TPM limits per key, team, and org stop traffic at the cap
MIT-licensed self-hosted gateway with 240M+ Docker pulls and air-gapped deployment support
Rust AI Gateway adds 0.66 ms p99 latency at 2,800+ requests per second in published benchmarks
Built-in guardrails cover PII masking and prompt-injection checks via Presidio and Lakera
Virtual keys, spend tracking, and Prometheus metrics ship in the free open-source tier
MIT-licensed open-source gateway you can self-host with no per-token license fee.
Swap backend models through configuration instead of rewriting application code.
Hard budgets and rate limits stop requests when a key or team hits its cap.
One OpenAI-compatible endpoint covers 140+ providers, MCP servers, and agent workflows.
Published benchmarks show 0.66 ms p99 gateway overhead on the Rust build.
Enterprise SSO, SCIM, and audit logs require a paid annual license.
Self-hosted deployments need your own Postgres, Redis, and operational upkeep.
Advanced guardrails and secret manager integrations take deliberate setup work.
Is LiteLLM free to use?
Yes. LiteLLM's open-source gateway is free to self-host with no license fee. The $0 Open Source plan includes 100+ provider integrations, virtual keys, budgets, rate limits, logging, and Prometheus metrics.
How is LiteLLM Enterprise priced?
LiteLLM Enterprise is priced by annual gateway request capacity, deployment architecture, and support needs, not per token. Contact sales for a quote, or start a 30-day Enterprise trial key from the Enterprise page.
Can LiteLLM run air-gapped?
Yes. LiteLLM supports self-hosted and air-gapped deployments on Enterprise plans. Your data and API keys stay on infrastructure you control, with no vendor telemetry required.
What providers does LiteLLM support?
LiteLLM supports 140+ LLM providers and 1,892 models through one OpenAI-compatible API. Documented integrations include OpenAI, Anthropic, Azure, Amazon Bedrock, Google Vertex AI, and Mistral.
Does LiteLLM see our prompts or keys?
No. LiteLLM is self-hosted, so prompts, responses, and provider keys stay inside your infrastructure. The vendor does not receive your traffic when you run the gateway yourself.
What does LiteLLM Enterprise add?
LiteLLM Enterprise adds SSO, SCIM, OIDC and JWT auth, audit logs, secret manager integrations, RBAC, multi-region control plane options, and 24/7 support with response-time SLAs on top of the open-source gateway.

