LiteLLM

LiteLLM

LiteLLM is a self-hosted AI gateway and LLM proxy that sits between your apps and model providers. You point clients at one OpenAI-compatible endpoint, and the gateway handles routing, logging, and spend controls across 140+ backends. Teams at Netflix, NVIDIA, and Okta use it to standardize model access without rebuilding every service when a provider changes.

Where hosted proxies bill per token and keep keys on their cloud, LiteLLM runs in your VPC or an air-gapped cluster under an MIT license. Budgets and RPM/TPM caps are hard stops, not alerts, so a leaked key or runaway job cannot keep spending after it hits the limit.

Platform and ML infrastructure teams reach for it when they need one front door for humans, agents, and batch jobs. The same gateway covers LLM calls, MCP servers, and agent workflows, with enterprise SSO, audit logs, and SLAs when rollout goes company-wide.

Top Features:
  1. One OpenAI-compatible API routes requests to 140+ providers and 1,892 models

  2. Hard budgets plus RPM and TPM limits per key, team, and org stop traffic at the cap

  3. MIT-licensed self-hosted gateway with 240M+ Docker pulls and air-gapped deployment support

  4. Rust AI Gateway adds 0.66 ms p99 latency at 2,800+ requests per second in published benchmarks

  5. Built-in guardrails cover PII masking and prompt-injection checks via Presidio and Lakera

  6. Virtual keys, spend tracking, and Prometheus metrics ship in the free open-source tier

Pros:
  1. MIT-licensed open-source gateway you can self-host with no per-token license fee.

  2. Swap backend models through configuration instead of rewriting application code.

  3. Hard budgets and rate limits stop requests when a key or team hits its cap.

  4. One OpenAI-compatible endpoint covers 140+ providers, MCP servers, and agent workflows.

  5. Published benchmarks show 0.66 ms p99 gateway overhead on the Rust build.

Cons:
  1. Enterprise SSO, SCIM, and audit logs require a paid annual license.

  2. Self-hosted deployments need your own Postgres, Redis, and operational upkeep.

  3. Advanced guardrails and secret manager integrations take deliberate setup work.

FAQs:

Is LiteLLM free to use?

Yes. LiteLLM's open-source gateway is free to self-host with no license fee. The $0 Open Source plan includes 100+ provider integrations, virtual keys, budgets, rate limits, logging, and Prometheus metrics.

How is LiteLLM Enterprise priced?

LiteLLM Enterprise is priced by annual gateway request capacity, deployment architecture, and support needs, not per token. Contact sales for a quote, or start a 30-day Enterprise trial key from the Enterprise page.

Can LiteLLM run air-gapped?

Yes. LiteLLM supports self-hosted and air-gapped deployments on Enterprise plans. Your data and API keys stay on infrastructure you control, with no vendor telemetry required.

What providers does LiteLLM support?

LiteLLM supports 140+ LLM providers and 1,892 models through one OpenAI-compatible API. Documented integrations include OpenAI, Anthropic, Azure, Amazon Bedrock, Google Vertex AI, and Mistral.

Does LiteLLM see our prompts or keys?

No. LiteLLM is self-hosted, so prompts, responses, and provider keys stay inside your infrastructure. The vendor does not receive your traffic when you run the gateway yourself.

What does LiteLLM Enterprise add?

LiteLLM Enterprise adds SSO, SCIM, OIDC and JWT auth, audit logs, secret manager integrations, RBAC, multi-region control plane options, and 24/7 support with response-time SLAs on top of the open-source gateway.

Pricing:

Freemium

Tags:

LLM Proxy
API Gateway
Model Routing
Self-Hosted
Open Source
Spend Management
MCP Gateway
Load Balancing

Tech used:

Chakra UI
jQuery
Webflow
Cloudflare
Amazon CloudFront
Amazon Web Services
Google Analytics
Google Tag Manager
Ruby
GitHub
Emotion
Tailwind CSS

Reviews:

Give your opinion on LiteLLM :-

Overall rating

Privacy and Your Privacy Choices

Join thousands of AI enthusiasts in the World of AI!

Best Free LiteLLM Alternatives (and Paid)

By Rishit