LiteLLM

What it does

LiteLLM is an open-source AI gateway that puts LLMs, agents, and MCP servers behind one OpenAI-compatible API. It reaches 140+ providers and about 1,900 models, and tracks spend by key, user, team, and org. Budgets and rate limits can stop requests at the cap, with guardrails and audit logs for governance.

Teams start it with a one-line quickstart script or a Docker Compose file, then do the rest in the built-in Admin UI at localhost:4000/ui: add provider keys and models, test in the Playground, and issue virtual keys. Applications then point any OpenAI SDK at the gateway. It also offers load balancing, lowest-cost routing, Auto Routing, and response and semantic caching.

It suits platform and infrastructure teams who want to govern LLM access across an organization and can operate a self-hosted gateway on Docker or Kubernetes with Postgres and Redis. The core gateway is MIT-licensed and free to self-host. SSO, SCIM, audit logs, and support SLAs belong to the Enterprise plan, which is priced on request.

What it can do

• OpenAI-compatible API across 140+ LLM providers and about 1,892 models
• Virtual keys with per-key, per-team, and per-model budgets, rate limits, and hard spend caps
• Spend tracking and chargeback attribution per key, user, team, org, and tag
• Load balancing, lowest-cost routing, and Auto Routing across deployments
• Response and semantic caching (Redis, S3, GCS) and guardrails such as PII masking and prompt-injection detection
• Self-hosted deployment via Docker, Helm, or Terraform, including air-gapped environments

Works with

• Langfuse
• Arize Phoenix
• LangSmith
• OpenTelemetry (OTEL)
• Prometheus
• Datadog
• CloudZero
• Redis
• Postgres
• AWS Secrets Manager

Before you choose

• Budgets and virtual keys require a connected database; without one, the global budget check does not enforce spend caps.
• Keep LITELLM_SALT_KEY stable, since changing it makes stored provider credentials unreadable.
• Enterprise features such as SSO, SCIM, audit logs, and 24/7 support SLAs require the Enterprise plan with pricing on request.

Listed under

• Coding
• AI gateway
• LLM proxy
• Mcp gateway
• Open source
• Platform teams
• Python SDK

Get started

Start the gateway, then log in to the Admin UI to add models and create virtual keys.

1. Start the gateway with the quickstart script or docker-compose.quickstart.yml, generating LITELLM_MASTER_KEY and LITELLM_SALT_KEY.
2. Log in to the Admin UI at http://localhost:4000/ui, add a model with your provider API key, and verify with Test Connect.
3. Create a virtual key under Virtual Keys, copy it immediately, and call http://localhost:4000 from your app with an OpenAI SDK.

The open-source gateway is free to self-host (MIT-licensed). Model provider costs are separate and billed by each provider. Enterprise pricing is available via the request-pricing form, and a 30-day Enterprise trial key is offered.

30-day free trial.

Source: litellm.ai, checked 29 Oct 2026. Prices can change; check before you buy.

Sources for this overview

Product pages, documentation, and pricing checked 29 Oct 2026. Each source supports part of the overview; availability and prices can change.

• LiteLLM — Open-Source AI Gateway & LLM Proxy ↗www.litellm.ai
• Quickstart | liteLLM ↗docs.litellm.ai
• Enterprise AI Gateway for Secure LLM Access | LiteLLM ↗www.litellm.ai

Company website

litellm.ai: product, pricing and policy pages

Hacker News

Launch posts and discussion

YC directory mirror

Batch and company name

Social accounts

Accounts the company links, and its posts on X

Human review

Checked by a person

manual_review

Last checked 29 Oct 2026. Every field keeps its source and date; unknown means we couldn’t confirm it.