Docker Deployment

Quick Start

The fastest way to deploy LiteLLM is using Docker Compose with the official image.

Pull the Docker Image

docker pull ghcr.io/berriai/litellm:main-latest

Stable releases use the -stable tag (undergoes 12-hour load tests):

docker pull ghcr.io/berriai/litellm:main-stable

Create Configuration File

Create a config.yaml file with your model configuration:

config.yaml

model_list:
  - model_name: gpt-4o
    litellm_params:
      model: gpt-4o
      api_key: os.environ/OPENAI_API_KEY
  
  - model_name: claude-sonnet-4
    litellm_params:
      model: anthropic/claude-sonnet-4-20250514
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  database_url: os.environ/DATABASE_URL

Run with Docker Compose

Use the provided docker-compose.yml for a complete setup with PostgreSQL and Prometheus:

docker-compose.yml

services:
  litellm:
    image: ghcr.io/berriai/litellm:main-stable
    ports:
      - "4000:4000"
    environment:
      DATABASE_URL: "postgresql://llmproxy:dbpassword9090@db:5432/litellm"
      STORE_MODEL_IN_DB: "True" # Allows adding models via UI
    volumes:
      - ./config.yaml:/app/config.yaml
    command:
      - "--config=/app/config.yaml"
    depends_on:
      - db
    healthcheck:
      test:
        - CMD-SHELL
        - python3 -c "import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')"
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s

  db:
    image: postgres:16
    restart: always
    environment:
      POSTGRES_DB: litellm
      POSTGRES_USER: llmproxy
      POSTGRES_PASSWORD: dbpassword9090
    ports:
      - "5432:5432"
    volumes:
      - postgres_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -d litellm -U llmproxy"]
      interval: 1s
      timeout: 5s
      retries: 10

  prometheus:
    image: prom/prometheus
    volumes:
      - prometheus_data:/prometheus
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"
    command:
      - "--config.file=/etc/prometheus/prometheus.yml"
      - "--storage.tsdb.path=/prometheus"
      - "--storage.tsdb.retention.time=15d"
    restart: always

volumes:
  prometheus_data:
    driver: local
  postgres_data:
    name: litellm_postgres_data

Start all services:

docker-compose up -d

Verify Deployment

Check health status:

curl http://localhost:4000/health/liveliness

Access the admin UI at http://localhost:4000/ui

Docker Image Details

Multi-Stage Build

The Dockerfile uses a multi-stage build for security and size optimization:

# Base images from Chainguard (minimal, security-focused)
ARG LITELLM_BUILD_IMAGE=cgr.dev/chainguard/wolfi-base
ARG LITELLM_RUNTIME_IMAGE=cgr.dev/chainguard/wolfi-base

# Builder stage
FROM $LITELLM_BUILD_IMAGE AS builder
WORKDIR /app

# Install build dependencies
RUN apk add --no-cache bash gcc py3-pip python3 python3-dev openssl openssl-dev
RUN python -m pip install build

# Copy source and build
COPY . .
RUN ./docker/build_admin_ui.sh
RUN rm -rf dist/* && python -m build
RUN pip install dist/*.whl
RUN pip wheel --no-cache-dir --wheel-dir=/wheels/ -r requirements.txt

# Runtime stage
FROM $LITELLM_RUNTIME_IMAGE AS runtime
WORKDIR /app

# Install runtime dependencies (includes Node.js for Prisma)
RUN apk add --no-cache bash openssl tzdata nodejs npm python3 py3-pip libsndfile

# Copy built artifacts
COPY --from=builder /app/dist/*.whl .
COPY --from=builder /wheels/ /wheels/
RUN pip install *.whl /wheels/* --no-index --find-links=/wheels/

# Generate Prisma client
RUN prisma generate --schema=./litellm/proxy/schema.prisma

EXPOSE 4000/tcp
ENTRYPOINT ["docker/prod_entrypoint.sh"]
CMD ["--port", "4000"]

Environment Variables

Security Best Practice: Use environment variables for sensitive data instead of hardcoding in config files.

Variable	Description	Required
`DATABASE_URL`	PostgreSQL connection string	Yes (for production)
`LITELLM_MASTER_KEY`	Master key for proxy authentication	Recommended
`STORE_MODEL_IN_DB`	Enable model management via UI	No
`OPENAI_API_KEY`	OpenAI API key	If using OpenAI
`ANTHROPIC_API_KEY`	Anthropic API key	If using Anthropic
`USE_DDTRACE`	Enable Datadog tracing	No
`SEPARATE_HEALTH_APP`	Run health checks on separate port	No

Standalone Docker Run

For simple deployments without Docker Compose:

# Without database (in-memory)
docker run -d \
  --name litellm \
  -p 4000:4000 \
  -e LITELLM_MASTER_KEY=sk-1234 \
  -e OPENAI_API_KEY=your-key \
  ghcr.io/berriai/litellm:main-stable \
  --model gpt-4o

# With external database
docker run -d \
  --name litellm \
  -p 4000:4000 \
  -e DATABASE_URL=postgresql://user:pass@host:5432/litellm \
  -e LITELLM_MASTER_KEY=sk-1234 \
  -v $(pwd)/config.yaml:/app/config.yaml \
  ghcr.io/berriai/litellm:main-stable \
  --config /app/config.yaml

Database Setup

LiteLLM uses Prisma with PostgreSQL for persistence.

Database Schema

The schema includes tables for:

API Keys (LiteLLM_VerificationToken) - Virtual keys with budgets and rate limits
Teams (LiteLLM_TeamTable) - Team management and spend tracking
Users (LiteLLM_UserTable) - User authentication and authorization
Models (LiteLLM_ProxyModelTable) - Model configurations
Spend Logs (LiteLLM_SpendLogs) - Request logs and cost tracking
Organizations (LiteLLM_OrganizationTable) - Multi-tenant support

Migrations

Migrations run automatically on container startup. The proxy takes 15-20 seconds to fully initialize.

The entrypoint script (docker/prod_entrypoint.sh) handles migrations:

#!/bin/sh
if [ "$SEPARATE_HEALTH_APP" = "1" ]; then
    export LITELLM_ARGS="$@"
    exec supervisord -c /etc/supervisord.conf
fi

if [ "$USE_DDTRACE" = "true" ]; then
    export DD_TRACE_OPENAI_ENABLED="False"
    exec ddtrace-run litellm "$@"
else
    exec litellm "$@"
fi

Prisma generates the client and applies migrations during the build process.

Prometheus Configuration

Create prometheus.yml for metrics collection:

prometheus.yml

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'litellm'
    static_configs:
      - targets: ['litellm:4000']

LiteLLM exposes metrics at /metrics endpoint.

Volume Persistence

Always use named volumes for production databases to prevent data loss.

volumes:
  postgres_data:
    name: litellm_postgres_data
    driver: local
  prometheus_data:
    driver: local

Health Checks

LiteLLM provides multiple health check endpoints:

/health/liveliness - Basic service health
/health/readiness - Ready to accept traffic
/health - Combined health status

Health check configuration:

healthcheck:
  test: ["CMD-SHELL", "python3 -c \"import urllib.request; urllib.request.urlopen('http://localhost:4000/health/liveliness')\""]
  interval: 30s
  timeout: 10s
  retries: 3
  start_period: 40s  # Allow time for migrations

Upgrading

# Pull latest stable image
docker-compose pull

# Restart services (migrations run automatically)
docker-compose up -d

# Check logs
docker-compose logs -f litellm

Troubleshooting

Container won’t start

# Check logs
docker logs litellm

# Common issues:
# 1. Database not ready - increase start_period in healthcheck
# 2. Invalid config.yaml - validate YAML syntax
# 3. Missing environment variables - check .env file

Database connection errors

# Test database connectivity
docker exec -it litellm_db psql -U llmproxy -d litellm

# Check DATABASE_URL format:
# postgresql://username:password@hostname:5432/database

Performance issues

# Check resource usage
docker stats litellm

# Increase container resources
docker run --memory=2g --cpus=2 ...

Next Steps

Kubernetes

Scale with Kubernetes and Helm charts

Monitoring

Set up observability and alerting

Security

Secure your deployment

High Availability

Deploy for production at scale

Deploy

Production

Docker Deployment

Quick Start

Docker Image Details

Multi-Stage Build

Environment Variables

Standalone Docker Run

Database Setup

Database Schema

Migrations

Prometheus Configuration

Volume Persistence

Health Checks

Upgrading

Troubleshooting

Container won’t start

Database connection errors

Performance issues

Next Steps

Kubernetes

Monitoring

Security

High Availability

Build docs developers (and LLMs) love

Deploy

Production

Documentation Index

​Quick Start

​Docker Image Details

​Multi-Stage Build

​Environment Variables

​Standalone Docker Run

​Database Setup

​Database Schema

​Migrations

​Prometheus Configuration

​Volume Persistence

​Health Checks

​Upgrading

​Troubleshooting

​Container won’t start

​Database connection errors

​Performance issues

​Next Steps

Kubernetes

Monitoring

Security

High Availability

Build docs developers (and LLMs) love

Quick Start

Docker Image Details

Multi-Stage Build

Environment Variables

Standalone Docker Run

Database Setup

Database Schema

Migrations

Prometheus Configuration

Volume Persistence

Health Checks

Upgrading

Troubleshooting

Container won’t start

Database connection errors

Performance issues

Next Steps