litellm

docker pull reg.mini.dev/litellm

Updated 2 hours ago

docker pull reg.mini.dev/litellm

LiteLLM Overview

Secure your stack with a hardened LiteLLM image freshly-built by Minimus. Minimus images always include the most up-to-date package version for all packages and dependencies contained in the image.

LiteLLM is a proxy server used to make LLM API calls using the OpenAI format.

Try It Out

Take the Minimus LiteLLM image for a test run by creating a configuration file, starting the proxy, and sending a request.

Create a Configuration File

Create a file called litellm_config.yaml that defines the models the proxy should expose. This example configures an OpenAI model:

model_list:
  - model_name: my-gpt-4o
    litellm_params:
      model: openai/gpt-4o
      api_key: os.environ/OPENAI_API_KEY

Substitute the model, key variable, and any additional parameters for your provider. See the LiteLLM configuration docs for the full range of options.

Start the Proxy

Run the container with the config file mounted and the required API key passed as an environment variable:

docker run --rm -p 4000:4000 \
  -v $(pwd)/litellm_config.yaml:/app/config.yaml \
  -e OPENAI_API_KEY=<your-api-key> \
  reg.mini.dev/litellm \
  --config /app/config.yaml --detailed_debug

The proxy will start listening on http://localhost:4000.

Send a Test Request

From another terminal, send a chat completion request using the model name defined in the config:

curl http://localhost:4000/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my-gpt-4o",
    "messages": [{"role": "user", "content": "hello"}]
  }'

For more deployment options (database, Redis, Kubernetes, Helm), see the official deployment guide.

Technical Considerations

The LiteLLM image provided by Minimus is a slim, security-hardened alternative to the public image from Docker Hub. The images are largely interchangeable, with a few differences as noted below.

LiteLLM built by Minimus:

Runs as non-root as user 65534.
Drill down on the version specification tab to see the default user, listening ports, entrypoint, volumes, environment variables, etc.

The Payoff

A hardened, minimal image that will remain more secure for the long run and accrue vulnerabilities at a slower rate.

See the risk reduction dashboard for a detailed CVE comparison over the past 30 days.
Review the compliance report to see the default hardening and security configurations for the image.

Terms & Info

Trademark

This catalog is published by Minimus. All product names, logos, and marks, other than those belonging to Minimus, shown are owned by their respective rights holders and appear here only to identify the open source software each image contains. Minimus claims no ownership of those marks and implies no affiliation with, endorsement by, certification by, or sponsorship by any rights holder.

Disclaimer

Images are provided "as-is" without warranty of any kind. "Hardened" refers to the security configuration applied at the time of build and does not constitute a guarantee of ongoing security or absence of vulnerabilities. The free tier is provided without support, SLA, or guaranteed patching timelines. Security updates may be applied to paid subscriptions before or instead of free tier images. By pulling or using any image you agree to our Terms of Use.