LiteLLM logo

Deploy LiteLLM

Run your own LLM gateway with LiteLLM

AIDevelopmentUtilitiesLiteLLM + PostgreSQL + Email

Deploy LiteLLM with managed Postgres, platform email, and a public HTTPS URL. Give every app and teammate its own key with a budget, call any model through one OpenAI-compatible API, and see what each request cost. Bring your own provider keys.

LiteLLM running on StackBlaze

What you get with LiteLLM

  • One API for every modelCall OpenAI, Anthropic, Gemini, Bedrock, a local Ollama model or any other provider LiteLLM supports with the same OpenAI SDK code.
  • Closed by defaultEvery API route needs the master key or a virtual key. The master key, the admin password and the encryption key are generated for this instance at deploy.
  • Virtual keys and teamsHand out a key per app, person or team, limited to the models you pick, with its own budget and rate limits. Revoke it without touching the provider key.
  • Spend trackingEvery request is logged with its tokens and cost per key, team and model, so you can see where the money goes.
  • Admin UIAdd models, create keys and teams, test a key in the playground and read the request logs from the browser.
  • Encrypted provider keysProvider API keys added in the UI are stored encrypted in Postgres with a salt key generated at deploy.

After you deploy LiteLLM

  1. 1
    Sign in to the admin UI

    Open the public URL; it lands on the LiteLLM login. Sign in as admin with UI_PASSWORD from the app’s Variables in the dashboard.

  2. 2
    Connect your own model (optional)

    For a model you run yourself, such as vLLM on a rented GPU, set under Variables:

    MODEL_URL
    Your model server’s base URL, with or without /v1
    MODEL_API_KEY
    Its API key or password
    MODEL_NAME
    The name of the model it serves
    MODEL_ALIAS
    The name your apps send. Empty: the same as MODEL_NAME

    Saving restarts the app, and the model is listed under Models.

  3. 3
    Add a model

    Under Models, add a provider and paste its API key, or point a model at an OpenAI-compatible or Ollama endpoint you run.

  4. 4
    Create a virtual key and call it

    Under Virtual Keys, create a key for your app with the models and budget it may use, then call your URL’s /v1 endpoint with any OpenAI SDK.

How LiteLLM is wired

LiteLLM and Postgres ship together, with the platform email add-on for invites and alerts. The public hostname serves the API, the admin UI at /ui and the API reference at /docs.

Postgres holds the gateway
Models, virtual keys, teams, users and spend logs live in the attached database. Redeploy the app and everything is still there.
Keep the salt key
LITELLM_SALT_KEY encrypts the provider keys you add. It is generated once at deploy. Changing it later makes the saved keys unreadable.
Email for invites and alerts
Turn on email alerting in the admin UI and LiteLLM sends invites, new key notices and budget alerts through the platform email add-on.
Scale when traffic grows
The proxy starts with 768 MB reserved and can use up to 2 GB. Raise CPU and RAM from the dashboard when request volume grows.

Why run LiteLLM on StackBlaze

  • Scale your appChange CPU, RAM, and replicas without a rebuild, or let autoscaling add replicas when CPU climbs.
  • Git-based CI/CDPush to GitHub and it builds and ships. Pull requests get their own preview URL on Pro and Business.
  • One-click rollbackA deploy went wrong? Go back to any previous successful deploy.
  • Environments built inProduction, staging, and dev as separate environments in one project, each with its own URL.
  • Built-in observabilityCPU, memory, and response time per app, live logs with search, and alerts to Slack, Discord, or a webhook.
  • Developer-first toolsA CLI and Stack MCP: deploy, read logs, and wire services from your terminal or your AI assistant.
  • Built for teamsInvite teammates with roles. An audit log records who changed what.
  • Managed Postgres with HAPostgres is run for you. Turn on high availability for a three-node cluster when the app needs it.

Common questions about LiteLLM

How do I sign in after deploy?

Open the public URL and sign in as admin. The password is generated at deploy and shown as UI_PASSWORD in the app’s Variables in the dashboard.

Is the API open to anyone who finds the URL?

No. Every model and admin route needs the master key (LITELLM_MASTER_KEY) or a virtual key you created. Only the health checks, the login page and the API reference answer without one.

Can I stop using the shared admin password?

Yes, and LiteLLM recommends it. Invite yourself as an admin under Internal Users, set your own password from the invite link, then turn off the UI_PASSWORD login with the master key (disable_env_credential_login). The master key can turn it back on.

Does it include a model?

No. LiteLLM routes to the providers you add, with your own API keys, or to a model you host yourself, such as Ollama.

Can I use a model I host myself?

Yes. Under Variables, set:

MODEL_URL
Your model server’s base URL, with or without /v1
MODEL_API_KEY
Its API key or password
MODEL_NAME
The name of the model it serves

After the restart it is listed under Models, and your apps reach it with a virtual key. You can also add it by hand under Models.

How do my apps call it?

Use any OpenAI SDK with the base URL set to your app’s /v1 endpoint and a virtual key as the API key. The model name is the one you gave the model in LiteLLM.

Are Enterprise features included?

The open source proxy is included. Features LiteLLM sells under its Enterprise licence, such as SSO for more than five users and audit logs, need a licence key from LiteLLM.

Can I use my own domain?

Yes. Attach a custom domain from the dashboard. The API, the admin UI and invite links move with it once you update PROXY_BASE_URL. Custom domains

What does it cost?

You pay for the compute, storage and bandwidth LiteLLM and Postgres use, not per request or per key. Model usage is billed by your providers. Pricing

More AI templates