Deploy LiteLLM
Run your own LLM gateway with LiteLLM
Deploy LiteLLM with managed Postgres, platform email, and a public HTTPS URL. Give every app and teammate its own key with a budget, call any model through one OpenAI-compatible API, and see what each request cost. Bring your own provider keys.
What you get with LiteLLM
- One API for every modelCall OpenAI, Anthropic, Gemini, Bedrock, a local Ollama model or any other provider LiteLLM supports with the same OpenAI SDK code.
- Closed by defaultEvery API route needs the master key or a virtual key. The master key, the admin password and the encryption key are generated for this instance at deploy.
- Virtual keys and teamsHand out a key per app, person or team, limited to the models you pick, with its own budget and rate limits. Revoke it without touching the provider key.
- Spend trackingEvery request is logged with its tokens and cost per key, team and model, so you can see where the money goes.
- Admin UIAdd models, create keys and teams, test a key in the playground and read the request logs from the browser.
- Encrypted provider keysProvider API keys added in the UI are stored encrypted in Postgres with a salt key generated at deploy.
After you deploy LiteLLM
- 1Sign in to the admin UI
Open the public URL; it lands on the LiteLLM login. Sign in as admin with UI_PASSWORD from the app’s Variables in the dashboard.
- 2Connect your own model (optional)
For a model you run yourself, such as vLLM on a rented GPU, set under Variables:
MODEL_URL- Your model server’s base URL, with or without
/v1 MODEL_API_KEY- Its API key or password
MODEL_NAME- The name of the model it serves
MODEL_ALIAS- The name your apps send. Empty: the same as
MODEL_NAME
Saving restarts the app, and the model is listed under Models.
- 3Add a model
Under Models, add a provider and paste its API key, or point a model at an OpenAI-compatible or Ollama endpoint you run.
- 4Create a virtual key and call it
Under Virtual Keys, create a key for your app with the models and budget it may use, then call your URL’s /v1 endpoint with any OpenAI SDK.
How LiteLLM is wired
LiteLLM and Postgres ship together, with the platform email add-on for invites and alerts. The public hostname serves the API, the admin UI at /ui and the API reference at /docs.
- Postgres holds the gateway
- Models, virtual keys, teams, users and spend logs live in the attached database. Redeploy the app and everything is still there.
- Keep the salt key
- LITELLM_SALT_KEY encrypts the provider keys you add. It is generated once at deploy. Changing it later makes the saved keys unreadable.
- Email for invites and alerts
- Turn on email alerting in the admin UI and LiteLLM sends invites, new key notices and budget alerts through the platform email add-on.
- Scale when traffic grows
- The proxy starts with 768 MB reserved and can use up to 2 GB. Raise CPU and RAM from the dashboard when request volume grows.
Why run LiteLLM on StackBlaze
- Scale your appChange CPU, RAM, and replicas without a rebuild, or let autoscaling add replicas when CPU climbs.
- Git-based CI/CDPush to GitHub and it builds and ships. Pull requests get their own preview URL on Pro and Business.
- One-click rollbackA deploy went wrong? Go back to any previous successful deploy.
- Environments built inProduction, staging, and dev as separate environments in one project, each with its own URL.
- Built-in observabilityCPU, memory, and response time per app, live logs with search, and alerts to Slack, Discord, or a webhook.
- Developer-first toolsA CLI and Stack MCP: deploy, read logs, and wire services from your terminal or your AI assistant.
- Built for teamsInvite teammates with roles. An audit log records who changed what.
- Managed Postgres with HAPostgres is run for you. Turn on high availability for a three-node cluster when the app needs it.
Common questions about LiteLLM
How do I sign in after deploy?
Open the public URL and sign in as admin. The password is generated at deploy and shown as UI_PASSWORD in the app’s Variables in the dashboard.
Is the API open to anyone who finds the URL?
No. Every model and admin route needs the master key (LITELLM_MASTER_KEY) or a virtual key you created. Only the health checks, the login page and the API reference answer without one.
Can I stop using the shared admin password?
Yes, and LiteLLM recommends it. Invite yourself as an admin under Internal Users, set your own password from the invite link, then turn off the UI_PASSWORD login with the master key (disable_env_credential_login). The master key can turn it back on.
Does it include a model?
No. LiteLLM routes to the providers you add, with your own API keys, or to a model you host yourself, such as Ollama.
Can I use a model I host myself?
Yes. Under Variables, set:
MODEL_URL- Your model server’s base URL, with or without
/v1 MODEL_API_KEY- Its API key or password
MODEL_NAME- The name of the model it serves
After the restart it is listed under Models, and your apps reach it with a virtual key. You can also add it by hand under Models.
How do my apps call it?
Use any OpenAI SDK with the base URL set to your app’s /v1 endpoint and a virtual key as the API key. The model name is the one you gave the model in LiteLLM.
Are Enterprise features included?
The open source proxy is included. Features LiteLLM sells under its Enterprise licence, such as SSO for more than five users and audit logs, need a licence key from LiteLLM.
Can I use my own domain?
Yes. Attach a custom domain from the dashboard. The API, the admin UI and invite links move with it once you update PROXY_BASE_URL. Custom domains
What does it cost?
You pay for the compute, storage and bandwidth LiteLLM and Postgres use, not per request or per key. Model usage is billed by your providers. Pricing
More AI templates
vLLMConnect a vLLM model server running on a GPU and use it from your project through one OpenAI-compatible API with its own key. Streaming, JSON output, tool calling and reasoning pass through unchanged.
open-webuiChatGPT-style web interface for Ollama and any OpenAI-compatible API — supports RAG, web search, image generation, and multi-model chat.
LangflowVisual builder for AI agents and RAG workflows. Drag components onto a canvas, connect any LLM or vector store, and serve each flow as an API.
SillyTavernSelf-hosted chat and roleplay front end for LLMs. Bring your own model (OpenAI, Claude, OpenRouter, Ollama and more), with characters, group chats, lorebooks and per-user accounts.