Deploy vLLM
Use your own vLLM model server from every app in the project

Run the model with vLLM on a GPU machine, then connect it here with its address, password and model name. Your apps get one OpenAI-compatible API with its own key, inside the project at http://vllm/v1 and on a public HTTPS URL. Streaming, JSON output, tool calling and reasoning pass through unchanged, and prompts are not logged.
What you get with vLLM
- One address for your appsApps in the project call http://vllm/v1. Code outside the project calls the public HTTPS URL. Both use the OpenAI format, so OpenAI SDKs work by changing the base URL.
- Its own API keyClients use the key generated in API_KEY. The GPU machine’s password stays in this one app and is never handed to clients.
- A stable model nameClients ask for MODEL_ALIAS. Move to another GPU machine and you change MODEL_URL and MODEL_API_KEY only. No client has to change.
- vLLM features pass throughStreaming, JSON output (response_format), tool calling and reasoning_content come back exactly as vLLM sends them.
- No prompt loggingThere is no database and message logging is off, so prompts and answers are not stored by this app.
- Up while the GPU sleepsThe health check does not depend on the GPU machine, so the app stays up while the machine is asleep or being replaced.
After you deploy vLLM
- 1Start vLLM on your GPU machine
Note its HTTP address, its password and the name of the model it serves.
- 2Fill in three variables
Under Variables, set:
MODEL_URL- The GPU machine’s HTTP address, with or without
/v1 MODEL_API_KEY- Its password
MODEL_NAME- The model name vLLM serves
Saving restarts the app with the new settings.
- 3Point your apps at it
Set each app’s OpenAI settings to:
Base URLhttp://vllm/v1inside the project, or the app’s public URL +/v1from outsideAPI key- The value of
API_KEY Model- The value of
MODEL_ALIAS
How vLLM is wired
The template runs a small OpenAI-compatible gateway in your project. Each request is passed to your vLLM server with its password, and the answer comes back unchanged.
- No GPU in the project
- The model is loaded and served by vLLM on your GPU machine. This app only routes requests, so it needs little CPU and memory.
- Keys kept apart
- Clients authenticate with API_KEY. Requests without it get 401. Only the gateway knows the GPU machine’s password.
- Long answers allowed
- REQUEST_TIMEOUT is 600 seconds by default, so long answers from reasoning models are not cut off.
- Encryption to the GPU
- Clients reach the gateway over HTTPS. Traffic from the gateway to the GPU machine is encrypted only when MODEL_URL starts with https://.
Why run vLLM on StackBlaze
- Scale your appChange CPU, RAM, and replicas without a rebuild, or let autoscaling add replicas when CPU climbs.
- Git-based CI/CDPush to GitHub and it builds and ships. Pull requests get their own preview URL on Pro and Business.
- One-click rollbackA deploy went wrong? Go back to any previous successful deploy.
- Environments built inProduction, staging, and dev as separate environments in one project, each with its own URL.
- Built-in observabilityCPU, memory, and response time per app, live logs with search, and alerts to Slack, Discord, or a webhook.
- Developer-first toolsA CLI and Stack MCP: deploy, read logs, and wire services from your terminal or your AI assistant.
- Built for teamsInvite teammates with roles. An audit log records who changed what.
- Managed Postgres with HAPostgres is run for you. Turn on high availability for a three-node cluster when the app needs it.
Common questions about vLLM
Does this template run the model?
No. It connects a vLLM server you run on a GPU machine. The app itself needs no GPU.
What do I need from the GPU machine?
Three values: its HTTP address (MODEL_URL), its password (MODEL_API_KEY) and the model name vLLM serves (MODEL_NAME). The password is sent as an Authorization: Bearer header.
How do my other apps connect?
Set their OpenAI base URL to http://vllm/v1, their API key to API_KEY and the model to MODEL_ALIAS. From outside the project, use the app’s public URL followed by /v1.
Is reasoning kept apart from the answer?
Yes, when vLLM runs with a reasoning parser (for example --reasoning-parser qwen3). The thinking comes back in reasoning_content, in normal and streamed replies.
Are prompts logged?
Not by this app: it has no database and message logging is off. What the GPU machine logs depends on how vLLM is run there.
What happens when I replace the GPU machine?
Update MODEL_URL and MODEL_API_KEY, and MODEL_NAME if the model changed, then save. Clients keep the same URL, key and model name.
What does it cost?
On StackBlaze you pay for the small amount of compute and memory the gateway uses. The GPU machine is billed by whoever runs it. Pricing
More AI templates
open-webuiChatGPT-style web interface for Ollama and any OpenAI-compatible API — supports RAG, web search, image generation, and multi-model chat.
LiteLLMLLM gateway with one OpenAI-compatible API for 100+ model providers. Virtual keys, budgets, rate limits, spend tracking and an admin UI.
LangflowVisual builder for AI agents and RAG workflows. Drag components onto a canvas, connect any LLM or vector store, and serve each flow as an API.
SillyTavernSelf-hosted chat and roleplay front end for LLMs. Bring your own model (OpenAI, Claude, OpenRouter, Ollama and more), with characters, group chats, lorebooks and per-user accounts.