vLLM logo

Deploy vLLM

Use your own vLLM model server from every app in the project

AIDevelopmentUtilitiesvLLM
The deployed vLLM app’s API page: GET /v1/models, authorized with the app’s API key, returns 200 with the model name clients use

Run the model with vLLM on a GPU machine, then connect it here with its address, password and model name. Your apps get one OpenAI-compatible API with its own key, inside the project at http://vllm/v1 and on a public HTTPS URL. Streaming, JSON output, tool calling and reasoning pass through unchanged, and prompts are not logged.

What you get with vLLM

  • One address for your appsApps in the project call http://vllm/v1. Code outside the project calls the public HTTPS URL. Both use the OpenAI format, so OpenAI SDKs work by changing the base URL.
  • Its own API keyClients use the key generated in API_KEY. The GPU machine’s password stays in this one app and is never handed to clients.
  • A stable model nameClients ask for MODEL_ALIAS. Move to another GPU machine and you change MODEL_URL and MODEL_API_KEY only. No client has to change.
  • vLLM features pass throughStreaming, JSON output (response_format), tool calling and reasoning_content come back exactly as vLLM sends them.
  • No prompt loggingThere is no database and message logging is off, so prompts and answers are not stored by this app.
  • Up while the GPU sleepsThe health check does not depend on the GPU machine, so the app stays up while the machine is asleep or being replaced.

After you deploy vLLM

  1. 1
    Start vLLM on your GPU machine

    Note its HTTP address, its password and the name of the model it serves.

  2. 2
    Fill in three variables

    Under Variables, set:

    MODEL_URL
    The GPU machine’s HTTP address, with or without /v1
    MODEL_API_KEY
    Its password
    MODEL_NAME
    The model name vLLM serves

    Saving restarts the app with the new settings.

  3. 3
    Point your apps at it

    Set each app’s OpenAI settings to:

    Base URL
    http://vllm/v1 inside the project, or the app’s public URL + /v1 from outside
    API key
    The value of API_KEY
    Model
    The value of MODEL_ALIAS

How vLLM is wired

The template runs a small OpenAI-compatible gateway in your project. Each request is passed to your vLLM server with its password, and the answer comes back unchanged.

No GPU in the project
The model is loaded and served by vLLM on your GPU machine. This app only routes requests, so it needs little CPU and memory.
Keys kept apart
Clients authenticate with API_KEY. Requests without it get 401. Only the gateway knows the GPU machine’s password.
Long answers allowed
REQUEST_TIMEOUT is 600 seconds by default, so long answers from reasoning models are not cut off.
Encryption to the GPU
Clients reach the gateway over HTTPS. Traffic from the gateway to the GPU machine is encrypted only when MODEL_URL starts with https://.

Why run vLLM on StackBlaze

  • Scale your appChange CPU, RAM, and replicas without a rebuild, or let autoscaling add replicas when CPU climbs.
  • Git-based CI/CDPush to GitHub and it builds and ships. Pull requests get their own preview URL on Pro and Business.
  • One-click rollbackA deploy went wrong? Go back to any previous successful deploy.
  • Environments built inProduction, staging, and dev as separate environments in one project, each with its own URL.
  • Built-in observabilityCPU, memory, and response time per app, live logs with search, and alerts to Slack, Discord, or a webhook.
  • Developer-first toolsA CLI and Stack MCP: deploy, read logs, and wire services from your terminal or your AI assistant.
  • Built for teamsInvite teammates with roles. An audit log records who changed what.
  • Managed Postgres with HAPostgres is run for you. Turn on high availability for a three-node cluster when the app needs it.

Common questions about vLLM

Does this template run the model?

No. It connects a vLLM server you run on a GPU machine. The app itself needs no GPU.

What do I need from the GPU machine?

Three values: its HTTP address (MODEL_URL), its password (MODEL_API_KEY) and the model name vLLM serves (MODEL_NAME). The password is sent as an Authorization: Bearer header.

How do my other apps connect?

Set their OpenAI base URL to http://vllm/v1, their API key to API_KEY and the model to MODEL_ALIAS. From outside the project, use the app’s public URL followed by /v1.

Is reasoning kept apart from the answer?

Yes, when vLLM runs with a reasoning parser (for example --reasoning-parser qwen3). The thinking comes back in reasoning_content, in normal and streamed replies.

Are prompts logged?

Not by this app: it has no database and message logging is off. What the GPU machine logs depends on how vLLM is run there.

What happens when I replace the GPU machine?

Update MODEL_URL and MODEL_API_KEY, and MODEL_NAME if the model changed, then save. Clients keep the same URL, key and model name.

What does it cost?

On StackBlaze you pay for the small amount of compute and memory the gateway uses. The GPU machine is billed by whoever runs it. Pricing

More AI templates