AI models
Run an open model on its own rented GPU and call it like the OpenAI API, from any app or script.
An AI model is an open model served on its own GPU, behind an OpenAI-compatible URL with its own API key. It lives in a project environment, next to your apps. Pick a model from the catalog or use any Hugging Face model by its id (owner/name).
To use the model from LiteLLM, Kestra, Langflow or your own code, see Connect AI models to apps.
Deploy a model
In the dashboard
- 1
Add an AI model
In a project environment, add a service and pick AI model (GPU). - 2
Pick the model
Choose a catalog model, or type a Hugging Face id. For gated models, the Hugging Face account menu next to the model shows which account downloads it. - 3
Choose where and how it runs
Pick a region (United States, Canada, Europe, Asia Pacific or Any region), the number of GPUs, and On demand or Always on. The dialog shows the on-demand price per hour, the asleep price, the always-on monthly price and your GPU credit.
Deploy starts a GPU right away to download the model, and that time is billed. The data the model transfers is billed as bandwidth.
From the terminal
stackblaze models search qwen # find chat models on Hugging Face
stackblaze models check Qwen/Qwen3-8B # machine, gated or not, price
stackblaze models deploy Qwen/Qwen3-8B --name chat --waitmodels check looks a model up without deploying. It shows the machine it needs, whether it is gated, and the price, or why it cannot run. models deploy takes these options:
| Option | What it does |
|---|---|
--name | Name of the model in the environment (default: a generated one). |
--region | Where its GPUs are rented: us, ca, eu, apac or any. |
--copies | How many machines run the model and share the requests (plan-limited, default 1). |
--gpu-type | Pin every machine to one GPU type from models sizes. |
--always-on | Never sleep. Billed while it runs. |
--sleep-after | On demand: sleep after 300, 900 or 3600 seconds without requests. |
--wait | Wait until the model is running, showing the step, percent and download speed. |
GPU sizes
The model decides its machine. It gets one GPU of the class it needs, or one machine with 2, 4 or 8 GPUs when no single GPU holds it. Prices are per GPU, per hour while a machine runs:
| Size | GPU memory | Price per GPU |
|---|---|---|
| small | 16 GB | up to $0.29 per hour |
| medium | 24 GB | up to $0.52 per hour |
| large | 48 GB | up to $0.92 per hour |
| gpu80 | 80 GB | up to $1.50 per hour |
| gpu94 | 94 GB | up to $2.76 per hour |
| gpu140 | 140 GB | up to $6.90 per hour |
| gpu178 | 178 GB | up to $14.95 per hour |
stackblaze models sizes shows the current prices, the regions, the catalog models and your plan's limits. Asleep, a model is billed for its disk only: up to $0.02 per hour.
Gated models
Some models, such as Llama and MedGemma, are gated on Hugging Face. To deploy one, sign in with Hugging Face for your workspace and accept the model's licence on huggingface.co with that same account. Until access is granted, the deploy is refused with a message like:
The Hugging Face account <user> cannot download <model> yet. Accept the model's licence on huggingface.co/<model> with that account, then deploy again.Call the model
Every model has its own base URL and API key. The URL keeps the same form:
https://api.stackblaze.cloud/api/inference/<id>/v1Copy the URL and key from the dashboard, or print them with stackblaze models url and stackblaze models key. Send the key as Authorization: Bearer <key>.
curl "$(stackblaze models url chat)/chat/completions" \
-H "Authorization: Bearer $(stackblaze models key chat)" \
-H 'Content-Type: application/json' \
-d '{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"Hello"}]}'| Supported | Details |
|---|---|
| Endpoints | /v1/models, /v1/chat/completions and /v1/completions. |
| Streaming | Set "stream": true for server-sent events. |
| JSON output | Ask for JSON with response_format. |
| Reasoning | Reasoning models (Qwen3, DeepSeek R1) return their thinking separately, in reasoning_content, not in the answer. |
Warning
On demand and always on
| Mode | How it runs | Billed |
|---|---|---|
| On demand (default) | Sleeps after 5 minutes without requests (or 15 or 60). The next request wakes it. | GPU time while awake; asleep, only a small disk charge. |
| Always on | Never sleeps. | The GPU the whole time. |
Switch at any time:
stackblaze models set chat --always-on
stackblaze models set chat --sleep-after 900What happens on wake
While a model is asleep, no GPU runs. The next request wakes it. That takes about a minute when its machine is still available. If the GPU was rented to someone else meanwhile, the platform rents a fresh one automatically. The model then downloads and loads again, which takes several minutes.
Requests that arrive during a wake are held, not failed. A request waits up to 150 seconds for a wake, and up to 270 seconds while a fresh GPU starts. If the model is still not ready, it gets HTTP 503 with a Retry-After header and one of these messages:
| Retry-After | Message |
|---|---|
| 30 | The model is waking up. Retry in about 30 seconds. |
| 60 | This model's GPU was given to someone else while it slept, and the new GPU is still starting. Retry in about a minute. |
Tip
stackblaze models wake. In your client, retry on 503 and honour Retry-After. If the first request must never wait, use Always on.A model you put to sleep yourself (stackblaze models sleep or the dashboard) does not wake on requests. They get 503 until you wake it.
Copies
More copies run the model on separate machines that share the requests. They handle more requests at once and keep the model up if one machine fails. With one copy, a machine failure or maintenance means a few minutes down while the model moves. The number of copies is limited by your plan.
stackblaze models scale chat --copies 2Billing
| Topic | Details |
|---|---|
| GPU credit | GPU time is paid from prepaid GPU credit. Check it with stackblaze billing gpu-credit and add more with stackblaze billing gpu-credit add <amount>. |
| Per second | Billed per second. GPU prices follow the market, so the price shown is the current range. |
| Deploy check | A deploy is refused when your credit does not cover one hour at the model's highest price. |
| Credit runs out | Models are paused. Requests get 503 until you add credit. |
Which GPUs it rents
Every rented machine has an x86-64 (amd64) CPU and, by default, a GPU with compute capability 8.0 or newer. Some GPU types are never rented; by default these are Tesla V100 and GB10. Platform admins can change the blocked types and set hardware minimums under Platform → Plans & rate card → Usage rates.
Manage a model
| Command | What it does |
|---|---|
stackblaze models list | The environment's models: model, machine, region, status, copies. |
stackblaze models get <name> | URL, status, machine, copies and sleep policy; while it starts, the step and download speed. |
stackblaze models logs <name> | Lifecycle events, or the model server's own output with --source model. |
stackblaze models metrics <name> | Requests, response time, tokens, errors and GPU load, over --since 5m to 7d. |
stackblaze models sleep / wake <name> | Put the model to sleep now, or wake it (about a minute). |
stackblaze models rename <name> <new> | Rename it. The URL and API key do not change. |
stackblaze models scale <name> | Change the number of copies with --copies. |
stackblaze models replace-gpu <name> | Release the GPU and rent a fresh one. Same URL and key; a cold start follows. |
stackblaze models delete <name> | Delete the model and release its GPU. |
See the CLI reference for all options.
Good to know
| Topic | Details |
|---|---|
| Prompts | The relay in front of your model does not store prompts or replies. It counts requests, response times and tokens for Metrics. |
| Slow starts | A start stuck in one step is moved to another GPU once. If that is slow too, the model is put to sleep with a message saying why. |
| Availability | AI models may be switched off on your platform or not be part of your plan. The dashboard and CLI then say which. |