AI models

AI models

Run an open model on its own rented GPU and call it like the OpenAI API, from any app or script.

An AI model is an open model served on its own GPU, behind an OpenAI-compatible URL with its own API key. It lives in a project environment, next to your apps. Pick a model from the catalog or use any Hugging Face model by its id (owner/name).

To use the model from LiteLLM, Kestra, Langflow or your own code, see Connect AI models to apps.

Deploy a model

In the dashboard

  1. 1

    Add an AI model

    In a project environment, add a service and pick AI model (GPU).
  2. 2

    Pick the model

    Choose a catalog model, or type a Hugging Face id. For gated models, the Hugging Face account menu next to the model shows which account downloads it.
  3. 3

    Choose where and how it runs

    Pick a region (United States, Canada, Europe, Asia Pacific or Any region), the number of GPUs, and On demand or Always on. The dialog shows the on-demand price per hour, the asleep price, the always-on monthly price and your GPU credit.

Deploy starts a GPU right away to download the model, and that time is billed. The data the model transfers is billed as bandwidth.

From the terminal

terminal
stackblaze models search qwen              # find chat models on Hugging Face
stackblaze models check Qwen/Qwen3-8B      # machine, gated or not, price
stackblaze models deploy Qwen/Qwen3-8B --name chat --wait

models check looks a model up without deploying. It shows the machine it needs, whether it is gated, and the price, or why it cannot run. models deploy takes these options:

OptionWhat it does
--nameName of the model in the environment (default: a generated one).
--regionWhere its GPUs are rented: us, ca, eu, apac or any.
--copiesHow many machines run the model and share the requests (plan-limited, default 1).
--gpu-typePin every machine to one GPU type from models sizes.
--always-onNever sleep. Billed while it runs.
--sleep-afterOn demand: sleep after 300, 900 or 3600 seconds without requests.
--waitWait until the model is running, showing the step, percent and download speed.

GPU sizes

The model decides its machine. It gets one GPU of the class it needs, or one machine with 2, 4 or 8 GPUs when no single GPU holds it. Prices are per GPU, per hour while a machine runs:

SizeGPU memoryPrice per GPU
small16 GBup to $0.29 per hour
medium24 GBup to $0.52 per hour
large48 GBup to $0.92 per hour
gpu8080 GBup to $1.50 per hour
gpu9494 GBup to $2.76 per hour
gpu140140 GBup to $6.90 per hour
gpu178178 GBup to $14.95 per hour

stackblaze models sizes shows the current prices, the regions, the catalog models and your plan's limits. Asleep, a model is billed for its disk only: up to $0.02 per hour.

Gated models

Some models, such as Llama and MedGemma, are gated on Hugging Face. To deploy one, sign in with Hugging Face for your workspace and accept the model's licence on huggingface.co with that same account. Until access is granted, the deploy is refused with a message like:

error
The Hugging Face account <user> cannot download <model> yet. Accept the model's licence on huggingface.co/<model> with that account, then deploy again.

Call the model

Every model has its own base URL and API key. The URL keeps the same form:

base URL
https://api.stackblaze.cloud/api/inference/<id>/v1

Copy the URL and key from the dashboard, or print them with stackblaze models url and stackblaze models key. Send the key as Authorization: Bearer <key>.

terminal
curl "$(stackblaze models url chat)/chat/completions" \
  -H "Authorization: Bearer $(stackblaze models key chat)" \
  -H 'Content-Type: application/json' \
  -d '{"model":"Qwen/Qwen3-8B","messages":[{"role":"user","content":"Hello"}]}'
SupportedDetails
Endpoints/v1/models, /v1/chat/completions and /v1/completions.
StreamingSet "stream": true for server-sent events.
JSON outputAsk for JSON with response_format.
ReasoningReasoning models (Qwen3, DeepSeek R1) return their thinking separately, in reasoning_content, not in the answer.

Warning

The API key is a live credential for your model, and the server records who reads it. Keep it in a secret variable, not in code.

On demand and always on

ModeHow it runsBilled
On demand (default)Sleeps after 5 minutes without requests (or 15 or 60). The next request wakes it.GPU time while awake; asleep, only a small disk charge.
Always onNever sleeps.The GPU the whole time.

Switch at any time:

terminal
stackblaze models set chat --always-on
stackblaze models set chat --sleep-after 900

What happens on wake

While a model is asleep, no GPU runs. The next request wakes it. That takes about a minute when its machine is still available. If the GPU was rented to someone else meanwhile, the platform rents a fresh one automatically. The model then downloads and loads again, which takes several minutes.

Requests that arrive during a wake are held, not failed. A request waits up to 150 seconds for a wake, and up to 270 seconds while a fresh GPU starts. If the model is still not ready, it gets HTTP 503 with a Retry-After header and one of these messages:

Retry-AfterMessage
30The model is waking up. Retry in about 30 seconds.
60This model's GPU was given to someone else while it slept, and the new GPU is still starting. Retry in about a minute.

Tip

Before a demo or a test run, wake the model first with stackblaze models wake. In your client, retry on 503 and honour Retry-After. If the first request must never wait, use Always on.

A model you put to sleep yourself (stackblaze models sleep or the dashboard) does not wake on requests. They get 503 until you wake it.

Copies

More copies run the model on separate machines that share the requests. They handle more requests at once and keep the model up if one machine fails. With one copy, a machine failure or maintenance means a few minutes down while the model moves. The number of copies is limited by your plan.

terminal
stackblaze models scale chat --copies 2

Billing

TopicDetails
GPU creditGPU time is paid from prepaid GPU credit. Check it with stackblaze billing gpu-credit and add more with stackblaze billing gpu-credit add <amount>.
Per secondBilled per second. GPU prices follow the market, so the price shown is the current range.
Deploy checkA deploy is refused when your credit does not cover one hour at the model's highest price.
Credit runs outModels are paused. Requests get 503 until you add credit.

Which GPUs it rents

Every rented machine has an x86-64 (amd64) CPU and, by default, a GPU with compute capability 8.0 or newer. Some GPU types are never rented; by default these are Tesla V100 and GB10. Platform admins can change the blocked types and set hardware minimums under Platform → Plans & rate card → Usage rates.

Manage a model

CommandWhat it does
stackblaze models listThe environment's models: model, machine, region, status, copies.
stackblaze models get <name>URL, status, machine, copies and sleep policy; while it starts, the step and download speed.
stackblaze models logs <name>Lifecycle events, or the model server's own output with --source model.
stackblaze models metrics <name>Requests, response time, tokens, errors and GPU load, over --since 5m to 7d.
stackblaze models sleep / wake <name>Put the model to sleep now, or wake it (about a minute).
stackblaze models rename <name> <new>Rename it. The URL and API key do not change.
stackblaze models scale <name>Change the number of copies with --copies.
stackblaze models replace-gpu <name>Release the GPU and rent a fresh one. Same URL and key; a cold start follows.
stackblaze models delete <name>Delete the model and release its GPU.

See the CLI reference for all options.

Good to know

TopicDetails
PromptsThe relay in front of your model does not store prompts or replies. It counts requests, response times and tokens for Metrics.
Slow startsA start stuck in one step is moved to another GPU once. If that is slow too, the model is put to sleep with a message saying why.
AvailabilityAI models may be switched off on your platform or not be part of your plan. The dashboard and CLI then say which.