Connect AI models to apps
Point LiteLLM, Kestra, Langflow or your own code at an AI model with three variables.
An AI model answers on an OpenAI-compatible URL, so any app that speaks the OpenAI API can use it. The AI-ready templates all read the same three variables:
| Variable | Value |
|---|---|
MODEL_URL | The model's base URL, ending in /v1: https://api.stackblaze.cloud/api/inference/<id>/v1 |
MODEL_API_KEY | The model's API key (dashboard, or stackblaze models key <name>). |
MODEL_NAME | The model id, for example meta-llama/Llama-3.1-8B-Instruct. |
Set them under the app's Variables. Saving a variable restarts the app.
Which to use
| Setup | Connect | Why |
|---|---|---|
| One model, one app | The app straight to the model | Nothing in between. |
| Several models or several apps | Apps to LiteLLM, LiteLLM to the models | One URL for every model, a key per app, usage per key and model. |
LiteLLM
The LiteLLM template is a gateway in front of your models. Apps call one URL with their own key.
- 1
Register the first model
SetMODEL_URL,MODEL_API_KEYandMODEL_NAME. After the restart the model is listed under Models. Optional:MODEL_ALIASis the name clients send; empty meansMODEL_NAME. - 2
Add more models
In the LiteLLM admin UI, add each further model under Models with its URL, key and name. - 3
Give each app a key
Create a virtual key per app under Virtual Keys. Apps then use base URLhttps://<litellm app URL>/v1, their virtual key, and the model name.
LiteLLM tracks spend per key and per model, so you can see which app uses what.
Kestra
In the Kestra template, set MODEL_URL to the model's URL or to LiteLLM's /v1, plus MODEL_API_KEY and MODEL_NAME. Flows read them as {{ envs.model_url }}, {{ envs.model_api_key }} and {{ envs.model_name }}.
This flow sends a question to the model and logs the answer. Kestra 2 uses the toJson filter and the fromJson() function:
id: ask-model
namespace: demo
inputs:
- id: question
type: STRING
defaults: "In one sentence, what is quantum annealing?"
tasks:
- id: ask
type: io.kestra.plugin.core.http.Request
uri: "{{ envs.model_url }}/chat/completions"
method: POST
contentType: application/json
headers:
Authorization: "Bearer {{ envs.model_api_key }}"
body: |
{"model": "{{ envs.model_name }}", "max_tokens": 800,
"messages": [{"role": "user", "content": {{ inputs.question | toJson }}}]}
- id: answer
type: io.kestra.plugin.core.log.Log
message: "{{ fromJson(outputs.ask.body).choices[0].message.content | trim }}"Kestra's built-in AI Copilot is separate: it uses Google Gemini, not the MODEL_* model.
Langflow
In the Langflow template, MODEL_URL, MODEL_API_KEY and MODEL_NAME become Global Variables at start. Langflow allows environment variables only in secret fields, so in an OpenAI component:
| Field | Value |
|---|---|
| OpenAI API Base | Type the base URL (the value of MODEL_URL). |
| API Key | Pick MODEL_API_KEY from the globe icon. |
| Model Name | Type the model name (the value of MODEL_NAME). |
Note
LANGFLOW_PROVIDER_CREDENTIAL_ALLOWED_HOSTS. The Langflow template adds the host of MODEL_URL to that list automatically. To call any other host with MODEL_API_KEY, add it to the variable, separated by commas. A Langflow app deployed before this was added needs the variable set by hand.vLLM gateway
The vLLM template is a small gateway for one model. It gives your apps a stable name and their own key, so the model can change without touching them.
| Side | Settings |
|---|---|
| Gateway | Set MODEL_URL, MODEL_API_KEY and MODEL_NAME. |
| Clients | Base URL https://<app URL>/v1, the key generated in API_KEY, and the model name in MODEL_ALIAS (default model). |
Any app or code
Use any OpenAI SDK. Set the base URL to the model's URL or to LiteLLM's /v1, with the matching key and model name.
Python
import os
from openai import OpenAI
client = OpenAI(
base_url=os.environ["MODEL_URL"], # ends in /v1
api_key=os.environ["MODEL_API_KEY"],
)
reply = client.chat.completions.create(
model=os.environ["MODEL_NAME"],
messages=[{"role": "user", "content": "Hello"}],
)
print(reply.choices[0].message.content)curl
curl "$MODEL_URL/chat/completions" \
-H "Authorization: Bearer $MODEL_API_KEY" \
-H 'Content-Type: application/json' \
-d "{\"model\":\"$MODEL_NAME\",\"messages\":[{\"role\":\"user\",\"content\":\"Hello\"}]}"Tip