How to trigger a dbt Cloud job from the API

A schedule is not always the right trigger. A job may need to start when a loader finishes, when a file arrives, or from an orchestrator. dbt’s Administrative API starts a job with a single POST request and returns the ID of the run, which you can then follow until it finishes.

Copy and paste

What you need:

• your access URL, shown in Account settings, for example ab123.us1.dbt.com;
• the account ID and the job ID, both visible in the job’s URL;
• a service token with permission to run jobs, created in Account settings > Service tokens.

Start the job:

curl -X POST "https://$DBT_CLOUD_HOST/api/v2/accounts/$DBT_CLOUD_ACCOUNT_ID/jobs/$DBT_CLOUD_JOB_ID/run/" \
  -H "Authorization: Token $DBT_CLOUD_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"cause": "Triggered by the nightly loader"}'

Override the job’s settings for this run only:

{
  "cause": "Rebuild EU after a late load",
  "steps_override": ["dbt build --select tag:regional --vars '{region: eu}'"],
  "git_branch": "main",
  "threads_override": 8,
  "target_name_override": "prod"
}

Start the job and wait for the result, in Python:

import os
import time

import requests

HOST = os.environ["DBT_CLOUD_HOST"]              # for example ab123.us1.dbt.com
ACCOUNT_ID = os.environ["DBT_CLOUD_ACCOUNT_ID"]
JOB_ID = os.environ["DBT_CLOUD_JOB_ID"]
TOKEN = os.environ["DBT_CLOUD_TOKEN"]

BASE = f"https://{HOST}/api/v2/accounts/{ACCOUNT_ID}"
HEADERS = {"Authorization": f"Token {TOKEN}", "Content-Type": "application/json"}
STATUS = {1: "queued", 2: "starting", 3: "running", 10: "success", 20: "error", 30: "cancelled"}

response = requests.post(
    f"{BASE}/jobs/{JOB_ID}/run/",
    headers=HEADERS,
    json={"cause": "Triggered by the nightly loader"},
    timeout=30,
)
response.raise_for_status()
run = response.json()["data"]
print(f"Started run {run['id']}: {run['href']}")

while True:
    run = requests.get(f"{BASE}/runs/{run['id']}/", headers=HEADERS, timeout=30).json()["data"]
    print(STATUS.get(run["status"], run["status"]))
    if run["status"] in (10, 20, 30):
        break
    time.sleep(30)

if run["status"] != 10:
    raise SystemExit(f"Run {run['id']} ended with status {STATUS[run['status']]}")

Started run 99: https://ab123.us1.dbt.com/deploy/1/projects/5/runs/99/
queued
running
success

The request

• cause is required: a short text that appears in the run’s history, so people know why it ran.
• The overrides apply to this run only: steps_override replaces the job’s commands, and git_branch, schema_override, threads_override, target_name_override, timeout_seconds_override and generate_docs_override replace the job’s other settings.
• The response contains the run under data: id for polling and href for the link to the run in dbt.

Run statuses

Things to know

• Tokens. Use a service token for automation: it belongs to the account, not to a person who may leave. Personal access tokens are for scripts you run yourself.
• The access URL. Older accounts used cloud.getdbt.com; newer ones have a host with an account prefix and a region, such as ab123.us1.dbt.com. Use the one in your account settings.
• Plans. The Administrative API is available on Starter, Enterprise and Enterprise+ plans.
• From Airflow. The dbt Cloud provider wraps these calls in one operator; see how to trigger a dbt Cloud job from Airflow.

The script was tested against a local stand-in for the API; the endpoints and fields come from the dbt documentation and the dbt Cloud provider for Airflow. Not run against a live dbt account.