By default a task runs only if every task before it succeeded. A trigger rule changes that: a cleanup task can run after a failure, a notification can run only when something failed. Used carelessly, though, a cleanup task makes a failed run look successful.

Copy and paste

Run once the upstream tasks have finished, whatever their result:

from airflow.sdk import task

@task(trigger_rule="all_done")
def cleanup():
    ...

extract() >> cleanup()

Clean up and keep the run failed: a teardown task

from airflow.sdk import task, teardown

@task
def create_tmp_table():
    ...

@task
def load():
    ...

@teardown
def drop_tmp_table():
    ...

tmp = create_tmp_table()
tmp >> load() >> drop_tmp_table().as_teardown(setups=tmp)

Run only when something failed:

@task(trigger_rule="one_failed")
def notify_failure():
    ...

A cleanup task can hide a failure

Airflow decides whether a DAG run failed by looking at its last tasks. In the first example, if extract fails and cleanup succeeds, the last task succeeded and the run is marked as successful. Nobody looking at the DAG list sees the failure.

Version extract Cleanup task DAG run
cleanup with all_done failed success success
Teardown task failed success failed
all_done plus a watcher failed success failed

A teardown task runs whatever happens to the tasks it follows, and Airflow ignores it when deciding the state of the run. If you cannot use teardowns, add a watcher task that fails the run whenever an upstream task failed:

@task(trigger_rule="one_failed", retries=0)
def watcher():
    raise RuntimeError("An upstream task failed")

e = extract()
e >> cleanup()
e >> watcher()

All trigger rules

trigger_rule The task runs when the upstream tasks…
all_success (default) all succeeded
all_failed all failed
all_done all finished, in any state
all_done_min_one_success all finished, none was skipped and at least one succeeded
all_skipped were all skipped
one_success include at least one success; it does not wait for the others
one_failed include at least one failure; it does not wait for the others
one_done include at least one success or failure
none_failed have no failure: each one succeeded or was skipped
none_failed_min_one_success have no failure and at least one success
none_skipped have no skipped task
always whatever their state, without waiting

none_failed_min_one_success is the usual rule for a task that joins the paths of a branch, where one path is always skipped. from airflow.sdk import TriggerRule provides the same values as constants, for example TriggerRule.ALL_DONE.

Tested with Airflow 3.3.2.

References