Airflow clearing on the latest DAG version can remove mapped tasks

A cleared Airflow run may appear complete while the mapped work never runs again. In an open issue, clearing a DAG run with “run on latest version” also clears the upstream task that supplies a mapped task’s input. The reporter found the mapped instances moved to removed and stayed there after the producer reran; downstream work then continued as though mapping were complete.

Identify runs at risk

Look for a DAG with dynamic task mapping where an upstream task emits the list or count used by expand(). Airflow’s documentation explains that mapped instances are created at runtime from that upstream output. The latest-version rerun option is documented as experimental. Before clearing an affected run, record the DAG bundle version, original map indexes, upstream XCom, expected item count, and downstream data partition or output.

Check the run after clearing

Do not use a green downstream task as proof that the mapped work was recreated. Compare the new producer output with the number and state of mapped task instances. The issue reports that when the producer is cleared, Airflow cannot yet know the new mapping cardinality and marks existing mapped rows removed. If the expected instances are absent, stop dependent processing and preserve the run state. Test any recovery procedure against the exact Airflow version and DAG shape in a lower environment before applying it to production.

Put a business assertion at the boundary

For consequential pipelines, assert expected source item count against completed mapped outputs and final destination rows. Treat a missing mapped result as a failure even when the scheduler marks downstream work successful. Attach that check to incident response and release validation for DAG changes. The guard should prove the work happened, not merely that orchestration reached its final node.

Adovo’s view

Workflow completion is not data completion. A scheduler can advance through a graph while the unit of work disappears. Tie the run’s success to the mapped outputs the business expects, and make a rerun prove it rebuilt them.