dbt batch-test errors vanish from run_results.json

A dbt run can print failing tests in the console while an alerting system sees no failures. An open issue reports that when a batched test query itself hits a SQL error, member tests are absent from run_results.json and on-run-end results, despite being counted as errors in the summary. The example includes a missing BigQuery column.

Check the failure boundary

Run a small test batch that contains an intentionally invalid column in a nonproduction project. Compare console totals, command exit status, run_results.json entries, and on-run-end hook input. Distinguish a member test that returns failing rows from a batch query that cannot execute.

Keep alerts from going silent

Until the defect is fixed, fail the pipeline on the dbt process status and reconcile expected test IDs with artifact entries. A zero-alert artifact is not proof of a clean run if the console reports SQL errors. Give your observability tool a separate signal for artifact incompleteness.

Retest the exact integration

When changing dbt versions or batch-test settings, exercise the error path as well as a normal failing test. Verify that each expected test has an error result and that downstream notifications fire. Keep this regression test small so it validates the contract, not a full warehouse load.

Adovo’s view

A quality check that fails invisibly is worse than one that stops the job. The contract runs from test execution to the alert a human receives. Measure both ends before calling the pipeline governed.