Lakeflow Connect CDC costs lose their pipeline ID in billing rows

A data team can see its Lakeflow Connect bill but cannot allocate it by pipeline. In a recent Databricks community question on Reddit, the author says usage_metadata.dlt_pipeline_id became null in their usage rows after October 1. A reply speculates that the field moved; another points out that current documentation still uses it. Treat the change as an incident to verify, not a settled migration instruction.

Confirm which records are affected

Databricks’ cost guide filters managed connector rows with billing_origin_product = 'LAKEFLOW_CONNECT' and joins usage_metadata.dlt_pipeline_id to pipeline metadata. Compare before-and-after rows by account, workspace, SKU, usage date and pipeline type. Check whether nulls are isolated to a connector, compute mode or time window.

Keep the accounting honest

The billing table reference still lists dlt_pipeline_id for Lakeflow usage. Do not attribute null rows to a pipeline by a loose timestamp match and present the result as exact. Reconcile total DBUs at the product level, then identify the unallocated portion explicitly. Resource tags can help future analysis, but only if their propagation is verified on the relevant billing records.

Escalate with a small sample

Send Databricks Support masked examples of one populated and one null usage row, their pipeline IDs, dates and relevant workspace information. Ask whether the field is expected to populate for the specific integrated CDC configuration and whether any correction is planned.

Adovo’s view

Cost transparency is part of data platform reliability. A pipeline that runs while its spend loses ownership is an operational defect in the control plane, even if the data remains fresh.