Data & Analytics
OpenMetadata quality tests can fail when a table changes between counts
Separate actual invalid rows from a moving-window false failure in OpenMetadata column tests.
November 9, 2026
An alert says a column contains unexpected values, but a direct query finds none. An OpenMetadata issue reports that columnValuesToBeInSet with matchEnum and columnValuesToBeUnique compare counts from separate queries. On a continuously appended table, those queries may see different rows. A rolling time filter can move between them too. The reporter reproduced a failure on Trino with Iceberg even when a single-query check found zero invalid values.
Test the same population twice
Record table snapshot or commit ID, partition window, test definition and start time. Re-run the rule against a frozen partition or transactionally consistent snapshot. OpenMetadata's column-test configuration identifies the exact test parameters; the quality UI guide helps preserve the result and run context.
Compare the test's numerator and denominator only when they refer to the same population. A count mismatch is not evidence of an invalid value if new rows landed between reads. Use a single query for independent confirmation when the engine supports it, and inspect actual offending row IDs before escalating to the source team.
Design an alert people can trust
Schedule the test after a partition closes or evaluate a stable snapshot. If the business needs checks on live data, label the result provisional and rerun after the window settles. Avoid disabling all quality alerts because one implementation creates noisy failures. Preserve an escalation path for genuine bad values.
Track whether the upstream test engine changes its query strategy. Validate any fix against a table that receives appends during the run, not just a static fixture.
Adovo’s view
A quality rule is only as sound as the population it measures. Two accurate counts from different moments can produce a false verdict. Make the observation window explicit before asking a team to act on the alert.
