Data & Analytics
One malformed Glue struct type can hide an entire catalog table
Diagnose an OpenMetadata Glue ingestion gap caused by one unparseable nested column type.
November 8, 2026
An OpenMetadata Glue issue describes a table from DynamoDB whose nested type string does not follow the parser's strict Hive-style grammar. Parsing raises a ValueError; the connector skips the whole table rather than just marking the field unknown. That means the catalog can lose columns and lineage for a real source table while an operator sees only a connector warning. The exact raw Glue type in the report was truncated, so reproduction needs a fresh source capture.
Find the missing table before rewriting types
Compare Glue's table inventory with the catalog's ingested tables for the same database and schema. Locate the first warning naming the table and fetch the full Type string through Glue, with sensitive values removed. Separate a parser failure from permissions, filtering, or a deleted source. The official Glue connector guide describes the metadata workflow and required read permissions; it does not promise that every nested type is parsed.
Contain the business impact
Identify reports, ownership records, and lineage checks that depend on the missing table. Mark the catalog entry as incomplete or temporarily document the source outside the catalog so users do not mistake absence for nonexistence. Do not silently edit production Glue schemas just to satisfy a catalog parser: downstream jobs may rely on the original type. If a metadata-only normalization is available, test it on a copy and record the mapping.
Set an ingestion completeness gate
After a connector upgrade or workaround, require the table and unaffected columns to appear, and compare row counts of expected catalog entities before and after the run. Capture the full type string, connector version, and sanitized warning for maintainers. A pipeline status alone cannot detect a table-level omission.
Adovo’s view
A catalog that quietly omits one table is more dangerous than a loud parsing error. Treat source-to-catalog completeness as a measured contract, especially for nested data.