Skip to main content

Unity Catalog Tag Inheritance: Schema Tags Not Propagated to Tables in DataHub

Angelo Bresci
Updated

Area: Ingestion Issues

Sub-Area: Databricks Unity Catalog Source — Tag Ingestion Behavior

Issue

When a schema in Databricks Unity Catalog is tagged (for example, with a tag like team:my_team), Unity Catalog automatically propagates that tag to all tables within the schema, and the Databricks UI displays those inherited tags on the table detail pages. However, DataHub's Unity Catalog ingestion connector does not replicate this inheritance behavior. Tags applied at the schema level are ingested and applied only to the schema container entity in DataHub — they are never automatically propagated down to child table entities. As a result, tables in DataHub will not display schema-level tags, even after a successful ingestion run.

You Might Be Asking

  • Why doesn't my table in DataHub show the tag I applied to its parent schema in Unity Catalog?
  • Is DataHub supposed to inherit schema-level tags on tables when ingesting from Databricks Unity Catalog?
  • Will running ingestion again cause schema tags to appear on tables in DataHub?
  • Is this a bug or expected behavior in the DataHub Unity Catalog connector?

Root Cause

DataHub's Unity Catalog connector queries tags at each metadata level independently using Databricks' information_schema SQL views:

  • Catalog tags: information_schema.catalog_tags
  • Schema tags: information_schema.schema_tags
  • Table tags: information_schema.table_tags
  • Column tags: information_schema.column_tags

The critical constraint is that information_schema.table_tags in Unity Catalog returns only tags directly assigned to a table. It does not include tags that are inherited from a parent schema or catalog, even though those inherited tags are visible in the Databricks UI. Because the connector queries this view directly and contains no tag merging or inheritance propagation logic, schema-level tags are never applied to child table entities in DataHub. This is a feature gap in the connector, not a regression from previously working behavior.

Solution

The current workaround is to apply tags directly to each table in Unity Catalog rather than relying on schema-level inheritance. Once a tag is assigned directly to a table, DataHub will ingest it on the next scheduled ingestion run.

  1. Apply the tag directly to each table in Unity Catalog. For example, using the Databricks SQL interface: 
    ALTER TABLE <catalog>.<schema>.<table_name>
SET TAGS ('team' = 'my_team');
    Replace <catalog>, <schema>, and <table_name> with your actual catalog, schema, and table names. Repeat for each table that should carry the tag.
  2. Verify the tag appears in information_schema.table_tags (not just inherited via the Databricks UI): 
    SELECT *
FROM <catalog>.information_schema.table_tags
WHERE schema_name = '<schema>'
  AND table_name = '<table_name>';
    Confirm the tag row is present with tag_name = 'team' and tag_value = 'my_team'. If this query returns a result, DataHub will ingest it.
  3. Run or wait for your next DataHub ingestion. The tag will appear on the table entity in DataHub after the ingestion completes successfully. No changes to the DataHub ingestion recipe configuration are required.

Additional Notes

  • This is a known feature gap in the DataHub Unity Catalog source connector, not a defect or regression. The tag inheritance feature has never been implemented in any version of the connector.
  • Re-running ingestion without applying tags directly to tables will not change the outcome — the timing of when a schema tag was applied does not affect whether it appears on tables in DataHub.
  • Schema-level tags are ingested correctly and will appear on the schema container entity in DataHub. Only the propagation to child tables is not supported.
  • A longer-term fix would require the connector to either (a) perform a SQL join between schema_tags and table_tags to merge inherited tags, or (b) use the Databricks Unity Catalog REST API's effective tags endpoint, which returns the full set of effective tags including inherited ones. Neither approach is currently implemented. If this capability is important to your workflow, consider submitting a feature request to the DataHub project.
  • This behavior applies to both DataHub Cloud and DataHub OSS deployments using the Unity Catalog ingestion source.

Related Documentation

Tags: unity-catalog, databricks, tag-inheritance, schema-tags, table-tags, ingestion, feature-gap, information_schema, metadata-propagation, workaround