Skip to main content

Fixing DBT and BigQuery Entity URN Alignment Issues

Angelo Bresci
Updated

Area: Ingestion Issues

Sub-Area: DBT Source Configuration

Issue

DBT ingestion unconditionally lowercases downstream BigQuery entity URNs regardless of the convert_urns_to_lowercase configuration flag, causing lineage mismatches between DBT and BigQuery entities. This occurs because BigQuery ingestion preserves original identifier casing while DBT ingestion forces lowercase conversion, resulting in case-sensitive URN mismatches that break lineage connections.

Error Messages

  • No explicit error messages - manifests as missing lineage between DBT models and BigQuery tables

You Might Be Asking

  • Why is lineage broken between my DBT models and BigQuery tables after upgrading DataHub?
  • Why do I see duplicate entities with slightly different casing in my catalog?
  • How can I fix URN case sensitivity issues between DBT and BigQuery?

Solution

  1. Add the convert_urns_to_lowercase: false configuration to your DBT ingestion recipe: 
    source:
  type: dbt
  config:
    convert_urns_to_lowercase: false
    # ... rest of your existing configuration
  2. Re-run your DBT ingestion with the updated configuration
  3. Verify that lineage now connects properly between DBT models and BigQuery tables
  4. Clean up any ghost lowercase entities that were created during the period when casing was misaligned:
    • Navigate to the ghost entity in the DataHub UI
    • Use the "Delete" option to soft-delete the phantom entry
    • Alternatively, use the DataHub REST API or SDK to programmatically remove these entities
  5. Monitor subsequent ingestion runs to ensure consistent URN casing

Additional Notes

This issue was introduced in DataHub v1.4.0.6 via a regression in the DBT ingestion source and was fixed in v1.5.0.15. The root cause was faulty conditional logic that bypassed the convert_urns_to_lowercase flag for non-DBT target platforms. BigQuery is case-sensitive for quoted identifiers, so exact URN matching is required for proper lineage. The fix makes the lowercasing behavior truly configurable rather than unconditional. This configuration change is required only once and affects only case-sensitive target platforms like BigQuery.

Related Documentation

Tags: dbt, bigquery, lineage, urn, case-sensitivity, convert_urns_to_lowercase, ingestion, configuration, regression

Related articles