Fixing Column Level Lineage Not Displaying in UI for Teradata – DataHub Customer Support Portal

Area: Product Issues

Sub-Area: Column Lineage UI Display

Issue

Column-level lineage data is present in the DataHub API but not displaying in the UI for Teradata datasets. While table-level lineage appears correctly, column lineage connections are missing from the lineage visualization, even though the fine-grained lineage relationships exist in the backend.

You Might Be Asking

Why can I see column lineage in API calls but not in the UI?
Why does table-level lineage work but column-level doesn't for Teradata?
How do I enable column lineage visualization for Teradata sources?

Solution

This issue occurs due to case mismatches between schema extraction and lineage resolution phases in Teradata ingestion. Follow these steps to resolve:

Upgrade to DataHub SDK v1.6.0 or later
```
pip install acryl-datahub==1.6.0
```

Update your Teradata ingestion recipe with required settings:

source:
  type: teradata
  config:
    convert_urns_to_lowercase: true
    incremental_lineage: false
    # ... other config parameters

For cross-database lineage scenarios, include all referenced databases in scope:

source:
  type: teradata
  config:
    databases:
      - YOUR_VIEW_DATABASE
      - YOUR_TABLE_DATABASE  # Include upstream table databases
    database_pattern:
      allow:
        - "^YOUR_VIEW_DATABASE$"
        - "^YOUR_TABLE_DATABASE$"
    convert_urns_to_lowercase: true
    incremental_lineage: false

Run the ingestion with incremental_lineage: false to force full reprocessing:
```
datahub ingest -c your_teradata_recipe.yml
```
After successful ingestion, re-enable incremental lineage for future runs:
```
incremental_lineage: true  # Can be re-enabled after initial fix
```
Verify column lineage in the UI by navigating to:
- Dataset → Lineage Tab → Toggle "Show Columns" option
- Or use the "Visualize Lineage" button for full graph view

Additional Notes

The root cause was a case mismatch between Teradata's uppercase system catalog responses and SQL parser normalization to lowercase during lineage resolution. This prevented the UI from linking column relationships even when the data was correctly captured. The fix ensures consistent identifier casing throughout the ingestion pipeline. Setting incremental_lineage: false is critical for the initial fix to replace existing incorrect lineage data rather than appending to it.

Issue

You Might Be Asking

Solution

Additional Notes

Related Documentation