Understanding and Troubleshooting Lineage Connections in DataHub – DataHub Customer Support Portal

Area: Product Issues

Sub-Area: Lineage Visualization and Troubleshooting

Issue

Users may encounter broken lineage connections that were previously working, or need clarification on what each lineage connection represents in the Lineage Explorer. Lineage breaks often occur due to ingestion failures, URN mismatches, or connector version regressions, while understanding lineage connection sources is important for data governance and impact analysis.

You Might Be Asking

Why did my lineage connections disappear when they were working before?
What exactly do the solid and dotted lines represent in the lineage graph?
How can I see what created a specific lineage connection?
Can I tell if a lineage edge came from SQL parsing, ingestion metadata, or manual creation?

Solution

For Broken Lineage Connections:

Check URN Consistency: Navigate to both affected entities and copy their full URNs from the "Copy URN" button. Verify that URNs haven't changed due to platform instance naming changes or environment differences (e.g., PROD vs PRODUCTION).
Review Ingestion History: Go to Ingestion → your source → run history and look for warnings or configuration changes in recent runs compared to successful runs from when lineage was working.
Examine Audit Logs: In either entity's Timeline or Audit tab, look for deleteLineage or updateLineage events that might explain the disappearance.
Check for Stateful Ingestion Changes: If stateful ingestion is enabled with cleanup strategies, soft-delete signals may have removed lineage edges that weren't detected in the current run.
Verify TTL Settings: Lineage edges can carry TTL values - if ingestion temporarily stopped emitting lineage for entities, edges may have expired.

For Understanding Lineage Connection Details:

Visual Indicators:
- Solid lines = System-generated lineage from ingestion sources
- Dotted lines = Manually created or edited lineage
Finding Connection Evidence:
- Look for query assets (Snowflake icons) between connected entities - click these to see the underlying SQL logic
- Check view definitions directly on assets when available
- Use APIs (Python, GraphQL, OpenAPI) to inspect upstreamLineage aspects for detailed metadata
Data Source Origins:
- Snowflake lineage comes from ACCESS_HISTORY, QUERY_HISTORY, and COPY_HISTORY system views
- Platform icons and source information are visible in the UI or via API
- Manual lineage can be identified by dotted line styling

For Managing Manual Lineage:

Editing Lineage: In the lineage view, click the three dots next to an asset, then select "Edit Upstream" or "Edit Downstream"
Preservation During Ingestion: Manual changes are preserved if the source has incremental_lineage: true configured, otherwise they get overwritten
Best Practices:
- Correct unexpected system-generated lineage at the source rather than manually overriding
- Use manual lineage for supplementing automated discovery, not correcting ingestion results
- Assign ownership of manually curated lineage to responsible data producers

Additional Notes

Lineage breaks are commonly caused by connector version updates or third-party library changes. The UI doesn't currently expose the specific SQL statement or evidence for each lineage edge directly on the graph - this information requires API queries or examining intermediate query assets. For business users, explain that each connection represents relationships discovered from data platforms or added manually based on metadata analysis.