Stored Procedure Lineage in SQL-Based Connectors: Required Configuration Settings – DataHub Customer Support Portal

Area: Ingestion Issues

Sub-Area: SQL Connector Lineage Configuration

Issue

When a stored procedure copies data between databases on the same host (for example, from one or more source databases into a data warehouse database), lineage does not appear in DataHub after ingestion. This occurs because DataHub must discover and parse the stored procedure's SQL code to understand data flow. Simply ingesting the destination database alone is insufficient — if the ingestion recipe targets only one database or has stored procedure parsing disabled, DataHub cannot resolve source-side table URNs and lineage edges are silently dropped.

Error Messages

No lineage edges generated for stored procedure
URN resolution failure for cross-database table references

You Might Be Asking

Why doesn't lineage appear for tables populated by a stored procedure even though ingestion completes successfully?
How do I configure DataHub to capture cross-database lineage from stored procedures in SQL Server, PostgreSQL, MySQL, or Oracle?
Why are upstream tables missing from lineage when my stored procedure references tables in a different database on the same host?
What permissions does the DataHub service account need to read stored procedure definitions?

Solution

The following configuration changes are required to capture stored procedure lineage across databases on a single host. The same configuration keys apply to all supported SQL-based connectors (MSSQL/SQL Server, PostgreSQL, MySQL, Oracle, etc.).

Enable stored procedure ingestion.
Set include_stored_procedures: true. This enables DataHub to discover stored procedures and parse their SQL to extract lineage. If this is set to false, no lineage will be generated from stored procedures regardless of other settings.
Remove the single-database scope restriction.
When stored procedures reference tables across multiple databases, the ingestion run must be able to discover tables from all involved databases — not just the destination warehouse. Set database to null (or omit it) to allow all databases on the host to be scanned. If only the destination database is specified, source-side table URNs will be unknown to DataHub's schema resolver and lineage edges will be dropped.
Ensure URN casing is consistent.
Set convert_urns_to_lowercase: true and convert_column_urns_to_lowercase: true. Without consistent casing, table URNs from different databases may not match correctly, causing lineage edges to be silently dropped even when all other settings are correct.
Enable lineage parsing (if not already set).
Ensure include_lineage: true is set. This enables SQL parsing of stored procedure code to extract upstream/downstream relationships.
Pair usage statistics with query lineage.
If include_usage_statistics: true is set, also set include_query_lineage: true. Without this pairing the recipe will fail validation.

The following is a corrected example recipe for MSSQL/SQL Server. Adjust the type and connection fields for your specific connector:

source:
  type: mssql
  config:
    host_port: "<your-host>:1433"
    # Remove or set to null so all databases on the host are scanned.
    # This is required for cross-database stored procedure lineage.
    database: null
    username: "<your-service-account>"
    password: "<your-password>"
    include_views: true
    include_tables: true
    # Must be true to enable stored procedure discovery and SQL parsing
    include_stored_procedures: true
    # Enables SQL parsing to extract lineage from stored procedure code
    include_lineage: true
    include_view_lineage: true
    include_usage_statistics: true
    # Required alongside include_usage_statistics to pass recipe validation
    include_query_lineage: true
    usage:
      enabled: true
      lookback_days: 7
    # Critical for cross-database lineage — ensures URNs from different
    # databases match correctly during schema resolution
    convert_urns_to_lowercase: true
    convert_column_urns_to_lowercase: true
    profiling:
      enabled: true
      profile_table_level_only: true
    stateful_ingestion:
      enabled: true

sink:
  type: datahub-rest
  config:
    server: "https://<your-instance>.acryl.io/gms"
    token: "<your-datahub-token>"

After applying this configuration, re-run the ingestion recipe. DataHub will represent each stored procedure as a Data Job with the source database tables as inputs and the destination warehouse tables as outputs, making the full lineage visible in the UI.

Required Database Permissions

The DataHub service account must have sufficient permissions to read stored procedure definitions. For SQL Server, ensure the account has at minimum:

VIEW DEFINITION permission on the databases containing stored procedures
SELECT permission on system catalog views (sys.sql_modules, sys.objects, sys.procedures)
Read access to all source and destination databases referenced in the stored procedures

Additional Notes

This configuration pattern applies to all SQL-based connectors that support stored procedure ingestion, including MSSQL, PostgreSQL, MySQL, and Oracle. Verify the exact source type value for your connector in the DataHub documentation.
Setting database: null causes DataHub to scan all databases accessible to the service account on the host. Ensure the service account's permissions are scoped appropriately to avoid ingesting unintended databases, or use database_pattern allow/deny lists to control scope.
The convert_urns_to_lowercase: true setting is not the default for all connectors. Omitting it is one of the most common causes of silently dropped lineage edges in cross-database scenarios.
Community work on improving MSSQL stored procedure lineage SQL parsing is ongoing in the DataHub open-source project. If you encounter edge cases with complex stored procedure SQL, check the latest connector release notes.
If you prefer not to disable the single-database scope, you can alternatively emit lineage manually using the DataHub Python SDK, modeling the stored procedure as a Data Job and explicitly linking source and destination datasets.