Power BI Column-Level Lineage Missing Due to Column Name Case Mismatch – DataHub Customer Support Portal

Area: Ingestion Issues

Sub-Area: Power BI Column-Level Lineage

Issue

After configuring the Power BI ingestion connector with all required flags for column-level lineage (CLL), table-level lineage renders correctly in the DataHub UI but no column-level lineage arrows appear. The underlying lineage data is present in the backend, but the UI cannot draw the edges because the column field paths stored in fineGrainedLineages are emitted in all-lowercase (e.g., orgid, settingtypeid), while the corresponding schemaMetadata field paths preserve the original casing from the Power BI REST API (e.g., OrgID, SettingTypeID). DataHub matches these case-sensitively when drawing lineage edges, so a mismatch of even one character means no column arrow is rendered. This affects any Power BI dataset whose columns contain uppercase characters and whose M-Query uses native SQL against a supported upstream source (SQL Server, Snowflake, Amazon Redshift, ODBC).

Error Messages

No column-level lineage displayed in UI despite ingestion completing successfully with no errors or warnings.

You Might Be Asking

Why does table-level lineage work but column-level lineage arrows are missing for my Power BI datasets?
I have extract_column_level_lineage: true set — why are no column arrows showing in the lineage view?
Will setting convert_urns_to_lowercase: true in my Power BI or upstream source recipe fix the missing column lineage?
Which CLI version contains the fix for the Power BI column name casing mismatch?
Why does column-level lineage work on some Power BI datasets but not others?

Solution

Step 1: Confirm All Required Recipe Flags Are Enabled

Before investigating the casing issue, verify that all prerequisite flags for Power BI column-level lineage are set to true in your ingestion recipe. If any one of these flags is absent or false, CLL is silently skipped.

source:
  type: powerbi
  config:
    extract_lineage: true
    extract_dataset_schema: true
    native_query_parsing: true
    enable_advance_lineage_sql_construct: true
    extract_column_level_lineage: true

Additionally, Power BI column-level lineage requires the following Power BI tenant settings to be enabled in the Power BI Admin portal:

Allow service principals to use read-only Admin APIs
Enhance Admin APIs responses with detailed metadata
Enhance Admin APIs responses with DAX and mashup expressions

Step 2: Identify Whether the Casing Mismatch Is the Root Cause

If all recipe flags are correctly set and table-level lineage is present but column arrows are absent, the root cause is likely the column name casing mismatch described in this article. You can confirm by inspecting the stored lineage via GraphQL. Query the affected dataset URN and compare the fineGrainedLineages downstream field paths against the schemaMetadata field paths. If the former are all lowercase and the latter preserve mixed case, the mismatch is confirmed.

Step 3: Upgrade to the Fixed CLI Version

This issue is caused by a bug in the Power BI connector where column names are unconditionally lowercased inside pattern_handler.py before the convert_urns_to_lowercase config flag is consulted. The fix was released in acryl-datahub version 1.6.0.16 (and corresponding cloud platform version v2.1.0-cloud). Upgrading to this version or later resolves the mismatch at ingestion time without requiring recipe changes.

To upgrade the CLI manually:

pip install acryl-datahub==1.6.0.16

If you are using the DataHub Cloud embedded executor, update the CLI version pinned in the Advanced section of your ingestion recipe configuration to 1.6.0.16 or later, then re-run the pipeline. Once updated to the fixed version, you can remove any previously pinned CLI version from your recipes and rely on the platform default.

Step 4: Re-Run the Power BI Ingestion Pipeline

After upgrading, trigger a full re-run of your Power BI ingestion pipeline. The fix normalizes column name casing at ingestion time, so new lineage records will have matching field paths on both the lineage edge and schema metadata sides.

Step 5: Validate Column-Level Lineage in the UI

After ingestion completes, navigate to an affected dataset in DataHub, open the Lineage tab, and enable the column view. Click an individual column to verify that lineage arrows now link through to the corresponding upstream source columns. Column-level lineage lives on datasets, not on reports — a report asset has no columns of its own, so column arrows are visible when you open the dataset that feeds the report.

Important Caveat: SQL Parser Coverage Gaps

Even after the casing fix is applied, column-level lineage may still be absent for some Power BI datasets. This is a separate, unrelated issue: the SQL parser cannot yet trace individual columns through M-Query native SQL blocks that use patterns such as variable declarations (DECLARE @variable) or leading comments. For these datasets, table-level lineage is extracted correctly (because table names are parsed successfully), but no column arrows are produced. Datasets whose queries are written as a plain SELECT or CTE pattern do receive full column-level lineage. Restructuring queries to avoid multi-statement or declared-variable patterns will enable CLL for those assets.

Note on `convert_urns_to_lowercase`

Setting convert_urns_to_lowercase: true in either the Power BI or upstream source recipe does not fix this issue. The unconditional lowercasing in the connector occurs before that flag is evaluated, so both branches of the flag-gated logic produce the same lowercase string. The only effective resolution is upgrading to the fixed CLI version.

Additional Notes

Affected versions: The column name casing mismatch was introduced in the Power BI connector around CLI version 1.5.0.8 (released April 2026). A prior release (approximately 1.5.0.3, March 2026) had temporarily addressed casing behavior. The definitive fix is included in acryl-datahub 1.6.0.16 / cloud platform v2.1.0-cloud.

Scope: The bug in pattern_handler.py affects all supported upstream platform handlers in the Power BI M-Query module — including SQL Server/MSSQL, Snowflake, Amazon Redshift, and ODBC sources — not just a single upstream type.

Supported upstream types for native SQL column-level lineage: Power BI column-level lineage via native SQL parsing is supported for Snowflake, Amazon Redshift, SQL Server (MSSQL), and ODBC sources. Pure M-Query transformations (without native SQL passthrough) are not supported for CLL regardless of CLI version.

Databricks column-level lineage is governed by a separate code path and is not affected by this casing bug. For Databricks Unity Catalog, ensure include_column_lineage: true is set and verify that no tables exceed the column_lineage_column_limit (default: 300 columns), which causes CLL to be silently skipped for wide tables.

Broader fix: Engineering has implemented a foundational URN casing normalization solution that addresses this class of mismatch across multiple ingestion sources, not just Power BI, to prevent recurrence.