Snowflake View Definitions Missing from Ingestion Due to SHOW VIEWS Batching Bug – DataHub Customer Support Portal

Area: Ingestion Issues

Sub-Area: Snowflake Connector — View Metadata & Lineage

Issue

During Snowflake ingestion, view definitions (the viewProperties aspect) are silently missing for a subset of views, even though those views appear as dataset entities in DataHub. Because the view SQL definition is absent, downstream lineage extraction also fails for the affected views. This is caused by a batching bug in the Snowflake connector's fallback mechanism: when the ingestion role is not the owner of a view (or cannot retrieve the definition from INFORMATION_SCHEMA), the connector issues batched SHOW VIEWS LIKE queries to recover definitions. A pagination-marker defect in that batching logic caused some views to be silently skipped, resulting in partial — not total — ingestion failures.

Error Messages

viewProperties aspect missing for affected view datasets
'aspects': {'dataset': {'status': 1, 'schemaMetadata': 1, 'datasetProperties': 1, 'container': 1, 'dataPlatformInstance': 1, 'subTypes': 1, 'browsePathsV2': 1}} # viewProperties absent

You Might Be Asking

Why are only some of my Snowflake views missing definitions while others ingest correctly?
Is there a workaround to ingest view definitions for specific views without re-running the full ingestion?
Which version of acryl-datahub contains the fix for this batching bug?
Could this be a Snowflake permissions issue rather than a DataHub bug?

Solution

Step 1: Confirm the Root Cause

First, distinguish between a permissions issue and the batching bug by running the following query as the DataHub ingestion role in Snowflake:

SELECT TABLE_NAME, VIEW_DEFINITION
FROM "<your_database>".INFORMATION_SCHEMA.VIEWS
WHERE TABLE_SCHEMA = '<your_schema>'
  AND TABLE_NAME IN ('<affected_view_1>', '<affected_view_2>');

If VIEW_DEFINITION is NULL for the affected views → the ingestion role is not the owner. Address the permissions issue (see Step 2).
If VIEW_DEFINITION is populated but definitions are still missing in DataHub → the batching bug is the cause. Upgrade the CLI package (see Step 3).

Step 2: Address Snowflake Permissions (If Applicable)

Snowflake only populates VIEW_DEFINITION in INFORMATION_SCHEMA for roles that own the view. If the ingestion role does not own all views, grant the REFERENCES privilege or transfer ownership where appropriate:

-- Grant REFERENCES privilege on affected views
GRANT REFERENCES ON VIEW <your_database>.<your_schema>.<affected_view>
  TO ROLE <datahub_ingestion_role>;

-- For secure views, grant access to ACCOUNT_USAGE
GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE
  TO ROLE <datahub_ingestion_role>;

To check whether affected views are secure views, run:

SHOW VIEWS IN SCHEMA <your_database>.<your_schema>;
-- Check the 'is_secure' column for affected views

Step 3: Upgrade the acryl-datahub CLI Package (Primary Fix)

The batching bug in the SHOW VIEWS fallback logic was fixed in the DataHub Snowflake connector. Upgrade the acryl-datahub package to version 1.7.0 or later (which includes the patch):

# Upgrade to the fixed version
pip install "acryl-datahub>=1.7.0"

# Verify the installed version
datahub version

If you manage the CLI version via a Dockerfile (e.g., using a DATAHUB_CLI_VERSION build argument), update that variable accordingly:

# In your Dockerfile or build configuration
ARG DATAHUB_CLI_VERSION=1.7.0
RUN pip install acryl-datahub==${DATAHUB_CLI_VERSION}[snowflake]

Step 4: (Optional) Target Specific Views as a Workaround

If you need to recover definitions for specific views before upgrading, you can narrow the ingestion scope using schema_pattern and table_pattern filters in your recipe to re-run ingestion targeting only the affected views:

source:
  type: snowflake
  config:
    account_id: "<your_snowflake_account>"
    username: "<your_username>"
    password: "<your_password>"
    role: "<datahub_ingestion_role>"
    warehouse: "<your_warehouse>"
    database_pattern:
      allow:
        - "^<your_database>$"
    schema_pattern:
      allow:
        - "^<your_schema>$"
    table_pattern:
      allow:
        - "^<affected_view_1>$"
        - "^<affected_view_2>$"
    include_views: true
    include_view_lineage: true

Note: This workaround does not fix the underlying bug — it only forces a targeted re-ingestion. Upgrading the CLI package (Step 3) is the definitive resolution.

Step 5: Re-run Ingestion and Validate

After upgrading the CLI package, re-run the Snowflake ingestion pipeline.
Verify that the previously affected views now display their SQL definitions under the Schema tab in DataHub.
Confirm that upstream lineage is populated correctly for those views.

Additional Notes

Bug details: The defect was in the SHOW VIEWS LIKE batching fallback used when view definitions cannot be retrieved from INFORMATION_SCHEMA. A pagination marker bug caused the connector to use an unqualified view_name instead of the fully qualified schema.view_name as the batch cursor, resulting in views at schema pagination boundaries being silently skipped. This is why only a subset of views — not all — were affected.

Affected versions: All versions of acryl-datahub prior to 1.7.0 are affected by this batching bug. The fix was introduced in the 1.7.0 release cycle. Release candidates (e.g., 1.7.0rc1) can be used for early validation in lower environments.

Partial vs. total failures: Because the bug only affects views that fall at certain batch boundaries in the SHOW VIEWS pagination sequence, the symptom is always partial — some views ingest correctly while others silently lose their definitions. If all views are missing definitions, the cause is more likely a Snowflake permissions issue rather than this bug.

Snowflake 2026_01 BCR: Snowflake's 2026_01 Breaking Change Release altered the DDL property format used in view definitions. If your Snowflake account has adopted this BCR, ensure your DataHub CLI is also on a version that handles the new format (1.7.0+) to avoid lineage extraction failures from SQL parsing errors.

Large view counts: If your Snowflake database contains more than 10,000 views, consider enabling fetch_views_from_information_schema: true in your recipe to bypass SHOW VIEWS pagination entirely where permissions allow.