Snowflake Tag Extraction with Lineage Configuration

When ingesting from Snowflake with tags enabled, setting `extract_tags: with_lineage` does not extract tags from tables, but changing to `extract_tags: without_lineage` successfully extracts tags. This is contrary to the documentation which states that `with_lineage` should extract both object tags and tag lineage.

You Might Be Asking:

• Why aren't Snowflake tags appearing with extract_tags: with_lineage?
• What's the difference between with_lineage and without_lineage?
• What privileges are required for tag extraction with lineage?
• How long does it take for tags to appear after configuration changes?

Tag extraction with lineage requires specific Snowflake privileges and has a processing delay due to Snowflake's internal view updates.

Required Privileges:

Grant the DataHub role access to Snowflake's ACCOUNT_USAGE schema:

-- Grant imported privileges on SNOWFLAKE database
GRANT IMPORTED PRIVILEGES ON DATABASE SNOWFLAKE TO ROLE DATAHUB_ROLE;

-- Verify the grant
SHOW GRANTS TO ROLE DATAHUB_ROLE;

Understanding the Latency:

When using extract_tags: with_lineage: - DataHub queries Snowflake's ACCOUNT_USAGE.TAG_REFERENCES view - This view has a latency of 45 minutes to 3 hours for data updates - Newly created or modified tags may not appear immediately - This is a Snowflake limitation, not a DataHub issue

Configuration Examples:

Option 1: With Lineage (Requires privileges + has latency):

source:
  type: snowflake
  config:
    account_id: ${SNOWFLAKE_ACCOUNT}
    username: ${SNOWFLAKE_USER}
    password: ${SNOWFLAKE_PASSWORD}
    extract_tags: with_lineage
    # Requires IMPORTED PRIVILEGES on SNOWFLAKE database

Option 2: Without Lineage (Faster, no special privileges):

source:
  type: snowflake
  config:
    account_id: ${SNOWFLAKE_ACCOUNT}
    username: ${SNOWFLAKE_USER}
    password: ${SNOWFLAKE_PASSWORD}
    extract_tags: without_lineage
    # Uses INFORMATION_SCHEMA, faster updates

Troubleshooting Steps:

1. Verify privileges are granted
2. Wait 45 minutes to 3 hours after tagging objects
3. Re-run ingestion
4. Check DataHub logs for privilege errors

When to Use Each Option:

• with_lineage: When you need tag lineage relationships and can accept the latency
• without_lineage: When you need faster tag updates and don't require lineage

• The latency is inherent to Snowflake's ACCOUNT_USAGE views and cannot be reduced
• Tags will appear eventually; patience is required after configuration changes
• If tags still don't appear after 3 hours, verify privileges and check ingestion logs
• Both options extract tags from tables, columns, and schemas

• Snowflake Source Configuration
• Snowflake ACCOUNT_USAGE Latency

• 4953