Missing Column Statistics from Snowflake Ingestion

Column-level statistics (such as null counts, distinct values, min/max values) are not appearing in DataHub after Snowflake ingestion, even though table-level metadata is being ingested successfully.

Note: For DataHub Cloud customers, infrastructure-level changes are managed by DataHub Support.

1. Enable profiling in Snowflake ingestion:

source:
  type: snowflake
  config:
    # ... connection config ...
    profiling:
      enabled: true
      profile_table_level_only: false  # Enable column profiling
      include_field_null_count: true
      include_field_distinct_count: true
      include_field_min_value: true
      include_field_max_value: true
      max_workers: 5
      # Limit profiling to specific tables for performance
      profile_pattern:
        allow: ["prod_db\.analytics\..*"]

2. Understand profiling performance impact:

   • Column profiling queries each table, which can be slow
   • Start with a subset of tables to test performance
   • Consider running profiling separately from metadata ingestion

3. Check Snowflake permissions:

   • Service account needs SELECT permission on profiled tables
   • Verify warehouse is available and properly sized
   • Check query history in Snowflake to confirm profiling queries ran

4. Verify column stats in UI:

   • Navigate to dataset → Schema tab
   • Column statistics appear in the column details section
   • May need to refresh or clear cache if recently ingested

Test configurations in a development environment before applying to production. Always backup configurations before making changes.

• DataHub Documentation

• 4963