Pattern-Based Domain Assignment Not Retaining

When using the `pattern_add_dataset_domain` transformer in ingestion recipes to automatically assign domains based on dataset name patterns, the domain assignments do not persist or are not retained after ingestion completes.

Note: For DataHub Cloud customers, infrastructure-level changes are managed by DataHub Support.

1. Configure domain assignment transformer:

transformers:
  - type: "pattern_add_dataset_domain"
    config:
      domain_pattern:
        rules:
          ".*\.finance\..*": "urn:li:domain:finance"
          ".*\.marketing\..*": "urn:li:domain:marketing"
          "prod_db\..*": "urn:li:domain:production"

2. Ensure domain entities exist:

   • Domains must be created in DataHub before assignment
   • Create domains via UI (Settings → Domains) or API
   • Verify domain URNs match exactly what's in your transformer config

3. Check transformer placement:

   • Place pattern_add_dataset_domain after other transformers
   • Ensure it's in the correct section of your recipe (under transformers:)
   • Test with a small dataset first to verify patterns match

4. Verify domain persistence:

   • Check if domains appear immediately after ingestion in UI
   • Query via GraphQL API to confirm domain association was persisted
   • Look for transformer execution messages in ingestion logs

5. Troubleshoot pattern matching:

   • Use Python regex tester to validate your patterns
   • Add debug logging to see which patterns are matching:

transformers:
  - type: "pattern_add_dataset_domain"
    config:
      domain_pattern:
        rules:
          ".*\.finance\..*": "urn:li:domain:finance"
      # Enable debug logging
      replace_existing: false  # Keep existing domains

Test configurations in a development environment before applying to production. Always backup configurations before making changes.

• DataHub Documentation

• 5638, 5703, 5928, 5570