Missing Tables After Environment Migration

After migrating DataHub between environments or tiers, users discovered that some tables and datasets were missing from the new environment, even though they were present in the source environment. This typically occurred during tier-to-tier migrations or when restoring from backups.

You Might Be Asking:

• Why are some of my tables missing after migration?
• How do I verify data completeness after migration?
• What's the proper way to migrate DataHub between environments?

Ensure proper migration procedures are followed to maintain data completeness:

1. Verify Source Data: Confirm all entities exist in source before migration
2. Export Metadata Completely: Use the full export functionality
3. Check Ingestion State: Ensure all ingestion sources have completed before export
4. Import with Validation: Monitor import process for errors

# Export from source environment
datahub put --urn "urn:li:dataset:*" -d source_backup.json

# Import to target environment
datahub put -f source_backup.json

# Verify entity counts
datahub get --urn "urn:li:dataset:*" --aspect list

Fixed in version v0.3.12. When migrating between environments, ensure that both environments are running compatible DataHub versions, and re-run ingestion sources after migration to refresh metadata.

• Backup & Restore
• DataHub CLI