Ingestion Pipeline Failures and Errors

Ingestion pipelines fail during execution with various error messages. This commonly occurs due to connectivity issues, authentication failures, permission problems, or resource constraints during data extraction from source systems.

Common Error Messages:

• ModuleNotFoundError: No module named 'datahub.metadata._schema_classes'
• Version mismatch between executor and server
• 401 Unauthorized error
• Ingestion jobs stuck in Pending state

Note: For DataHub Cloud customers, infrastructure-level changes (Kubernetes, AWS resources, operating system configurations) are managed by DataHub Support. The solutions below focus on configuration changes that customers can implement.

1. Check ingestion source connectivity and credentials:

   • Verify that the source system is accessible from the DataHub ingestion executor
   • Confirm authentication credentials are valid and have not expired
   • Check firewall rules and network connectivity

2. Review ingestion recipe configuration:

   • Validate all required configuration parameters are present
   • Check for typos in connection strings, database names, or table patterns
   • Ensure the ingestion source version in the recipe matches your installed CLI version

3. Examine executor logs for detailed error messages:

   • For UI-based ingestion: Check the ingestion run details in the DataHub UI
   • For CLI-based ingestion: Review the console output or log files
   • Look for specific error codes or exception stack traces

4. Verify source system permissions:

   • Ensure the service account has necessary read permissions
   • Check for required grants on metadata views (e.g., INFORMATION_SCHEMA access)
   • Confirm the account can access all schemas/databases specified in the recipe

5. For cloud customers experiencing consistent failures:

   • Contact DataHub Support to review executor health and resource allocation
   • Support can provision additional resources if needed

Always backup your DataHub configuration before making changes. Test solutions in a non-production environment when possible.

• Ingestion Sources

• 5937, 5936, 5908, 5886, 5752, 5715, 5670, 5669, 5583, 5643