Redshift Ingestion Connection Errors

Users encountered unexpected connection errors when attempting to ingest metadata from Redshift databases into DataHub. These errors were not standard Redshift connection failures but rather issues with how the connector handled authentication, SSL requirements, or connection pooling.

Common Error Messages:

• Connection to Redshift failed with unexpected error
• SSL/TLS handshake failure
• Authentication failed despite correct credentials

The Redshift connector has been fixed to properly handle various connection scenarios. Ensure your ingestion configuration includes the necessary connection parameters:

Example for Redshift. This can be adapted for other data sources/connectors.

source:
  type: redshift
  config:
    host_port: "cluster.region.redshift.amazonaws.com:5439"
    database: "your_database"
    username: "${REDSHIFT_USERNAME}"
    password: "${REDSHIFT_PASSWORD}"

    # SSL configuration
    include_tables: true
    include_views: true

    # Optional: Connection options
    options:
      sslmode: require
      connect_timeout: 60

Fixed in version v0.3.11. Verify that your Redshift security groups allow connections from your DataHub instance. For AWS environments, ensure proper IAM roles are configured if using IAM authentication.

• Redshift Ingestion Source
• Troubleshooting Ingestion