Resolving DBT Cloud and Snowflake Ingestion Authentication Failures – DataHub Customer Support Portal

Area: Ingestion Issues

Sub-Area: Authentication and Token Management

Issue

Ingestion recipes fail with authentication errors when connecting to external APIs like DBT Cloud or Snowflake. These failures typically manifest as 401 Unauthorized errors for DBT Cloud connections or JWT token expiration errors for Snowflake connections using key-pair authentication. The errors often occur after recipes have been working successfully, indicating token expiration, rotation, or permission changes.

Error Messages

ValueError: Failed to fetch environments from dbt Cloud API. Status code: 401: 401 Client Error: Unauthorized for url: https://cloud.getdbt.com/api/v2/accounts/{account_id}/environments/
sqlalchemy.exc.DatabaseError: Failed to connect to DB: {instance}.snowflakecomputing.com:443. JWT token is invalid.
HTTPSConnectionPool: Max retries exceeded with url: /gms/aspects?action=ingestProposalBatch

You Might Be Asking

Why did my ingestion recipe stop working when it was fine before?
How do I fix 401 authentication errors in DBT Cloud ingestion?
What causes JWT token expiration in Snowflake ingestion?
Why are my ingestion errors not showing in the DataHub UI?

Solution

For DBT Cloud Authentication Failures

Log into your DBT Cloud account and navigate to Account Settings → API Tokens
Check if your existing Service Account token is still active and has the correct permissions
Generate a new Service Account token with both "Metadata API" and "REST API" permissions (or "Account Admin"/"Read All" permissions)
In DataHub, navigate to your DBT ingestion recipe configuration

Update the token field with the new token value:

dbt_cloud_url: "https://cloud.getdbt.com/"
dbt_cloud_auth_token: "${YOUR_DBT_TOKEN_SECRET}"
account_id: 
project_id:

Verify that the secret referenced (e.g., ${YOUR_DBT_TOKEN_SECRET}) exists in your DataHub secrets management
Save the recipe and trigger a new ingestion run

For Snowflake JWT Token Expiration

Verify your Snowflake key-pair authentication configuration:

warehouse: 
username: '${SNOWFLAKE_USER}'
authentication_type: KEY_PAIR_AUTHENTICATOR
private_key: '${SNOWFLAKE_PRIVATE_KEY}'
private_key_password: '${SNOWFLAKE_PASSPHRASE}'
account_id: ''

Check if your Snowflake account has token expiration policies that are shorter than your ingestion runtime
Contact your Snowflake administrator to review JWT token expiration settings
Consider breaking large ingestion jobs into smaller, more frequent runs to stay within token validity windows
Ensure your private key and passphrase secrets are correctly configured and accessible

For UI Error Display Issues

If using DataHub Cloud and ingestion errors are not displaying in the UI, this may be due to a frontend parsing bug in older versions
Check your DataHub Cloud version and request an upgrade if running version 0.3.15 or earlier
As a temporary workaround, review ingestion logs directly through the execution details or log files
Contact DataHub Support if the UI continues to not display error information after upgrading

Additional Notes

Authentication failures often occur due to external changes rather than DataHub issues. DBT Cloud tokens can be rotated or revoked by team members, and Snowflake accounts may have policy changes affecting token expiration. For long-running ingestion jobs, be aware that JWT tokens typically expire within 1 hour and cannot be refreshed mid-execution. DataHub Cloud versions prior to 0.3.16.7 had known issues with error display in the UI that are resolved in later versions.