Skip to main content

**NEW/IMPORTANT** FIXED (ACTION MAY BE REQUIRED): Ingestions Failing as of December 31, 2025 due to ruamel library update

Angelo Bresci
Updated

 

What Happened

The acryl-datahub python package has an unpinned dependency on the open source ruamel-yaml project (meaning our package isn't tied to one known/tested version of ruamel, and is subject to automatic upgrades of that library). 

On December 31st, 2025, the ruamel-yaml library released version 0.19.0 with breaking changes that require C compilation tools during installation. This new version is incompatible with DataHub's execution environment and caused ingestion failures under various circumstances. DataHub has fixed all accessible ingestion connectors. Action will be required if you are utilizing connectors that DataHub does not have access to change.

Who Is Affected

Most ingestion connectors were affected (See the Appendix below for the list of affected connectors). 

(FIXED) For all ingestion connectors running via Managed Ingestion in a hosted DataHub Cloud instance, DataHub has already applied a fix. Subsequent ingestion runs will succeed.

For any ingestion running via CLI (command-line interface) on a customer machine, or any ingestion running via Managed Ingestion in a DataHub instance deployed within a customer's network, customers will need to apply the fixes described below themselves.

For customers on a DataHub server version earlier than v0.3.15, all new ingestion recipes will fail unless they are using CLI version 1.3.1.5. -> You will need to upgrade to this new CLI version.

DataHub Cloud: Remediation for UI Ingestion Managed Recipes

(FIXED) Nothing needs to be done for Ingestion recipes created via Managed Ingestion before Jan 1st, 2025. All have already been updated to have the Extra Pip Libraries set to include `[“ruamel-yaml<0.19”]` under Advanced on the final page of ingestion configuration:

On January 2nd, 2026, we will apply a change to all Server versions on 0.3.15.X to set the default CLI version to 1.3.1.5. However, new recipes on DataHub server versions under 0.3.15 will not have this applied. When creating a new recipe via Managed Ingestion on server versions under 0.3.15, either set the extra pip libraries to the fix, or set the cli version to 1.3.1.5. DataHub will also be scanning for these recipes and automatically applying the remediation of the Extra Pip Library setting.

 

Remediation for Customer-Managed Ingestion Recipes (CLI fix)

We've released DataHub CLI version 1.3.1.5 with the fix. Upgrading will address the issue:

#!/bin/bash

python3 -m pip install acryl-datahub==1.3.1.5

If you're using connector plugins, reinstall them as well:

#!/bin/bash
pip install 'acryl-datahub[mysql]==1.3.1.5'

Replace mysql above with your specific connector(s) as needed.

Alternative: Temporary Workaround

If you cannot upgrade immediately, you can temporarily pin ruamel-yaml to a compatible version:

#!/bin/bash
python3 -m pip install --upgrade --force-reinstall acryl-datahub 'ruamel-yaml<0.19'

Verification

After upgrading or applying the workaround, verify your installation:

#!/bin/bash
python3 -m datahub version
datahub ingest -c <path/to/your-recipe.yml>

 

Remediation for Customer-Managed (on-premise) DataHub Instances

Set the helm value `global.datahub.managed_ingestion.defaultCliVersion` to `1.3.1.5`. For all existing recipes, there are two options:

  1. Upgrade them to `1.3.1.5`
  2. Set the Extra Pip Libraries to include `[“ruamel-yaml<0.19”]` under Advanced on the final page of ingestion configuration:

 

Timeline of DataHub Remediation for Managed Ingestion in DataHub Cloud Instances

On Jan 1 2026, at 5:28pm PST, DataHub pinned the version of ruamel-yaml for all customer recipes being run via Managed Ingestion. This was in order to fix the issue as soon as possible.  For all ingestion recipes run via managed ingestion, on their next run, they will succeed. Some recipes are already succeeding. All recipes that are not configured via DataHub UI in a DataHub Cloud instance will continue to fail.

Questions or Issues?

If you continue to experience problems after upgrading, please:

We apologize for the inconvenience and are implementing stricter dependency pinning across all our packages to prevent similar issues in the future.

 

Appendix - Affected Connectors

Ingestion connectors that were affected by the unpinned ruamel-yaml issue:

  1.  abs - Azure Blob Storage
  2.  airflow - Airflow integration
  3.  azure-ad - Azure Active Directory
  4.  bigquery-slim - BigQuery without SQL profiling
  5.  cassandra - Apache Cassandra
  6.  circuit-breaker - Circuit breaker pattern
  7.  datahub-business-glossary - Business glossary files
  8.  datahub-debug - Debug utilities
  9.  datahub-lineage-file - Lineage file ingestion
  10.  dataplex - Google Dataplex
  11.  dbt - dbt integration
  12.  dbt-cloud - dbt Cloud
  13.  delta-lake - Delta Lake
  14.  dynamodb - AWS DynamoDB
  15.  elasticsearch - Elasticsearch
  16.  excel - Excel files
  17.  feast - Feast feature store
  18.  fivetran - Fivetran integration
  19.  gcs - Google Cloud Storage
  20.  glue - AWS Glue
  21.  grafana - Grafana
  22.  great-expectations - Great Expectations plugin
  23.  iceberg - Apache Iceberg
  24.  iceberg-catalog - Iceberg catalog
  25.  json-schema - JSON Schema
  26.  kafka - Apache Kafka
  27.  ldap - LDAP
  28.  looker - Looker
  29.  lookml - LookML
  30.  metabase - Metabase
  31.  mlflow - MLflow
  32.  mode - Mode Analytics
  33.  mongodb - MongoDB
  34.  neo4j - Neo4j
  35.  nifi - Apache NiFi
  36.  okta - Okta
  37.  powerbi - Power BI
  38.  powerbi-report-server - Power BI Report Server
  39.  preset - Preset (Superset cloud)
  40.  pulsar - Apache Pulsar
  41.  qlik-sense - Qlik Sense
  42.  redash - Redash
  43.  s3 - AWS S3 (with profiling)
  44.  s3-slim - AWS S3 (without profiling)
  45.  sac - SAP Analytics Cloud
  46.  sagemaker - AWS SageMaker
  47.  salesforce - Salesforce
  48.  sigma - Sigma Computing
  49.  slack - Slack integration
  50.  snaplogic - SnapLogic
  51.  snowflake-slim - Snowflake without SQL profiling
  52.  sql-parser - SQL parser utility
  53.  sql-queries - SQL queries from logs
  54.  superset - Apache Superset
  55.  tableau - Tableau
  56.  vertexai - Google Vertex AI

Related to

Related articles