Area: Product Issues
Sub-Area: Performance Degradation / Background Processing
Issue
DataHub Cloud production instances can experience severe, intermittent slowness — including unresponsive home pages, failed search results, and complete UI unavailability — caused by inefficient background group and permission processing. This condition can affect all users on an instance simultaneously and may worsen during peak usage hours. The issue has been observed independently across multiple production environments and is not caused by ingestion load alone. In some cases the slowness predates or is unrelated to a concurrent version upgrade, making root-cause identification more complex. The underlying defect is resolved in a specific hotfix release.
Error Messages
UI not responding / page fails to loadSearch returns no results or times outHigh API latency observed in backend dashboards (>5 seconds)
You Might Be Asking
- Why is my DataHub production instance extremely slow or completely unresponsive?
- Could a recent version upgrade have caused the slowness?
- Why does the slowness come and go rather than being constant?
- Could the number or structure of groups ingested via OIDC be contributing to the slowness?
- Does upgrading to a hotfix version fix production performance issues related to group processing?
Solution
-
Confirm the symptoms and scope.
Verify that the slowness is affecting all users on the production instance (not just a single user or browser session). Typical indicators include:
- Home page and catalog pages taking many seconds to render or failing to load entirely.
- Search (including "blind search" / global search) returning no results or timing out.
- API latency consistently above 5 seconds as observed in backend monitoring dashboards.
- Symptoms that spike during peak business hours and partially recover during off-peak hours.
-
Identify the current DataHub version.
Check the version of your DataHub Cloud instance. The group-processing inefficiency described in this article has been confirmed in version
v1.1.3. If you are running this version, proceed to the hotfix upgrade step below.# If you have API access, you can query the version endpoint: curl -X GET "https://.acryl.io/openapi/v3/healthcheck" \ -H "Authorization: Bearer " -
Understand the root cause.
The slowness is caused by an inefficiency in how DataHub processes groups and associated permission policies in the background. Under certain conditions — particularly when a non-trivial number of groups are present — background group resolution jobs can consume disproportionate CPU and I/O resources, starving foreground user requests of capacity. This manifests as intermittent spikes in latency that correlate with periods of higher user activity. The issue is distinct from, though it may overlap with, cluster-level resource contention events.
-
Apply the hotfix upgrade to
v1.1.4.For DataHub Cloud (managed) customers, contact DataHub Support to request an upgrade to
v1.1.4, which contains the fix for the group-processing inefficiency. The upgrade is a minor version bump (v1.1.3→v1.1.4) and does not require downtime when zero-downtime upgrades are enabled.Reference the release notes before approving the upgrade:
https://docs.datahub.com/docs/managed-datahub/release-notes/v_1_1_0#v114For self-hosted deployments, update your Helm values or Docker Compose configuration to pin to
v1.1.4:# Example Helm values update datahub: image: tag: "v1.1.4"# Apply the updated Helm release helm upgrade datahub datahub/datahub \ --namespace\ --values values.yaml -
Validate performance after the upgrade.
After the upgrade completes, verify that:
- The home page and catalog pages load within an acceptable time frame (under 5 seconds).
- Global search returns results promptly.
- Backend API latency metrics return to normal baseline values.
- Performance remains stable during peak usage periods, not just immediately after the upgrade.
-
Review group ingestion configuration (optional hardening).
As a supplementary hardening step, audit the groups being ingested into DataHub — for example via OIDC — and ensure that only groups actively used for permission administration are included. Reducing unnecessary group ingestion lowers the background processing burden. Example OIDC source configuration to filter groups:
source: type: okta # or your OIDC provider config: ingest_groups: true # Use a filter to include only groups relevant to DataHub permissions group_filter_pattern: allow: - "^DataHub-.*" # Replace with your actual group naming convention
Additional Notes
This issue was confirmed in DataHub Cloud instances running v1.1.3 and is resolved in v1.1.4. The slowness can appear to correlate with a concurrent version upgrade event but may have an independent onset — root-cause analysis should not assume the upgrade is the trigger without backend metric review. Instances hosted in shared regional clusters may also be subject to cluster-level resource contention as a separate contributing factor; if slowness persists after upgrading to v1.1.4, contact DataHub Support to investigate infrastructure-level causes. For DataHub Cloud customers, all version upgrades, cluster migrations, and infrastructure remediation are performed by the DataHub Cloud Engineering team — no customer action on infrastructure is required. If you operate a self-hosted DataHub deployment and observe similar symptoms, upgrade to at least v1.1.4 and review your group ingestion volume as described above.
Related Documentation
- DataHub Cloud Managed Service Overview
- DataHub Managed Release Notes — v1.1.4
- DataHub Authorization and Policies
- Configuring OIDC SSO and Group Ingestion
- DataHub Performance Optimization
Tags: slowness, performance, group-processing, hotfix, v1.1.3, v1.1.4, OIDC, permissions, production, upgrade