Looker Ingestion Failing with HTTP 400 Error on Container Ownership Workunits – DataHub Customer Support Portal

```html

Area: Ingestion Issues

Sub-Area: Transformer / Ownership Patch Regression

Issue

When running Looker ingestion with the pattern_add_dataset_ownership transformer configured with is_container: true, all container ownership workunits fail with an HTTP 400 error reported as "Cannot parse request entity." The extraction phase completes successfully and the vast majority of records are written, but every workunit of the form txform-<container_urn>-ownership is rejected by the DataHub GMS. This is a regression introduced in CLI versions 1.5.0.6 and later (up to and including 1.5.0.17) and is caused by two compounding defects: (1) the ownership patch builder began emitting a wrapped GenericJsonPatch envelope keyed on a four-field compound key (owner, type, typeUrn, attribution.source) instead of the two-field key (owner, type) that GMS expects, producing paths with a trailing // for plain TECHNICAL_OWNER entries; and (2) when container ownership is enabled, the same owner is duplicated once per child asset, causing the ownership patch payload for large Looker folders to exceed the server's 15 MB request-size limit (~18 MB observed), which also produces the same "Cannot parse request entity" 400 response. CLI versions ≤ 1.5.0.4 are not affected by the malformed-patch defect. The complete fix — de-duplicating owners so the payload stays within the size limit and correcting the patch format — is available in CLI 1.6.0.16 (GA, released 2026-07-27).

Error Messages

Unable to emit metadata to DataHub GMS: Cannot parse request entity
com.linkedin.restli.server.RestLiServiceException: Cannot parse request entity
status: 400, workunit_id: txform-<container_urn>-ownership
MCP object ...-ownership has size 19210888 that exceeds the max payload size of 15728640

You Might Be Asking

Why does my Looker ingestion succeed for datasets and dashboards but fail only for container ownership workunits?
Why am I seeing HTTP 400 errors at the sink stage even though extraction completed without errors?
Is this a network or TCP keepalive problem causing the 400 responses?
Does setting is_container: true on other sources like BigQuery cause the same failure?
What CLI version contains the fix for the container ownership patch regression?

Solution

Option 1 — Upgrade to the Fixed CLI Version (Recommended)

Upgrade the DataHub CLI to 1.6.0.16 or later. This version contains the de-duplication fix (merged via PR datahub-project/datahub#18392) that prevents oversized ownership patches and corrects the compound-key patch format.
```
pip install "acryl-datahub==1.6.0.16"
```
If you are using a Remote Executor deployed via Helm, update the executor image to the version that bundles CLI 1.6.0.16 (executor image tag v2.1.1-cloud or later) and rebuild your custom image if needed:
```
# Example: rebuild a custom executor image pinned to the fixed CLI version
FROM docker.datahub.com/re/datahub-executor:v2.1.1-cloud-slim-unbundled
# The base image already bundles acryl-datahub==1.6.0.16
```
Re-run the Looker ingestion and confirm that no txform-*-ownership workunits appear in the failure list.

Option 2 — Immediate Workaround: Disable Container Ownership (No Upgrade Required)

If upgrading the CLI immediately is not feasible, set is_container: false on the pattern_add_dataset_ownership transformer in your Looker recipe. This prevents the transformer from emitting ownership patches for containers while leaving ownership assignment for datasets, dashboards, and charts intact. The pattern_add_dataset_domain transformer is not affected — it writes a full aspect rather than a patch and does not exhibit this bug, so is_container: true can remain on that transformer.

# Looker ingestion recipe excerpt
source:
  type: looker
  config:
    # ... your existing Looker connection config ...

transformers:
  - type: "pattern_add_dataset_ownership"
    config:
      is_container: false          # Set to false to avoid the ownership patch regression
      owner_pattern:
        rules:
          ".*": ["urn:li:corpUser:<your-owner-urn>"]
      ownership_type: "TECHNICAL_OWNER"

  - type: "pattern_add_dataset_domain"
    config:
      is_container: true           # Safe to keep true; domain transformer is not affected
      domain_pattern:
        rules:
          ".*": ["urn:li:domain:<your-domain-urn>"]

Option 3 — Pin CLI to ≤ 1.5.0.4 (Short-Term Only)

Downgrading to CLI 1.5.0.4 or earlier avoids the malformed patch format. However, this version predates several memory and SQL parsing fixes available in later releases and is not recommended for long-term use. Use this only as a short-term bridge while arranging an upgrade to 1.6.0.16.

pip install "acryl-datahub==1.5.0.4"

Important: If you are running a Remote Executor, changing the CLI version in the recipe UI alone does not change the CLI version used by the executor. You must rebuild the executor image with the desired acryl-datahub version pinned at image build time.

Reproducing the Malformed Patch Locally (for Diagnosis)

You can confirm whether your installed CLI version emits the malformed patch without running a full ingestion. The presence of a trailing // in the patch path indicates the affected version:

python -c "
import json
from datahub.metadata.schema_classes import OwnerClass, OwnershipTypeClass
from datahub.specific.dashboard import DashboardPatchBuilder

o = OwnerClass(owner='urn:li:corpGroup:test', type=OwnershipTypeClass.TECHNICAL_OWNER)
b = DashboardPatchBuilder('urn:li:container:abc')
b.add_owner(o)
print(json.dumps(json.loads(list(b.build())[0].aspect.value), indent=2))
"

A healthy output uses a two-segment path such as /owners/urn:li:corpGroup:test/TECHNICAL_OWNER. An affected output contains a trailing //, for example: /owners/urn:li:corpGroup:test/TECHNICAL_OWNER//, and wraps the payload in an arrayPrimaryKeys envelope that GMS cannot deserialize.

Additional Notes

Affected CLI version range: 1.5.0.6 through 1.5.0.17 (and likely intermediate releases in this range). CLI ≤ 1.5.0.4 does not produce the malformed patch but lacks other improvements. Fixed in: CLI 1.6.0.16 (GA, 2026-07-27), included in Remote Executor image v2.1.1-cloud.

This is not a TCP/network issue. The 400 "Cannot parse request entity" response originates from GMS deserializing the request body and is unrelated to idle TCP connections, SSL-EOF errors, or keepalive settings. A network-layer failure would not return a structured HTTP 4xx response.

Other ingestion sources: Any source using pattern_add_dataset_ownership with is_container: true and an affected CLI version is theoretically exposed to the same defect. Sources with smaller folder hierarchies or stateful ingestion that results in an empty entity map may not produce failures in practice, but the underlying patch format issue exists. Applying the workaround (is_container: false) to other affected recipes is advisable until the upgrade to 1.6.0.16 is complete.

The pattern_add_dataset_domain transformer is not affected by this regression. It writes a full aspect rather than using the patch builder, so is_container: true on that transformer is safe regardless of CLI version.

Remote Executor image builds: The -slim-unbundled image variant may lag behind the standard image tag at release time. If your build pipeline depends on that variant, confirm image availability on docker.datahub.com before rebuilding. You can verify which CLI version is bundled in the executor image at runtime with:

python -c 'import datahub; print(datahub.__version__)'