Remote Executor Deployment Timeout Due to Stale values.yaml and Dockerize Dependency – DataHub Customer Support Portal

Area: Deployment Issues

Sub-Area: Remote Executor Helm Upgrade

Issue

When upgrading a remote executor deployment to a new DataHub Cloud release version, the upgrade process can time out — even after increasing the default timeout threshold significantly. While slow image pulls from the ECR registry are a common first suspect, the root cause may instead be a stale values.yaml configuration file that contains outdated or incompatible settings. Specifically, retaining a dockerize init container dependency in the Helm values from a previous release can cause the deployment to hang indefinitely and ultimately fail.

You Might Be Asking

Why does my remote executor Helm upgrade time out even after I increased the timeout limit?
Is the timeout caused by a slow ECR image pull, or is something else wrong?
What changed in the remote executor Helm chart that would cause my existing values.yaml to break?
How do I know if my values.yaml is stale after a DataHub Cloud version upgrade?

Error Messages

Error: timed out waiting for the condition
deadline exceeded

Solution

Identify whether the timeout is caused by a stale configuration rather than a slow image pull.
Before increasing timeout values further, check your deployment logs to determine where the hang occurs. If the container never reaches a running state even after the image is pulled successfully, the issue is likely configuration-related, not network-related.
Review your values.yaml for outdated settings.
Compare your current values.yaml against the latest Helm chart defaults for the target release. Look for deprecated or removed keys. A common culprit is a lingering dockerize init container configuration that is no longer supported or required in newer chart versions.

Example of a stale entry to remove:
```
# REMOVE if present in your values.yaml — no longer required in recent releases
initContainers:
  - name: wait-for-prerequisites
    image: jwilder/dockerize
    args:
      - -wait
      - tcp://<some-host>:<port>
      - -timeout
      - 60s
```
Remove the dockerize init container block from your values.yaml.
Delete any init container entries that reference dockerize. This dependency was used in older chart versions to wait for upstream services but has been removed in newer releases.
Add the ephemeral-storage resource configuration if required by the new chart version.
Newer remote executor Helm charts may require explicit ephemeral storage resource requests/limits. Add the appropriate block under your executor's resource configuration:
```
resources:
  requests:
    cpu: "500m"
    memory: "1Gi"
    ephemeral-storage: "1Gi"
  limits:
    cpu: "2"
    memory: "4Gi"
    ephemeral-storage: "4Gi"
```
Adjust the values above to match your environment's capacity requirements.

Apply the updated Helm values and redeploy.

Run your Helm upgrade command with the cleaned-up values.yaml:

helm upgrade <release-name> <chart-repo>/datahub-remote-executor \
  --version <target-version> \
  --namespace <your-namespace> \
  --values values.yaml \
  --timeout 20m \
  --wait

(Optional) Verify image pull is not also a contributing factor.
If the deployment still times out after fixing the configuration, also verify that your environment can pull from the DataHub Cloud ECR registry without network restrictions:
```
# Test image pull directly from the node or a debug pod
docker pull <ecr-registry-url>/datahub-remote-executor:<target-version>
```
Ensure the node IAM role has ECR read permissions and that no VPC or firewall rules block outbound access to the ECR endpoint.

Additional Notes

Stale values.yaml files are a common source of upgrade failures when the Helm chart structure changes between DataHub Cloud releases. It is strongly recommended to diff your existing values.yaml against the new chart's default values file (helm show values <chart>) before every major version upgrade. The dockerize dependency was removed in recent remote executor chart versions and should not be present in deployments targeting current releases. Additionally, DataHub Cloud upgrade notifications will specify the recommended CLI version and image tag — always align your remote executor image version to the release version announced for your instance to avoid compatibility issues.

Issue

You Might Be Asking

Error Messages

Solution

Additional Notes

Related Documentation