Remote Executor Version Pinning Best Practices – DataHub Customer Support Portal

Area: Deployment Issues

Sub-Area: Remote Executor Image Management & Registry Access

Issue

Organizations deploying the DataHub Cloud Remote Executor in self-hosted or on-premises environments frequently ask whether a latest floating image tag is available, or request guidance on how to pull the correct versioned executor image. Without a clear versioning strategy, infrastructure teams risk version mismatches between the Remote Executor and the DataHub Cloud instance, which can break ingestion pipelines and complicate auditing and rollback procedures.

You Might Be Asking

Is there a latest tag for the Remote Executor image that automatically tracks my DataHub Cloud instance version?
How do I know which executor image version to pull?
How do I get credentials to pull the Remote Executor image from the DataHub registry?
What happens if my executor version is behind or ahead of my DataHub Cloud instance version?
Can I use AWS ECR or GCP Artifact Registry instead of a token-based pull?

Solution

Why There Is No `latest` Tag

DataHub intentionally does not publish a latest floating tag for the Remote Executor image. The reasons are:

Version mismatch risk: DataHub Cloud upgrades roll out in waves. A global latest tag would frequently place the executor ahead of your instance version, which breaks ingestion.
Unreliable rollout behavior: Kubernetes and ECS cache images, so individual replicas can silently end up on different versions with no change reflected in your configuration.
Loss of change control: A moving tag changes the code running in your environment without a deliberate deploy on your side, making auditing, compliance, and rollback significantly harder.

Step 1 — Identify the Correct Version Tag

The executor image tag must match your DataHub Cloud instance version. Tags follow the pattern:

vX.Y.Z-cloud

Optional image variants append a suffix to this base tag:

vX.Y.Z-cloud-slim — smaller image footprint
vX.Y.Z-cloud-locked — dependency-locked build
vX.Y.Z-cloud-slim-unbundled — slim image without bundled connectors

Contact DataHub Support if you are unsure of your current instance version. As a general compatibility rule: if your server version is ahead of your executor version, they remain compatible unless a breaking change is explicitly announced in the release notes.

Step 2 — Obtain Registry Access

Access to the Remote Executor image is granted by DataHub Support. The method depends on where your cluster runs:

AWS (EKS): Provide your AWS account ID or the IAM principal used for image pulls. DataHub Support will grant pull access to the image in AWS ECR. Your node or pod role needs the AmazonEC2ContainerRegistryReadOnly managed policy.
```
795586375822.dkr.ecr.us-west-2.amazonaws.com/datahub-executor:<version>-cloud
```
GCP (GKE): Provide the cluster IAM service account (e.g., <project-number>-compute@developer.gserviceaccount.com). DataHub Support will grant pull access via GCP Artifact Registry.
```
us-docker.pkg.dev/acryl-prod/datahub/datahub-executor:<version>-cloud
```
On-premises, OpenShift, or other environments: DataHub Support will provide a registry token. Use it to authenticate against the DataHub registry directly.
```
docker login docker.datahub.com -u re
docker pull docker.datahub.com/re/datahub-executor:<version>-cloud
```
Username: re | Password: the token provided by Support via a one-time secret link.
Note: the token-based registry enforces a 20 GB daily download limit. AWS account grants do not apply to this endpoint.

Step 3 — Pin the Version in Your Deployment Configuration

Set the image tag explicitly in your deployment tooling. Do not rely on a mutable or default tag.

Helm values (datahub-executor-worker chart):

image:
  repository: docker.datahub.com/re/datahub-executor
  tag: v<X.Y.Z>-cloud
  pullPolicy: Always

Terraform variable:

datahub_image_tag = "v<X.Y.Z>-cloud"

CloudFormation parameter:

ImageTag: "v<X.Y.Z>-cloud"

If your cluster requires authentication to pull from the DataHub registry, create an image pull secret in the executor namespace:

kubectl create secret docker-registry datahub-re-pull-secret \
  --docker-server=docker.datahub.com \
  --docker-username=re \
  --docker-password=<your-registry-token> \
  --namespace=<executor-namespace>

Then reference the secret in your Helm values:

imagePullSecrets:
  - name: datahub-re-pull-secret

Step 4 — Update the Tag When Your Instance Is Upgraded

Confirm the new DataHub Cloud instance version with your DataHub representative or via the platform UI.
Update the image tag in your Helm values, Terraform variable, or CloudFormation parameter to the matching vX.Y.Z-cloud tag.
Redeploy the executor through your standard change process. This is a rolling update — your IAM roles, secrets, and pool configuration remain in place.
Record the tag change in your change management system for audit purposes.

Step 5 — Confirm the Executor Pool ID

When configuring the executor, the pool ID must match the value configured in the DataHub Cloud UI. If you have not customized this value, the default pool ID is:

remote

Additional Notes

A latest floating tag for the Remote Executor is not on the current product roadmap. This is by design, not an oversight. The official Helm chart for the executor worker always pins an exact vX.Y.Z-cloud tag, and this pattern should be followed in all customer deployments. Future work (tracked internally) may eventually decouple the executor version lifecycle from server releases, but this is exploratory and not committed. When requesting registry access, always provide your current DataHub Cloud instance version to Support so they can confirm the correct image tag. The token-based registry (docker.datahub.com/re/) and the cloud-provider registries (ECR, GCP Artifact Registry) are separate endpoints; credentials for one do not apply to the other.