Remote Executor ECS Deployment: Granting ECR Access and Matching Executor Image to Backend Version – DataHub Customer Support Portal

Area: Deployment Issues

Sub-Area: Remote Executor / AWS ECS Setup

Issue

When deploying a DataHub Remote Executor on AWS ECS using the provided CloudFormation template, two common failures can occur in sequence. First, ECS tasks fail immediately with a CannotPullContainerError (HTTP 403 Forbidden) because the customer's AWS account has not been granted permission to pull the DataHub executor image from the DataHub-managed Amazon ECR repository. Second, after ECR access is granted and the stack deploys successfully, the executor may still fail to register with the DataHub backend due to a GraphQL schema validation error caused by a version mismatch between the executor container image and the DataHub Cloud backend version.

Error Messages

CannotPullContainerError: pull image manifest has been retried 1 time(s): failed to resolve ref <ecr-registry>.dkr.ecr.us-west-2.amazonaws.com/datahub-executor:<image-tag>: unexpected status from HEAD request to https://<ecr-registry>.dkr.ecr.us-west-2.amazonaws.com/v2/datahub-executor/manifests/<image-tag>: 403 Forbidden
GraphError: Error executing graphql query: [{'message': "Validation error (FieldUndefined@[listExecutorConfigs/executorConfigs/previousChannel]) : Field 'previousChannel' in type 'ExecutorConfigs' is undefined", 'locations': [...], 'extensions': {'classification': 'ValidationError'}}]

You Might Be Asking

Why does my ECS task get a 403 Forbidden error when trying to pull the DataHub executor image from ECR?
My CloudFormation stack deployed successfully but the executor pool shows no active connections — what went wrong?
How do I know which executor image tag to use when deploying the Remote Executor?
Why am I seeing a GraphQL FieldUndefined validation error in my executor logs after a successful ECR pull?

Solution

Part 1: Granting Your AWS Account ECR Pull Access

The DataHub executor container image is hosted in a DataHub-managed private ECR repository. Your AWS account must be explicitly allowlisted in the ECR repository policy before ECS tasks in your account can pull the image. This is a one-time action performed by the DataHub Support team.

Identify your AWS account ID. You can find it in the AWS Console under Account Settings or via the CLI:
```
aws sts get-caller-identity --query Account --output text
```
Open a support ticket with DataHub Support and provide:
- Your AWS account ID (e.g., <your-aws-account-id>)
- The AWS region where your ECS cluster is deployed
DataHub Support will add your account to the ECR repository policy. You will be notified when the change is complete.
Once confirmed, re-run your CloudFormation stack deployment or restart the failing ECS task. The 403 Forbidden error should no longer occur.

Part 2: Matching the Executor Image Version to Your DataHub Backend Version

The CloudFormation template may default to a newer executor image tag (e.g., v2.0.0-cloud) that is ahead of your DataHub Cloud backend version. When the executor image version is newer than the backend, the executor queries GraphQL fields that do not yet exist in your backend schema, causing a FieldUndefined validation error and preventing the executor pool from registering active connections.

Identify your current DataHub Cloud backend version. You can find this in the DataHub UI under Settings > About, or by asking your DataHub account team.
When deploying the CloudFormation stack, pin the ImageTag parameter to match your DataHub backend version. For example, if your backend is on v1.1.3, set:
```
ImageTag: v1.1.3-cloud
```

If you already deployed the stack with a mismatched image tag, update the CloudFormation stack with the corrected ImageTag parameter value and apply the change:

aws cloudformation update-stack \
  --stack-name <your-stack-name> \
  --use-previous-template \
  --parameters ParameterKey=ImageTag,ParameterValue=<your-backend-version>-cloud \
               ParameterKey=<OtherParamKey>,UsePreviousValue=true

After the stack update completes, verify in the DataHub UI under Data Sources > Executors that your executor pool shows active connections.

Additional Workaround: Assigning a Custom Executor Pool via CLI

If the DataHub UI does not allow you to select a custom executor pool when creating a new ingestion source (the source defaults to the default pool regardless of selection), use the DataHub CLI to deploy the ingestion recipe directly and route it to the correct executor pool:

datahub ingest deploy \
  --executor-id <your-pool-id> \
  -c <path-to-your-recipe.yaml>

Additional Notes

ECR allowlisting is required for every distinct AWS account ID that needs to pull the DataHub executor image. If you add a new AWS account to your organization, you must request allowlisting for that account separately.
Always ensure the executor image tag used in your CloudFormation parameters exactly matches your DataHub Cloud backend version (e.g., v1.1.3-cloud for backend version v1.1.3). Using a newer executor image against an older backend will cause GraphQL schema validation failures.
The executor pool name remote is case-sensitive and must be created in the DataHub UI under Data Sources > Executors before deploying the CloudFormation stack.
The UI limitation around executor pool assignment for new ingestion sources may be addressed in future DataHub releases. Check the DataHub changelog for updates.