Skip to main content

Remote Executor Version Upgrade and Image Location

Angelo Bresci
Updated

Area: Deployment Issues

Sub-Area: Remote Executor Management

Issue

Teams upgrading DataHub Cloud remote executors need guidance on correct version numbers, image locations, and common configuration errors that occur during upgrades. This includes identifying the proper ECR registry path and avoiding formatting mistakes in version tags.

Error Messages

  • CannotPullContainerError: pull image manifest has been retried 7 time(s): failed to resolve ref 795586375822.dkr.ecr.us-west-2.amazonaws.com/datahub-executor:v.1.0.0-cloud: not found
  • 401 Client Error: Unauthorized for url: https://.acryl.io/gms/api/graphql

You Might Be Asking

  • What is the latest remote executor version for DataHub Cloud?
  • Where can I find the correct image registry path?
  • Why is my remote executor failing to pull the new image?
  • Do I need to update my access token when upgrading?

Solution

  1. Verify the correct version format: The latest DataHub Cloud remote executor version uses the format without a period between "v" and the version number: 
    v1.0.0-cloud
    NOT: 
    v.1.0.0-cloud
  2. Use the correct ECR registry path: DataHub Cloud customers should use the Acryl-managed ECR registry: 
    795586375822.dkr.ecr.us-west-2.amazonaws.com/datahub-executor:v1.0.0-cloud
  3. Update your deployment configuration: Replace the version tag in your configuration file: 
    {
  "image": "795586375822.dkr.ecr.us-west-2.amazonaws.com/datahub-executor:v1.0.0-cloud"
}
  4. Generate new access tokens if needed: If you've changed your DataHub instance subdomain or are seeing 401 authorization errors, create a new access token:
    • Navigate to your DataHub instance settings
    • Generate a new access token
    • Update your remote executor configuration with the new token
  5. Verify AWS permissions: Ensure your AWS account has proper permissions to access the ECR registry and that your deployment method (CloudFormation, ECS, etc.) is configured correctly.
  6. Consider available image variants: The executor offers several options:
    • v1.0.0-cloud (default) - standard bundled image
    • v1.0.0-cloud-slim - no bundled virtual environments
    • v1.0.0-cloud-locked - hardened with pinned dependencies
    • v1.0.0-cloud-unbundled - no pre-packaged dependencies

Additional Notes

The remote executor version should typically match your DataHub Cloud server version for optimal compatibility. DataHub Cloud manages the backend infrastructure, and customers only need to update the remote executor image configuration. Always verify the recommended version with DataHub support before upgrading production environments. Common deployment methods include CloudFormation templates and ECS task definitions.

Related Documentation

Tags: remote-executor, deployment, ecr-registry, version-upgrade, cloud, image-pull, aws, configuration, authentication, troubleshooting

Related articles