Skip to main content

Remote Executor Image Security Vulnerabilities: certifi, Starlette, and Other Dependencies

Angelo Bresci
Updated

Area: Deployment Issues

Sub-Area: Remote Executor Image — Security Scan Findings

Issue

Security scanners (e.g., Xray, GHSA-based tools) report critical or high-severity vulnerabilities in the DataHub Cloud Remote Executor image. Common findings include outdated versions of certifi (advisory GHSA-6q48-6c7w-9gw2), starlette (CVE-2026-48710 / "BadHost"), and other Python or Java dependencies. These findings appear after upgrading the executor image to a newer version and persist because the affected packages are baked into the image at build time — they cannot be patched at runtime. Users may also observe a stale uv.lock file inside the image that reports older pinned versions even when the runtime environment uses a newer, patched version.

Error Messages

  • vuln is GHSA-6q48-6c7w-9gw2, detected in 3 packages pkg:pypi/certifi@<version>
  • CVE-2026-48710 detected at /datahub-executor/uv.lock
  • CVE-2026-8838 detected at /datahub-executor/uv.lock (redshift-connector)

You Might Be Asking

  • Why does my security scanner still flag vulnerabilities after I upgraded to a newer Remote Executor image?
  • Can I patch certifi or starlette inside the running container to clear the scan findings?
  • The uv.lock file inside the image shows an old package version — does that mean the vulnerability is actually present at runtime?
  • Which executor image version contains the fix for GHSA-6q48-6c7w-9gw2 (certifi) or CVE-2026-48710 (Starlette)?
  • Why does my scanner report multiple copies of certifi at different versions in the same image?

Solution

  1. Understand why multiple copies of a package appear in the scan.

    The Remote Executor image bundles Python dependencies in several distinct locations. For certifi, scanners commonly report three separate copies:

    • The executor's own Python virtual environment (e.g., /home/datahub/.venv)
    • The copy vendored inside pip itself (pip vendors its own certifi for TLS bootstrapping)
    • Bundled ingestion plugin environments

    All three locations must be updated simultaneously, which requires a new image build from DataHub. Upgrading packages inside a running container will not remove them from image-layer scans.

  2. Identify whether the finding is in a file that is actually used at runtime.

    The uv.lock file located at /datahub-executor/uv.lock inside some earlier image versions was a stale artifact left over from the build process. It was not read or used by the executor at runtime. If your scanner flags a CVE solely based on a package version pinned in this lock file, verify the actual installed version in the runtime environment before treating the finding as exploitable.

    # Run inside the executor container to check the runtime certifi version
python -c "import certifi; print(certifi.__version__)"

# Check the pip-vendored certifi version
pip debug --verbose 2>/dev/null | grep certifi || \
  python -c "import pip._vendor.certifi; print(pip._vendor.certifi.__version__)"

    If your scanner flags only the stale uv.lock and the runtime version is patched, you may document this as a non-exploitable finding. The stale lock file issue was permanently resolved in image builds starting with v2.2.0-cloud.

  3. Upgrade to the image version that contains the fix.

    The table below summarizes which image versions address which findings:

    • GHSA-6q48-6c7w-9gw2 (certifi — compromised emSign root CA certificates): Fixed in v2.3.6-cloud and later. This advisory was published after the v2.3.x series was initially built, so all v2.3.x images prior to v2.3.6 carry affected certifi versions. The fix raises certifi to 2026.10.19 or later in all three locations and upgrades pip to 26.2.2 (which vendors the fixed certifi).
    • CVE-2026-48710 / "BadHost" (Starlette — host header validation bypass): The runtime environment in v2.1.1-cloud and v2.1.1-cloud-slim and later uses Starlette 1.3.1, which is patched. The stale uv.lock artifact that caused false positives was removed in v2.2.0-cloud.
    • Stale uv.lock file (affects multiple CVE findings reported against the lock file path): Fixed in v2.2.0-cloud and all later v2.2.x-cloud and v2.3.x-cloud builds.

    To upgrade, update your executor deployment to reference the target image tag. For example, using the slim variant:

    # Pull the fixed executor image (slim variant)
docker pull docker.datahub.com/re/datahub-executor:v2.3.6-cloud-slim

# Or, for the standard variant
docker pull docker.datahub.com/re/datahub-executor:v2.3.6-cloud

    Update your Kubernetes, OpenShift, or other deployment manifests to reference the new tag, then restart the executor pool.

  4. Rescan after upgrading and verify the expected versions.

    After pulling the new image, run your security scanner again. For the certifi fix in v2.3.6-cloud, expect all three certifi locations to report version 2026.10.19 or later, and pip to report 26.2.2 or later. GHSA-6q48-6c7w-9gw2 should no longer appear.

    # Verify certifi in the executor venv
docker run --rm docker.datahub.com/re/datahub-executor:v2.3.6-cloud-slim \
  /home/datahub/.venv/bin/python -c "import certifi; print(certifi.__version__)"
# Expected: 2026.10.19 or later

# Verify pip version (which determines vendored certifi)
docker run --rm docker.datahub.com/re/datahub-executor:v2.3.6-cloud-slim \
  /home/datahub/.venv/bin/pip --version
# Expected: pip 26.2.2 or later
  5. If your organization requires a signed statement of non-exploitability before the patched image is available:

    If a scan finding is located only in code that the executor image never executes (e.g., a stale lock file or a PySpark-bundled Java library that is not on any active code path), DataHub Support can review the specific finding and, where appropriate, provide documentation confirming that the vulnerable code path is not reachable in your deployment. Contact DataHub Support with the full scan report, including the scanner name, CVE/GHSA IDs, affected package names, and file paths.

Additional Notes

  • Security findings in the Remote Executor image originate from three distinct layers: Python packages in the executor virtual environment, packages vendored inside tools such as pip, and Java libraries bundled within PySpark. Fixes are applied per-finding and per-layer; no single update clears all possible future CVEs.
  • The advisory GHSA-6q48-6c7w-9gw2 was published after the initial v2.3.x series was built. It is not a regression — earlier 2.3.x images were compliant at the time of release. The fix was backported to the 2.3 branch and shipped as v2.3.6-cloud.
  • The stale uv.lock artifact issue (causing false-positive CVE reports based on the lock file path rather than the runtime environment) affects images prior to v2.2.0-cloud. All v2.2.x-cloud and later images have this artifact removed.
  • The -slim image variant omits some less commonly used features to reduce surface area. Confirm that any features your pipelines depend on are available in the slim variant before switching.
  • Runtime patching of packages inside the executor container (e.g., pip install --upgrade certifi) will not update packages baked into image layers and will not clear image-layer scanner findings. The correct remediation is always to upgrade to a new image version.
  • If your scanner reports findings that were previously marked as fixed in an earlier image version, collect the full report (CVE IDs, package names, file paths, and image digest) and open a support ticket so the DataHub team can determine whether a pin was dropped during a subsequent build.

Related Documentation

Tags: remote-executor, security-scan, CVE, certifi, GHSA-6q48-6c7w-9gw2, starlette, CVE-2026-48710, uv-lock, datahub-executor, image-vulnerability