Skip to main content

Databricks Unity Catalog Ingestion Failing with 403 Forbidden: Required Permissions

Angelo Bresci
Updated

Area: Ingestion Issues

Sub-Area: Databricks Unity Catalog — Service Principal Permissions

Issue

When running a DataHub ingestion recipe for Databricks Unity Catalog, the ingestion job fails immediately with a 403 Forbidden error before any metadata is scanned. The failure occurs on the very first API call DataHub makes to enumerate Unity Catalog catalogs. This is not a DataHub infrastructure or SDK bug — it is a Databricks-side permissions issue: the service principal or token configured in the ingestion recipe does not have the necessary Unity Catalog privileges to list catalogs, schemas, or tables.

Error Messages

  • GET /api/2.1/unity-catalog/catalogs?include_browse=true&max_results=0
  • 403 Forbidden
  • User not authorized.
  • This is likely a bug in the Databricks SDK for Python or the underlying API. (misleading SDK wrapper message — the real cause is the 403)

You Might Be Asking

  • Why does my Databricks ingestion show events_produced: 0 and tables_scanned: 0?
  • Is the "likely a bug in the Databricks SDK for Python" message a real bug?
  • Why does the ingestion fail immediately without scanning any metadata?
  • What Unity Catalog privileges does the DataHub service principal need?

Solution

The service principal or token used in the DataHub ingestion recipe must be granted the appropriate Unity Catalog SQL privileges in Databricks. Follow the steps below as a Databricks metastore admin:

  1. Identify the service principal configured in your DataHub ingestion recipe for the Databricks source. You can find this in the ingestion recipe under the token or client_id / client_secret fields.
  2. Check existing grants for the service principal on the target catalog: 
    SHOW GRANTS ON CATALOG <your_catalog_name>;
  3. Grant the minimum required privileges for metadata discovery. At minimum, the service principal needs USE CATALOG on each catalog to ingest, and USE SCHEMA on the schemas within those catalogs: 
    GRANT USE CATALOG ON CATALOG <your_catalog_name> TO `<your_service_principal>`;
GRANT USE SCHEMA ON ALL SCHEMAS IN CATALOG <your_catalog_name> TO `<your_service_principal>`;
  4. Grant SELECT if profiling is enabled. This privilege is only required when column profiling is configured in your ingestion recipe: 
    GRANT SELECT ON ALL TABLES IN CATALOG <your_catalog_name> TO `<your_service_principal>`;
  5. Grant lineage privileges if you want DataHub to capture query-based lineage via the Databricks system.access schema: 
    GRANT USE SCHEMA ON SCHEMA system.access TO `<your_service_principal>`;
  6. For broad discovery across all catalogs, grant USE CATALOG or BROWSE at the metastore level rather than catalog-by-catalog. Consult your Databricks workspace admin to apply metastore-level grants appropriate for your environment.
  7. Validate the configuration before re-running a full ingestion. In the DataHub UI, navigate to the ingestion source and click the Test Connection button. A successful connection test confirms the service principal now has sufficient privileges to list catalogs.
  8. Re-run the ingestion. With correct permissions applied, the connector will proceed past the catalog listing step and begin scanning metadata.

Additional Notes

The Databricks SDK emits the message "This is likely a bug in the Databricks SDK for Python or the underlying API" as generic wrapper text whenever it receives a response it cannot parse according to its expected schema. This message is misleading — the root cause is always visible in the underlying HTTP response: 403 Forbidden / User not authorized. This is a Databricks permissions configuration problem, not a DataHub product defect or Databricks SDK bug.

Starting with newer versions of the Databricks SDK (approximately databricks-sdk==0.117.0 and later, used in acryl-datahub==1.6.0+), the catalog listing call includes the query parameter include_browse=true. This parameter is intended to surface catalogs for which the principal has BROWSE privilege even without USE CATALOG, but it also changes the permission evaluation path on the Databricks side. If you previously had a working ingestion with an older DataHub version and the same service principal, verify whether the principal's grants still satisfy the requirements under this updated API behavior.

For DataHub Cloud (managed) deployments, all executor infrastructure is managed by the DataHub Cloud team. Ingestion failures in this environment are almost always caused by configuration or credential issues in the ingestion recipe, not infrastructure problems. Use the Test Connection feature and review ingestion logs in the UI as your primary diagnostic tools.

Related Documentation

Tags: databricks, unity-catalog, ingestion, 403-forbidden, service-principal, permissions, unity-catalog-grants, databricks-sdk, metadata-ingestion, cloud