Orphaned Iceberg View Returning 404 from Iceberg REST Catalog Due to Missing S3 Metadata – DataHub Customer Support Portal

Area: API Issues

Sub-Area: Iceberg REST Catalog — Catalog State Drift / Orphaned Entities

Issue

An Iceberg view is visible in the DataHub UI but returns a 404 Not Found error when accessed via the Iceberg REST Catalog (IRC) API. Other views in the same warehouse function correctly. Investigation reveals that the underlying S3 metadata file referenced by DataHub's IRC no longer exists — the metadata pointer stored in DataHub points to a file that has been deleted or otherwise removed from object storage. As a result, DataHub's metadata store still contains the entity (making it visible in the UI), but the IRC cannot resolve the view because its physical metadata is missing. This catalog state drift leaves an orphaned entity that cannot be re-created through the IRC because DataHub believes the view already exists.

Error Messages

Received GET response: InboundJaxrsResponse{context=ClientResponse{method=GET, uri=https://<your-instance>.acryl.io/gms/iceberg/v1/<warehouse>/namespaces/<namespace>/views/<view_name>, status=404, reason=Not Found}}

You Might Be Asking

Why does my Iceberg view appear in the DataHub UI but return 404 from the IRC API?
How do I delete an orphaned Iceberg entity from DataHub so I can re-create it?
What causes catalog state drift between the DataHub metadata store and the Iceberg REST Catalog?
Can I re-create an Iceberg view when DataHub still thinks the old one exists?

Solution

Confirm the root cause — verify the S3 metadata file is missing.
Check whether the metadata JSON file referenced by DataHub's IRC still exists in object storage. The metadata pointer will follow a path similar to:
```
s3://<your-bucket>/<warehouse-uuid>/<namespace>/<view_name>/metadata/<metadata-file>.gz.metadata.json
```
If this file is absent, the IRC cannot serve the view regardless of what is recorded in DataHub's metadata store. This confirms the entity is orphaned.
Delete the orphaned entity via the DataHub CLI or REST API — not via IRC APIs.
Because the view no longer has valid backing metadata, it must be removed directly from DataHub using the entity's URN. Use the DataHub CLI:
```
datahub delete --urn "urn:li:dataset:(urn:li:dataPlatform:iceberg,<warehouse>.<warehouse-uuid>,PROD)" --hard
```
Alternatively, use the DataHub REST API to issue a hard delete:
```
curl -X DELETE \
  "https://<your-instance>.acryl.io/api/v2/entity?urn=urn%3Ali%3Adataset%3A(urn%3Ali%3AdataPlatform%3Aiceberg%2C<warehouse>.<warehouse-uuid>%2CPROD)&hard=true" \
  -H "Authorization: Bearer <your-api-token>"
```
Replace <warehouse>, <warehouse-uuid>, and <your-api-token> with your actual values. Use --hard (CLI) or hard=true (API) to fully remove the entity from DataHub's metadata store and search index.
Verify the entity no longer appears in the DataHub UI.
After deletion, confirm the view is no longer listed in the UI under the relevant warehouse and namespace before attempting to re-create it.

Re-create the Iceberg view via the IRC API.

Once the orphaned entity has been fully removed, you can issue a standard IRC CREATE VIEW request. DataHub will register the new view and create fresh S3 metadata.

POST https://<your-instance>.acryl.io/gms/iceberg/v1/<warehouse>/namespaces/<namespace>/views
Content-Type: application/json
Authorization: Bearer <your-api-token>

{
  "name": "<view_name>",
  "schema": { ... },
  "view-definition": { ... }
}

Investigate how the S3 metadata file was deleted.
Determine whether the file was removed by an external process, a lifecycle policy, a manual deletion, or another pipeline operating on the same bucket. Preventing external modification of DataHub-managed IRC metadata paths will avoid recurrence of this issue.

Additional Notes

This issue is a form of catalog state drift: DataHub's metadata store and the physical object storage backing the Iceberg REST Catalog have fallen out of sync. The DataHub UI reflects only what is recorded in the metadata store, so an entity can appear healthy in the UI while being entirely unresolvable by the IRC. This scenario can arise when S3 lifecycle policies, external cleanup scripts, or manual operations delete Iceberg metadata files without going through the IRC API. Note that deletion must always be performed via the DataHub CLI or DataHub REST API using the entity's URN — attempting to delete through the IRC API will also fail with a 404 because the IRC itself cannot locate the backing metadata. On DataHub Cloud (Acryl-managed) deployments, customers do not have direct access to the underlying catalog infrastructure; if you are unable to perform the deletion yourself, contact DataHub Support to perform the cleanup on your behalf.

Issue

Error Messages

You Might Be Asking

Solution

Additional Notes

Related Documentation