Ghost Container Entries Persisting After Deletion Due to Search Index Lag – DataHub Customer Support Portal

Area: Product Issues

Sub-Area: Container Deletion / Search Index Synchronization

Issue

After deleting a container in the DataHub UI — particularly containers with no remaining datasets or child entities — the deleted container may continue to appear as a "ghost" entry in the browse sidebar or search results, even after performing a hard browser refresh. This behavior occurs because DataHub uses a soft-delete mechanism by default, and the Elasticsearch/OpenSearch search index may not immediately synchronize with the primary metadata store. If the asynchronous index update process (MAE consumer) encounters an error or lag during the delete event, the container entity remains visible in the browse and search layers despite being logically removed from the underlying store.

Error Messages

No visible error in the UI — the container entry simply continues to appear after deletion and hard refresh.

You Might Be Asking

Why does a deleted container still show up in the DataHub UI after I refresh the page?
How do I permanently remove a ghost container entry that reappears after deletion?
Why does hard-deleting a container in DataHub not remove it from browse or search results?
Is there a way to force DataHub to re-sync the search index after a container deletion?
Will deleting child datasets also remove the parent container from the UI?

Solution

Follow these steps in order to resolve ghost container entries after deletion:

Verify that all child entities are removed first.
DataHub will not automatically delete child entities (datasets, schemas, etc.) when a container is deleted. If any child entities still reference a deleted container via browsePathsV2, the container will continue to surface in browse and search results. Before proceeding, confirm the container has zero associated child entities.

You can check for remaining children using the DataHub REST API:
```
curl -X GET "https://<your-instance>.datahubproject.io/api/v2/entity/container/urn:li:container:<your-container-urn>" \
  -H "Authorization: Bearer <your-token>"
```
Perform a hard delete via the DataHub CLI.
A soft delete (the default UI action) sets status.removed = true but does not physically remove the entity. A hard delete via the CLI removes the entity and all associated aspects from the primary store, which is the correct remediation for ghost entries.
```
# Install the DataHub CLI if not already installed
pip install acryl-datahub

# Authenticate
datahub init

# Hard delete the ghost container by URN
datahub delete --urn "urn:li:container:<your-container-urn>" --hard
```
Replace <your-container-urn> with the actual URN of the ghost container. For DataHub Cloud deployments, this step may need to be performed by the DataHub Support team on your behalf.
Trigger a search index re-index for the affected entity type (DataHub Cloud only).
If the container URN has already been hard-deleted but the ghost entry still appears, the issue is a stale Elasticsearch/OpenSearch index entry. In DataHub Cloud, customers do not have direct access to the search index. Contact DataHub Support and request a targeted re-index for the affected container URN(s). This will flush the stale entries from the browse and search indices and bring them into alignment with the primary metadata store.

When contacting support, provide the following information:
- The full URN(s) of the ghost container(s) (e.g., urn:li:container:<your-container-urn>)
- The approximate date and time the deletion was attempted
- Your DataHub Cloud instance URL
For self-hosted deployments: manually trigger a re-index.
If you manage your own DataHub deployment, you can trigger a re-index of the affected entity type using the DataHub System Update job or via the GraphQL API. The following GraphQL mutation can initiate a re-index for containers:
```
mutation {
  rollbackIngestion(input: {
    runId: "<your-ingestion-run-id>"
  })
}
```
Alternatively, use the DataHub admin endpoint to trigger a full or partial re-index of the container entity type through your Elasticsearch/OpenSearch cluster management interface.
Verify resolution.
After the hard delete and re-index are complete, perform a hard browser refresh (Ctrl+Shift+R or Cmd+Shift+R) and confirm the ghost container entries no longer appear in the browse sidebar or search results.

Additional Notes

This behavior is related to a known bug in older DataHub Cloud versions (prior to v0.3.7.7) where the MAE (Metadata Audit Event) consumer could throw an exception during container delete events, causing the Elasticsearch index update to fail silently. Ensure your DataHub Cloud instance is up to date.
A fix related to browse path synchronization (fix(browse): use platform instance URN in default BrowsePathsV2 generation) was released in DataHub Cloud v2.0.0-cloud. If you are running a version prior to this, upgrading may resolve or prevent recurrence of this issue.
DataHub Cloud customers do not have direct access to Elasticsearch/OpenSearch operations. All index-level remediation (re-index, restore) must be performed by the DataHub Support team.
Ghost entries that survive a hard delete are always a server-side issue — client-side cache clearing or hard browser refreshes will not resolve the underlying index inconsistency.
If ghost container entries reappear after a successful re-index, investigate whether any active ingestion sources are re-creating the containers. Check all ingestion pipeline configurations for references to the deleted container names.

Issue

Error Messages

You Might Be Asking

Solution

Additional Notes

Related Documentation