Resolving Stale Glossary Term References That Reappear After Hard Deletion – DataHub Customer Support Portal

Area: Product Issues

Sub-Area: Glossary Term Management / Sibling Propagation

Issue

When glossary terms are hard-deleted in DataHub, the deletion removes the term entities themselves but does not automatically cascade to remove glossaryTerms aspect references on entities that were previously tagged with those terms. As a result, deleted term URNs continue to appear in the UI on datasets and other entities. Attempts to clean up the stale references with a script may appear to succeed but then reappear — most commonly because DataHub's sibling propagation automation re-applies the deleted term references by copying them from a sibling entity that was not cleaned in the same pass. In more complex cases, case-variant duplicate URNs (e.g., differing only in the casing of the dataset name within the URN) create a hidden third node in the sibling propagation loop, causing terms to circulate indefinitely even after visible entities appear clean.

You Might Be Asking

Will running a reindex remove stale glossary term associations after a hard delete?
Why do deleted glossary terms keep reappearing on my datasets even after I remove them with a script?
Why did my cleanup script report success but the terms came back after I re-enabled automations?
How does sibling propagation cause deleted glossary terms to return?
What should I do if a dataset has a case-variant duplicate URN that is re-seeding deleted glossary terms?

Solution

Follow these steps in order. The process has two phases: a standard cleanup for straightforward cases, and an extended cleanup for environments where case-variant duplicate URNs are involved.

Phase 1: Standard Cleanup (Most Cases)

Understand why a reindex will not help.
A reindex rebuilds the Elasticsearch/OpenSearch index from the primary metadata store. If the glossaryTerms aspect on an entity still contains the deleted term URNs in the metadata store, reindexing simply re-populates those same stale references back into search. The root data must be corrected in the metadata store itself first.
Pause sibling propagation automation before running any cleanup.
Navigate to Settings > Automations in the DataHub UI and disable the sibling propagation automation. If propagation remains active, it will copy glossary term references from any uncleaned sibling back onto the entity you just cleaned, undoing your work.

Identify all affected entities.

Use the DataHub Python SDK to search for entities still referencing each deleted term URN. Note that in some cases the search index may already be clean while the stored metadata still holds the stale reference. If a search returns zero results but the term still appears in the UI, target the entity directly by URN using graph.get_aspect().

import datahub.emitter.mce_builder as builder
from datahub.ingestion.graph.client import DatahubClientConfig, DataHubGraph
from datahub.metadata.schema_classes import GlossaryTermsClass

DATAHUB_SERVER = "https://<your-instance>/gms"
DATAHUB_TOKEN = "<your-personal-access-token>"  # or None if no auth

DELETED_TERM_URNS = [
    "urn:li:glossaryTerm:<term-guid-1>",
    "urn:li:glossaryTerm:<term-guid-2>",
    # add all deleted term URNs here
]

graph = DataHubGraph(
    DatahubClientConfig(
        server=DATAHUB_SERVER,
        token=DATAHUB_TOKEN,
    )
)

def find_entities_with_term(term_urn: str):
    """Search for all entities still referencing this term URN."""
    results = []
    for entity_urn in graph.get_urns_by_filter(
        extraFilters=[
            {
                "field": "glossaryTerms",
                "value": term_urn,
                "condition": "EQUAL",
            }
        ]
    ):
        results.append(entity_urn)
    return results

Remove stale term references from every affected entity and all its siblings in a single pass.

For each entity identified (including all siblings — e.g., both a dbt dataset and its Snowflake sibling), retrieve the current glossaryTerms aspect, filter out the deleted term URNs, and emit the updated aspect. Cleaning only one side of a sibling pair while propagation is paused will not be permanent once propagation is re-enabled.

from datahub.emitter.mcp import MetadataChangeProposalWrapper
from datahub.metadata.schema_classes import GlossaryTermsClass, ChangeTypeClass

def remove_stale_terms(entity_urn: str, deleted_term_urns: list):
    """Remove deleted term URNs from the glossaryTerms aspect of an entity."""
    aspect = graph.get_aspect(entity_urn, GlossaryTermsClass)
    if aspect is None:
        print(f"No glossaryTerms aspect found for {entity_urn}")
        return

    original_count = len(aspect.terms)
    aspect.terms = [
        t for t in aspect.terms
        if t.urn not in deleted_term_urns
    ]
    removed_count = original_count - len(aspect.terms)

    if removed_count == 0:
        print(f"No stale terms found on {entity_urn} (aspect may already be clean)")
        return

    mcp = MetadataChangeProposalWrapper(
        entityUrn=entity_urn,
        aspect=aspect,
        changeType=ChangeTypeClass.UPSERT,
    )
    graph.emit(mcp)
    print(f"Removed {removed_count} stale term(s) from {entity_urn}")

# Example: clean an entity and its sibling together
entities_to_clean = [
    "urn:li:dataset:(urn:li:dataPlatform:dbt,<your-schema>.<your-table>,PROD)",
    "urn:li:dataset:(urn:li:dataPlatform:snowflake,<your-schema>.<your-table>,PROD)",
]

for entity_urn in entities_to_clean:
    remove_stale_terms(entity_urn, DELETED_TERM_URNS)

Verify all entities are clean before re-enabling propagation.

Confirm each entity directly using graph.get_aspect() rather than relying solely on the UI or search results, as those may lag behind the metadata store.

def verify_entity_clean(entity_urn: str, deleted_term_urns: list):
    aspect = graph.get_aspect(entity_urn, GlossaryTermsClass)
    if aspect is None:
        print(f"{entity_urn}: no glossaryTerms aspect (clean)")
        return
    remaining = [t.urn for t in aspect.terms if t.urn in deleted_term_urns]
    if remaining:
        print(f"{entity_urn}: STILL HAS stale terms: {remaining}")
    else:
        print(f"{entity_urn}: clean")

for entity_urn in entities_to_clean:
    verify_entity_clean(entity_urn, DELETED_TERM_URNS)

Re-enable sibling propagation automation.
Return to Settings > Automations and re-enable the sibling propagation automation only after all siblings have been verified clean.

Phase 2: Extended Cleanup for Case-Variant Duplicate URNs

If terms continue to reappear after completing Phase 1, a case-variant duplicate URN may be creating a hidden sibling node that is not visible in the standard UI. This can happen when ingestion has produced two URNs for the same logical dataset that differ only in the casing of the dataset name string (e.g., urn:li:dataset:(urn:li:dataPlatform:dbt,SCHEMA.TABLE,PROD) and urn:li:dataset:(urn:li:dataPlatform:dbt,schema.table,PROD)). The hidden copy participates in sibling propagation and re-seeds the deleted terms onto the visible entities.

Identify the hidden case-variant entity.

Construct the expected alternate-case URN for any dataset where terms keep returning, then check whether it exists and holds stale term references:

# Check if a case-variant duplicate URN exists and holds stale terms
hidden_urn = "urn:li:dataset:(urn:li:dataPlatform:dbt,<alternate-case-schema>.<alternate-case-table>,PROD)"
aspect = graph.get_aspect(hidden_urn, GlossaryTermsClass)
if aspect:
    stale = [t.urn for t in aspect.terms if t.urn in DELETED_TERM_URNS]
    print(f"Hidden entity holds stale terms: {stale}")
else:
    print("No glossaryTerms aspect on hidden entity")

Clean all three nodes in a single pass while propagation is paused.

With propagation disabled, run remove_stale_terms() against the visible dbt entity, its Snowflake sibling, and the case-variant dbt entity in one execution.

all_nodes = [
    "urn:li:dataset:(urn:li:dataPlatform:dbt,SCHEMA.TABLE,PROD)",          # visible dbt entity
    "urn:li:dataset:(urn:li:dataPlatform:snowflake,schema.table,PROD)",     # Snowflake sibling
    "urn:li:dataset:(urn:li:dataPlatform:dbt,schema.table,PROD)",           # hidden case-variant
]

for entity_urn in all_nodes:
    remove_stale_terms(entity_urn, DELETED_TERM_URNS)

Soft-delete the case-variant duplicate entity.
After cleaning it, soft-delete the case-variant URN so it cannot re-enter the propagation loop in the future. This can be done via the DataHub CLI or the REST API:
```
# Using the DataHub CLI
datahub delete \
  --urn "urn:li:dataset:(urn:li:dataPlatform:dbt,<alternate-case-schema>.<alternate-case-table>,PROD)" \
  --soft
```
Verify all nodes are clean, then re-enable propagation.
Run verify_entity_clean() against all nodes before re-enabling the automation.

Additional Notes

Hard-delete does not cascade to entity aspects. DataHub's cascade deletion logic (DeleteEntityService.deleteReferencesTo()) relies on the graph index being current at the time of deletion. If graph index entries are stale or missing, the cascade may find no incoming edges and leave glossaryTerms aspects on downstream entities untouched. This is a known limitation; aspect cleanup must be performed explicitly.
Reindexing is not a fix. A reindex copies data from the metadata store into Elasticsearch/OpenSearch. If the stale reference exists in the metadata store, it will be reindexed. Fix the metadata store first.
Search results may already be clean while stored metadata is not. In some cases the search index is cleaned by a previous operation while the underlying aspect still holds the stale reference. Always verify directly via graph.get_aspect() rather than relying on search or UI results alone when confirming cleanup.
Propagation must be paused for the entire cleanup window. Re-enabling propagation before all siblings (including any hidden case-variant entities) are clean will cause the automation to re-apply the deleted term references from any remaining dirty node.
Case-variant duplicate URNs are an ingestion artifact. They arise when different ingestion sources or runs produce URNs for the same logical dataset with inconsistent casing. Resolving the upstream ingestion configuration to produce consistent URN casing prevents this class of issue from recurring. See the related ticket on duplicate URN detection for additional guidance.
After calling graph.emit(mcp), the MAE consumer automatically propagates the changes to Elasticsearch; no manual reindex is needed for the cleaned entities.