Skip to main content

Semantic Search Not Returning Results for Datasets and Tables

Angelo Bresci
Updated

Area: API Issues

Sub-Area: GraphQL API Semantic Search

Issue

The semanticSearchAcrossEntities GraphQL API returns empty results when querying for datasets, tables, or other asset types, even though these entities exist in DataHub and contain relevant metadata. This occurs because semantic search functionality is currently limited to DOCUMENT entities only, not datasets or other entity types.

You Might Be Asking

  • Why does semanticSearchAcrossEntities return no results for my datasets?
  • Can I use semantic search to find tables and datasets in DataHub?
  • How do I enable semantic search for all entity types?

Solution

  1. Understand current limitations: Semantic search currently only supports DOCUMENT entities (runbooks, FAQs, knowledge articles from Notion, Confluence, or DataHub Documents).
  2. Verify your query scope: Ensure you're querying the correct entity types: 
    query {
  semanticSearchAcrossEntities(
    input: {
      query: "your search term"
      types: [DOCUMENT]  # Only DOCUMENT type is currently supported
      start: 0
      count: 10
    }
  ) {
    searchResults {
      entity {
        urn
        type
      }
      score
    }
  }
}
  3. For dataset/table search: Use the standard searchAcrossEntities API instead: 
    query {
  searchAcrossEntities(
    input: {
      query: "your search term"
      types: [DATASET, TABLE]
      start: 0
      count: 10
    }
  ) {
    searchResults {
      entity {
        urn
        type
      }
    }
  }
}
  4. Check backend configuration: Verify semantic search is enabled in your deployment: 
    # In application.yaml
searchService:
  semantic:
    enabled: true
    enabledEntities: [document]  # Currently hardcoded to documents only
  5. Verify prerequisites: Ensure your deployment has:
    • OpenSearch 2.17+ with k-NN plugin, or Elasticsearch 8.x
    • Configured embedding provider (OpenAI, Cohere, AWS Bedrock, or Vertex AI)
    • Environment variable: SEARCH_SERVICE_SEMANTIC_SEARCH_ENABLED=true

Additional Notes

Future Support: DataHub is actively developing semantic search support for additional entity types including datasets, glossary terms, domains, and data products. This expanded functionality is planned for future releases. Current Workaround: For now, continue using keyword-based search via searchAcrossEntities for datasets and tables. Version Info: This limitation exists in all current DataHub versions as of 2024.

Related Documentation

Tags: semantic-search, api, graphql, search, documents, datasets, elasticsearch, opensearch, embeddings, vector-search

Related articles