GraphQL API Query Timeout or Performance Issues

GraphQL queries to the DataHub API are timing out, returning slowly, or causing high server load. This commonly occurs when querying large datasets without pagination, requesting too many nested relationships, or not using appropriate filters.

Common Error Messages:

• Query execution timeout
• Gateway timeout (504)
• Response size too large
• Maximum query depth exceeded

You Might Be Asking:

• How do I paginate through large result sets?
• Why is my GraphQL query so slow?
• What are best practices for DataHub GraphQL queries?

1. Use pagination for large result sets:

query searchDatasets {
  search(
    input: {
      type: DATASET
      query: "*"
      start: 0
      count: 100  # Limit results per page
    }
  ) {
    start
    count
    total
    searchResults {
      entity {
        ... on Dataset {
          urn
          name
        }
      }
    }
  }
}

2. Avoid deep nested queries:

# Bad: Too many nested levels
query {
  dataset(urn: "...") {
    upstreamLineage {
      entities {
        upstreamLineage {
          entities {
            upstreamLineage {  # 3 levels deep
              entities { ... }
            }
          }
        }
      }
    }
  }
}

# Good: Query one level at a time
query {
  dataset(urn: "...") {
    upstreamLineage {
      entities {
        urn
        name
      }
    }
  }
}

3. Use specific filters to reduce result size:

query {
  search(
    input: {
      type: DATASET
      query: "sales"
      filters: [
        { field: "platform", values: ["snowflake"] }
        { field: "origin", values: ["PROD"] }
      ]
      start: 0
      count: 50
    }
  ) {
    searchResults {
      entity {
        urn
        name
      }
    }
  }
}

4. Request only needed fields:

# Bad: Requesting everything
query {
  dataset(urn: "...") {
    properties { ... }
    ownership { ... }
    tags { ... }
    glossaryTerms { ... }
    editableProperties { ... }
    # ... many more aspects
  }
}

# Good: Only what you need
query {
  dataset(urn: "...") {
    name
    description
    platform {
      name
    }
  }
}

5. For bulk operations, use batch queries:

query batchGetDatasets {
  dataset1: dataset(urn: "urn1") { name }
  dataset2: dataset(urn: "urn2") { name }
  dataset3: dataset(urn: "urn3") { name }
}

Consider using the REST API for bulk data operations. The GraphQL endpoint is optimized for UI interactions and targeted queries rather than large data exports.

• GraphQL API Guide
• DataHub APIs