Ingesting GraphQL Schemas into DataHub via OpenAPI Conversion – DataHub Customer Support Portal

Area: Ingestion Issues

Sub-Area: API Schema Ingestion / OpenAPI Connector

Issue

Organizations that expose data through GraphQL APIs want to catalog those API schemas in DataHub so that non-technical users can browse available datasets and, ideally, trace lineage from underlying data tables through to the API endpoints. DataHub does not currently provide a native ingestion connector that points directly at a GraphQL endpoint and pulls in the schema, leaving teams uncertain about the best path forward.

You Might Be Asking

Is there a native GraphQL ingestion connector in DataHub?
How can I catalog my GraphQL API schemas in DataHub?
What tools can convert a GraphQL schema to an OpenAPI specification?
Can DataHub show lineage between GraphQL API endpoints and the underlying database tables they query?

Solution

The recommended approach is a two-step process: convert your GraphQL schema to an OpenAPI (Swagger) specification, then ingest that OpenAPI spec into DataHub using the OpenAPI ingestion source.

Generate an OpenAPI specification from your GraphQL service.
Several open-source tools can perform this conversion. Choose the one that fits your existing build toolchain:
- WunderGraph / Cosmo — if you are already building with WunderGraph, it can emit an OpenAPI spec as part of its build process.
- Sofa — wraps a GraphQL schema and exposes REST endpoints with an accompanying OpenAPI spec.
- graphql-to-openapi — a standalone conversion utility.
Note: The tool openapi-to-graphql works in the opposite direction (wrapping an OpenAPI source in a GraphQL layer) and is not suitable for this use case.
Host or expose the generated OpenAPI spec over HTTP.
The DataHub OpenAPI ingestion source fetches the spec via an HTTP URL. Ensure your generated .json or .yaml spec file is reachable from the machine running the DataHub ingestion process. No authentication against the spec endpoint is required by the connector.

Configure the DataHub OpenAPI ingestion source.

Create a recipe file (YAML) that points the OpenAPI source at your spec URL. A minimal example is shown below:

source:
  type: openapi
  config:
    url: "https:///openapi.json"   # URL to your OpenAPI v2 or v3 spec
    name: ""

sink:
  type: datahub-rest
  config:
    server: "https:///api/gms"
    token: ""

Run the ingestion recipe.
Execute the recipe using the DataHub CLI or schedule it through the DataHub UI ingestion manager:
```
# Using the DataHub CLI
datahub ingest -c openapi-recipe.yaml
```
The OpenAPI source will read each endpoint defined in the spec and model it as a dataset in DataHub, with the schema derived from the spec's request/response definitions.

(Optional) Establish lineage between API endpoints and underlying tables.

If your underlying data tables are already cataloged in DataHub, you can create lineage edges between those datasets and the API endpoint datasets. At present, this lineage must be emitted manually or via a custom ingestion script using the DataHub Python SDK. An example of emitting an upstream lineage edge programmatically:

from datahub.emitter.rest_emitter import DatahubRestEmitter
from datahub.metadata.schema_classes import (
    DatasetLineageTypeClass,
    UpstreamClass,
    UpstreamLineageClass,
)
from datahub.emitter.mcp import MetadataChangeProposalWrapper
import datahub.metadata.schema_classes as models

emitter = DatahubRestEmitter(
    gms_server="https:///api/gms",
    token="",
)

upstream_table_urn = "urn:li:dataset:(urn:li:dataPlatform:,..,PROD)"
api_dataset_urn    = "urn:li:dataset:(urn:li:dataPlatform:openapi,.,PROD)"

upstream_lineage = UpstreamLineageClass(
    upstreams=[
        UpstreamClass(
            dataset=upstream_table_urn,
            type=DatasetLineageTypeClass.TRANSFORMED,
        )
    ]
)

mcp = MetadataChangeProposalWrapper(
    entityUrn=api_dataset_urn,
    aspect=upstream_lineage,
)

emitter.emit_mcp(mcp)
print("Lineage emitted successfully.")

    Replace all placeholder values (<your-datahub-instance>, <platform>, <database>, etc.) with your actual environment details.
  


Additional Notes

  Native GraphQL schema ingestion is a known feature request and is not available as of the time of this writing. The OpenAPI conversion approach is the officially recommended workaround.
  The DataHub OpenAPI ingestion source supports both OpenAPI v2 (Swagger) and OpenAPI v3 specifications.
  For teams interested in automatically deriving lineage between GraphQL API endpoints and the OLTP tables they query (e.g., by crawling source code repositories), this is an area of active exploration. LLM-assisted code crawling is being considered as a mechanism to generate this lineage automatically, similar to how dbt lineage is handled. Contact DataHub support for the latest status on this capability.
  If you are already using WunderGraph/Cosmo as your GraphQL gateway, its built-in OpenAPI spec generation is a convenient starting point and avoids a separate conversion step.


Related Documentation

  DataHub OpenAPI Ingestion Source
  DataHub Metadata Ingestion Overview
  DataHub Lineage Feature Guide
  DataHub Python SDK — Emitter Reference


Tags: graphql, openapi, api-schema, ingestion, openapi-connector, schema-ingestion, lineage, api-catalog, graphql-to-openapi, no-native-connector