Ephemeral dbt Model Lineage Visibility

Lineage for ephemeral dbt models (models that exist only temporarily during dbt execution) is not visible or tracked in DataHub, making it difficult to understand data transformations that occur through ephemeral models.

Note: For DataHub Cloud customers, infrastructure-level changes are managed by DataHub Support.

1. Understand ephemeral model limitations:

   • Ephemeral models don't create physical database objects
   • Lineage for ephemeral models is inferred from dbt manifest
   • DataHub shows ephemeral models but lineage may be limited

2. Enable ephemeral model lineage:

source:
  type: dbt
  config:
    manifest_path: "target/manifest.json"
    catalog_path: "target/catalog.json"
    # Include ephemeral models
    node_type_pattern:
      allow: ["model"]  # Includes ephemeral
    write_semantics: "OVERRIDE"

3. Verify ephemeral models in manifest:

   • Check manifest.json contains ephemeral model definitions
   • Ensure materialization type is set to "ephemeral" in dbt
   • Confirm models have dependencies defined

4. View ephemeral model lineage:

   • Ephemeral models appear in lineage graph as intermediate nodes
   • Lineage flows through ephemeral models to downstream tables
   • Click on ephemeral model node to see its transformations

Test configurations in a development environment before applying to production. Always backup configurations before making changes.

• DataHub Documentation
• dbt Integration

• 5566