dbt Ingestion: Compiled SQL Missing Because Source Manifest Was Generated Without Full Compilation – DataHub Customer Support Portal

Area: Ingestion Issues

Sub-Area: dbt Ingestion — Compiled SQL / View Definition

Issue

After ingesting dbt metadata into DataHub, the compiled SQL tab is missing or blank in the View Definition panel for one or more dbt models, even though the raw (Jinja) SQL is present. The formattedLogic field (which powers the "Compiled" tab in the UI) is null in the ingested ViewProperties aspect. This can affect individual models or large portions of a dbt project simultaneously, and the problem can appear or disappear between ingestion runs.

Error Messages

Missing compiled code for model <model_name> (logged in dbt ingestion run output)

You Might Be Asking

Why did the compiled SQL view disappear for my dbt models when nothing changed in my DataHub configuration?
The manifest.json in my target directory contains compiled_code — why isn't it showing up in DataHub?
Will upgrading the DataHub CLI fix the missing compiled SQL tab?
Does setting include_compiled_code: true in my recipe fix this issue?

Root Cause

dbt only populates the compiled_code field in manifest.json for models that were fully compiled during that specific dbt invocation. Commands such as dbt test, dbt source freshness, dbt parse, or a targeted dbt run that selects only a subset of models will produce a manifest where compiled_code is absent for models that were not compiled. When DataHub ingests one of these incomplete manifests, every model it touches has its ViewProperties aspect rewritten — including overwriting any previously stored compiled SQL with an empty value. The result is that the compiled tab disappears from the UI for affected models.

This is not a bug in DataHub's display logic or authorization layer, and it is not resolved by changing the CLI version alone. The include_compiled_code: true setting in the ingestion recipe instructs DataHub to ingest compiled SQL when it is present in the manifest — but if the manifest itself does not contain compiled_code, there is nothing to ingest regardless of this setting.

Solution

Identify which dbt command generates the manifest your ingestion reads.
Check your dbt ingestion recipe to find the path to manifest.json:
```
source:
  type: dbt
  config:
    manifest_path: 's3://<your-bucket>/<your-path>/target/manifest.json'
    catalog_path: 's3://<your-bucket>/<your-path>/target/catalog.json'
    target_platform: <your-platform>
```
Trace back which dbt CI/CD job or pipeline writes to this path and determine which dbt command it runs.
Ensure the manifest is produced by a full-compilation command.
Only the following dbt commands populate compiled_code for all selected models:
- dbt compile
- dbt build
- dbt run (only for the selected models)
- dbt docs generate
Commands that do not produce compiled SQL for all nodes:
- dbt parse
- dbt test
- dbt source freshness
- Any targeted run that selects only a subset of models
Update your pipeline so that the manifest uploaded to the path your ingestion recipe reads is always generated by dbt compile, dbt build, or dbt docs generate covering the full project.
```
# Recommended pipeline step before DataHub ingestion
dbt compile --profiles-dir <your-profiles-dir> --project-dir <your-project-dir>
# or
dbt docs generate --profiles-dir <your-profiles-dir> --project-dir <your-project-dir>
```
Confirm include_compiled_code is enabled in your recipe.
Ensure your dbt ingestion recipe includes (or relies on the default of) include_compiled_code: true. If you have an older recipe that uses the deprecated use_compiled_code key, update it:
```
source:
  type: dbt
  config:
    manifest_path: 's3://<your-bucket>/<your-path>/target/manifest.json'
    catalog_path: 's3://<your-bucket>/<your-path>/target/catalog.json'
    target_platform: <your-platform>
    include_compiled_code: true   # default is true; ensure this is not set to false
```
Note: The old key use_compiled_code was deprecated in early 2024 and replaced by include_compiled_code. If your recipe still uses the old key it will be silently ignored.
Trigger a fresh ingestion run after correcting the manifest source.
Once your pipeline is writing a fully-compiled manifest to the configured path, re-run your dbt ingestion. DataHub will rewrite the ViewProperties aspect for each ingested model and the compiled SQL will be repopulated.
Verify in the UI.
Navigate to a dbt model's dataset page and open the View Definition tab. If compiled SQL is present, you will see a Source / Compiled toggle. If only the Source tab is visible, the formattedLogic field is still empty, which indicates the manifest used for ingestion still lacks compiled_code for that model.

Diagnosing Scope of Impact

To understand how many models are affected before re-ingesting, review the ingestion run logs for occurrences of "Missing compiled code." You can also query the manifest directly:

# Check how many nodes in the manifest are missing compiled_code
python3 - <<'EOF'
import json

with open("manifest.json") as f:
    manifest = json.load(f)

nodes = manifest.get("nodes", {})
missing = [k for k, v in nodes.items() if not v.get("compiled_code")]
present = [k for k, v in nodes.items() if v.get("compiled_code")]

print(f"Nodes with compiled_code:    {len(present)}")
print(f"Nodes missing compiled_code: {len(missing)}")
EOF

A large number of missing nodes is a strong signal that the manifest was produced without a full compilation step.

Additional Notes

This issue can appear intermittently if multiple pipelines write to the same manifest path and only some of them run a full dbt compile. A later pipeline that uses dbt parse or dbt test will overwrite the S3 path with an incomplete manifest, causing compiled SQL to disappear after the next DataHub ingestion.
Models materialized as tables in the warehouse (e.g., Snowflake or Redshift tables) are particularly susceptible to losing compiled SQL if a secondary warehouse connector (such as the Snowflake or Redshift source) runs after dbt ingestion, because those connectors only emit ViewProperties for database-level views, not tables. Using include_compiled_code: true in the dbt recipe ensures compiled SQL is emitted by the dbt source itself, independent of downstream warehouse connectors.
Both the raw (Jinja) SQL and compiled SQL are supported simultaneously as of DataHub backend version 0.13.1 and later. The raw SQL is stored in viewProperties.logic and the compiled SQL in viewProperties.formattedLogic.
Upgrading the DataHub CLI version alone will not fix this issue if the underlying manifest does not contain compiled_code.