GitHub Actions Framework for Impact Analysis

Development teams want to integrate DataHub's impact analysis capabilities into their GitHub Actions workflows to automatically assess the downstream impact of code changes (e.g., dbt model modifications, schema changes) before merging pull requests. Teams are looking for pre-built GitHub Actions or frameworks to trigger impact analysis during CI/CD pipelines.

You Might Be Asking:

• Is there a GitHub Actions integration for DataHub impact analysis?
• How can I trigger impact analysis from GitHub?
• What frameworks are available beyond dbt?
• Can I block PR merges based on impact analysis results?

DataHub provides a dbt-specific GitHub Action for impact analysis. For other use cases, custom integrations can be built using the DataHub API.

Available Framework: dbt Impact Analysis Action

GitHub Action: datahub-dbt-impact-action

Repository: acryldata/datahub-dbt-impact-action

Usage Example:

# .github/workflows/dbt-impact-analysis.yml
name: dbt Impact Analysis

on:
  pull_request:
    paths:
      - 'dbt/**/*.sql'
      - 'dbt/**/schema.yml'

jobs:
  impact-analysis:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout code
        uses: actions/checkout@v3

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install dbt
        run: pip install dbt-snowflake

      - name: Run DataHub Impact Analysis
        uses: acryldata/datahub-dbt-impact-action@v1
        with:
          datahub-url: ${{ secrets.DATAHUB_URL }}
          datahub-token: ${{ secrets.DATAHUB_TOKEN }}
          dbt-project-dir: './dbt'
          dbt-target: 'dev'
          # Optional: fail on high-impact changes
          fail-on-high-impact: false

      - name: Comment PR with Impact
        uses: actions/github-script@v6
        with:
          script: |
            // Add impact analysis results as PR comment

Custom Impact Analysis for Non-dbt Sources:

For other data sources, build custom GitHub Actions using the DataHub API:

# .github/workflows/custom-impact-analysis.yml
name: Custom Impact Analysis

on:
  pull_request:
    paths:
      - 'sql/**'

jobs:
  analyze-impact:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v3

      - name: Get Changed Files
        id: changed-files
        uses: tj-actions/changed-files@v35
        with:
          files: |
            sql/**/*.sql

      - name: Setup Python
        uses: actions/setup-python@v4
        with:
          python-version: '3.10'

      - name: Install DataHub CLI
        run: pip install acryl-datahub

      - name: Analyze Impact
        env:
          DATAHUB_URL: ${{ secrets.DATAHUB_URL }}
          DATAHUB_TOKEN: ${{ secrets.DATAHUB_TOKEN }}
        run: |
          python scripts/analyze_impact.py \
            --changed-files "${{ steps.changed-files.outputs.all_changed_files }}"

      - name: Post Results
        run: |
          # Post impact analysis to PR comments

Impact Analysis Script Example:

# scripts/analyze_impact.py
import sys
from datahub.emitter.rest_emitter import DatahubRestEmitter
import requests

def get_downstream_impact(dataset_urn, datahub_url, token):
    """Get downstream datasets and dashboards affected by a change"""

    headers = {"Authorization": f"Bearer {token}"}

    query = f"""
    query {{
      entity(urn: "{dataset_urn}") {{
        ... on Dataset {{
          downstreamLineage {{
            entities {{
              urn
              type
              ... on Dataset {{
                name
                platform {{
                  name
                }}
              }}
              ... on Dashboard {{
                name
              }}
            }}
          }}
        }}
      }}
    }}
    """

    response = requests.post(
        f"{datahub_url}/api/graphql",
        json={"query": query},
        headers=headers
    )

    return response.json()

def analyze_changes(changed_files):
    """Analyze impact of changed SQL files"""
    impacts = []

    for file in changed_files:
        # Parse file to extract affected tables
        # Query DataHub for lineage
        # Collect downstream dependencies
        dataset_urn = extract_dataset_from_file(file)
        impact = get_downstream_impact(dataset_urn, DATAHUB_URL, TOKEN)
        impacts.append(impact)

    # Generate impact report
    return impacts

def post_pr_comment(impact_report):
    """Post impact analysis as PR comment"""
    comment = f"""
    ## 📊 DataHub Impact Analysis

    ### Changes Detected
    - {len(changed_files)} SQL files modified

    ### Downstream Impact
    - {impact_report['datasets_affected']} datasets affected
    - {impact_report['dashboards_affected']} dashboards may be impacted
    - {impact_report['high_priority_assets']} high-priority assets affected

    ### Recommendations
    {impact_report['recommendations']}
    """

    # Post using GitHub API

Advanced: Block PRs Based on Impact:

- name: Check Impact Threshold
  run: |
    if [ $AFFECTED_DASHBOARDS -gt 10 ]; then
      echo "::error::Too many dashboards affected. Please review with data team."
      exit 1
    fi

Integration with Other Tools:

Airflow: - Use the DataHub Airflow plugin for automatic lineage - No special GitHub Actions needed

Spark: - Use DataHub Spark lineage integration - Capture lineage during job execution

Custom ETL: - Emit lineage programmatically using the SDK - Query lineage via API in GitHub Actions

• The dbt impact action is the most mature integration
• For other sources, custom development is required
• Impact analysis requires lineage to be ingested and up-to-date
• Consider impact analysis as a blocking or warning step based on your needs
• Integrate with CODEOWNERS for automatic reviewer assignment based on impact

• dbt Impact Analysis Action
• Lineage in DataHub
• GraphQL API

• 5093