Looker Integration Missing Explores or Dashboards

When ingesting from Looker, some explores, dashboards, or charts are not appearing in DataHub, or the lineage between Looker views and underlying database tables is incomplete. This typically occurs due to insufficient Looker API permissions, incorrect project configuration, or LookML parsing issues.

You Might Be Asking:

• Why aren't all my Looker dashboards showing up?
• How does DataHub extract lineage from Looker?
• What permissions does DataHub need in Looker?

1. Configure Looker source with proper credentials:

Example for Looker. This can be adapted for other BI tools.

source:
  type: looker
  config:
    base_url: "https://your-instance.looker.com"
    client_id: "${LOOKER_CLIENT_ID}"
    client_secret: "${LOOKER_CLIENT_SECRET}"

    # Enable comprehensive extraction
    extract_owners: true
    extract_usage_history: true

    # LookML parsing
    parse_table_names_from_sql: true
    project_name: "your_project"  # Specify if needed

2. Ensure API3 credentials have sufficient permissions:

   • Admin or equivalent access to Looker instance
   • Ability to read LookML project files
   • Access to usage/query history
   • Can view all explores and dashboards

3. Grant Looker API permissions:

Example for Looker. This can be adapted for other BI tools.

In Looker Admin:
1. Go to Admin → Users → API3 Credentials
2. Create credentials for DataHub integration user
3. Assign permissions:
   - see_lookml
   - see_user_dashboards
   - see_looks
   - see_queries
   - explore

4. For private/restricted dashboards:

Example for Looker. This can be adapted for other BI tools.

source:
  config:
    # Include specific folders
    dashboard_pattern:
      allow: ["^Shared/.*", "^Marketing/.*"]

    # Specific user's private dashboards
    user_id_pattern:
      allow: ["user123", "user456"]

5. Additional information about Looker configuration options that determine asset ingestion:

The main Looker configuration options that determine which assets are ingested by DataHub are: dashboard_pattern, chart_pattern, folder_path_pattern, and emit_used_explores_only. These allow you to include/exclude specific dashboards, charts, folders, and whether to ingest only explores used by dashboards/looks. Additional options like skippersonalfolders and view_pattern (for LookML) also control asset selection.

Here’s a step-by-step breakdown of the key Looker configuration options that control which assets are ingested by DataHub:

dashboard_pattern: Allows you to specify regex patterns to include or exclude dashboards by their IDs. Only dashboards matching the allow patterns and not matching any deny patterns will be ingested.

chart_pattern: Similar to dashboardpattern, but for chart IDs. You can filter which charts are ingested using regex patterns. folderpath_pattern: Lets you include or exclude dashboards and looks based on their folder paths in Looker. For example, you can allow only dashboards in "Shared/Sales" or deny those in "Shared/Deprecated". The folder path is constructed by joining the folder hierarchy with slashes.

emit_used_explores_only: When set to true (default), only explores that are referenced by a dashboard or look are ingested. If set to false, all explores are ingested regardless of usage.

skip_personal_folders: If true, dashboards in personal folders are skipped, and only those in shared folders are ingested.

view_pattern (LookML): For LookML ingestion, this option allows you to filter which views are included based on regex patterns.

model_pattern (LookML): Similarly, this allows you to filter which LookML models are included.

These options can be combined to precisely control which Looker assets appear in DataHub. For example, you could ingest only dashboards from a specific folder and only charts with certain IDs, or include all explores regardless of usage by setting emitusedexplores_only: false

6. Verify LookML connection definitions match DataHub platform:

Example for Looker with Snowflake. This can be adapted for other BI tools and data warehouses.

# In your LookML connection file
connection: "production_snowflake" {
  host: "account.snowflakecomputing.com"
  # DataHub will map this to platform URNs
}

# Ensure platform_mapping in recipe
source:
  config:
    connection_to_platform_map:
      production_snowflake: "snowflake"

Looker ingestion extracts three types of entities: Dashboards (with charts), Explores (with fields), and the lineage connecting them to database tables. Ensure your warehouse metadata is already in DataHub for proper lineage.

• Looker Source
• BI Tool Integration