Accessing Audit Logs for Recipe Configuration Changes – DataHub Customer Support Portal

Area: API Issues

Sub-Area: Audit and Compliance

Issue

Organizations need to track who made changes to DataHub ingestion recipes for compliance and governance purposes. This includes identifying the actors who created or modified ingestion sources, when these changes occurred, and ideally what specific configurations were changed in the recipe definitions.

You Might Be Asking

How can I see who last modified an ingestion recipe?
Is there a way to track all historical changes to recipe configurations?
Can I get before/after comparisons when recipe settings are updated?
Where are audit events for ingestion sources stored in DataHub?

Solution

DataHub provides several methods to access audit information for ingestion recipe changes, each with different capabilities and limitations:

Option 1: Audit Events API (Recommended for Ongoing Monitoring)

Use the Audit Events API to track recipe changes as they happen going forward:

Configure audit event monitoring using the Audit Events Search API
Query for ingestion source creation and modification events
Set up automated monitoring to capture future changes

curl -X POST '.datahub.com/api/v2/audit-events/search' \
  -H 'Authorization: Bearer ' \
  -H 'Content-Type: application/json' \
  -d '{
    "entityType": "INGESTION_SOURCE",
    "eventType": "UPDATE",
    "timeRange": {
      "start": 1640995200000,
      "end": 1672531200000
    }
  }'

Option 2: GraphQL Timeline Query

Use the GraphQL API to retrieve basic audit information for specific ingestion sources:

Identify the URN of your ingestion source
Query the timeline endpoint for that entity
Extract actor and timestamp information from the results

query GetIngestionSourceTimeline {
  getTimeline(
    urn: "urn:li:dataHubIngestionSource:"
    startTime: 1640995200000
    endTime: 1672531200000
  ) {
    timeline {
      timestamp
      actor
      eventType
    }
  }
}

Option 3: DataHub Cloud Admin Audit Interface

For DataHub Cloud deployments:

Navigate to Settings → Platform → Audit in your DataHub Cloud instance
Filter by entity type or actor to find ingestion source changes
Review timestamps and user information for modifications

Note: This feature may need to be enabled by your DataHub support team.

What Information Is Available

Currently trackable audit information includes:

Actor URN: Who made the change (user identifier)
Timestamp: When the change occurred (milliseconds epoch)
Event Type: Whether it was a creation or update event
Entity URN: Which ingestion source was affected

Additional Notes

Current Limitations: DataHub does not currently provide full historical change tracking for ingestion recipes. The system tracks the most recent modification metadata (who and when) but does not maintain a complete version history with before/after configuration comparisons. Historical audit data from before implementing monitoring may require direct database access through DataHub support.

For Historical Data: If you need audit information for changes that occurred before implementing API monitoring, contact DataHub support for a SQL database export of historical audit stamps.

Future Monitoring: The Audit Events API is the best approach for ongoing compliance tracking as it captures real-time changes to recipe configurations.