DataHub Audit Log Data Retention: What Is Stored, For How Long, and Configuration Options – DataHub Customer Support Portal

```html

Area: Observability Issues

Sub-Area: Audit Log Retention and Compliance

Issue

Organizations subject to data privacy regulations (such as GDPR) or internal legal review often need to understand exactly what personal and behavioral data DataHub retains in its audit and usage event logs, where that data is stored, how long it persists, and whether retention or redaction policies can be configured. This article covers the full lifecycle of audit event data in DataHub Cloud, including usage events stored in OpenSearch and user profile data stored in the relational database, as well as the current options and limitations for configuring retention.

You Might Be Asking

How long does DataHub retain audit log data such as sourceIP, userAgent, timestamp, actorUrn, entityUrn, entityType, and aspectName?
Is the audit log retention period configurable in DataHub Cloud?
Does DataHub delete or anonymize usage event records when an employee is deprovisioned from the identity provider (e.g., Okta)?
What personal data is captured in usage events versus user profile aspects?
Can we configure a deletion or redaction policy on the datahub_usage_event OpenSearch index?
Are usage events forwarded or exported outside of the DataHub Cloud environment?

Solution

1. Usage Events in OpenSearch (`datahub_usage_event` index)

DataHub Cloud stores behavioral and audit events in the datahub_usage_event Elasticsearch/OpenSearch index. The index lifecycle is managed by an Index State Management (ISM) policy with the following two phases:

Active (Rollover) — first 14 days: Writes are accepted. When an index reaches 5 GB, it rolls over and a new index becomes the active write target. All data in the rolled-over index remains fully queryable.
ReadOnly — after 14 days: Writes are blocked, but all data remains fully queryable. There is no configured deletion phase. Under current product defaults, usage event data persists indefinitely.

2. Fields Captured in Usage Events

Each usage event record may contain the following fields:

actorUrn — the URN of the user who performed the action (e.g., urn:li:corpuser:user@example.com when email is used as the username claim)
timestamp
userAgent
browserId — a UUID generated on first page load, stored in a browser cookie with a 1-year expiration; not tied to a user account and not cleared on logout; recorded on every usage event including page views
entityUrn, entityType, aspectName
query — the search query text, which may occasionally contain personal data if a user types a name or email address into the search bar
sourceIP — captured only on a subset of administrative and write events (e.g., user creation, access token creation, ingestion source updates, authentication events). Read-only activity such as page views, searches, and entity navigation does not carry sourceIP.

All of these fields are stored in raw, unmasked form. No hashing or anonymization is applied at write time.

3. User Profile Data in the Relational Database (`corpuser` aspects)

Employee profile data populated via identity provider (IdP) ingestion (e.g., Okta, SCIM) is stored as aspects on corpuser entities in MySQL/PostgreSQL. The relevant aspects include:

corpUserInfo — display name, first/last/full name, email, title, manager, department, country, and any custom IdP attributes
corpUserEditableInfo — email, phone, messaging handle, profile picture, title, about-me text
corpUserStatus — provisioning state and audit timestamps
corpUserCredentials — salt, hashed password, and password reset token (for local-credential users; values are hashed)

4. User Deprovisioning and the 10-Day Soft-Delete Window

When a user is deprovisioned in the IdP, the next ingestion run marks the user as removed in DataHub. This is a soft delete: the user disappears from search results, but all underlying database records are retained for 10 days. After 10 days, a garbage collection (GC) job hard-deletes the entity and all of its associated aspects. There is no intermediate anonymization step — the record either exists in full or is fully deleted.

Important: This deletion applies only to corpuser aspects in the database. Historical usage events in OpenSearch that reference the deleted user are not touched. The actorUrn, and on relevant event types the sourceIP and userAgent, captured on past actions remain in OpenSearch indefinitely under current product defaults.

5. Configuring Retention — Current Options and Limitations

Retention policy for the datahub_usage_event index is currently not configurable by end users through the DataHub Cloud UI or standard settings. The ISM policy is defined at the infrastructure level. The DataHub Garbage Collection (GC) source does not include audit event cleanup. It manages:

Dataset, chart, and dashboard usage statistics
Expired tokens
Data process instances
Execution requests
Soft-deleted entities

If your organization requires a specific retention period or data deletion policy for compliance reasons (e.g., GDPR data minimization or right to erasure), contact DataHub Support to discuss the following options:

Custom ISM deletion phase: Adding a "Delete" state to the OpenSearch ISM policy with a configurable retention window (e.g., delete indices older than 90 days).
Redaction or anonymization: Obfuscating raw personal data values (e.g., actorUrn, sourceIP) in historical events rather than deleting them. Note: this requires custom engineering work and is not available out of the box.
Feature requests: If configurable retention is a hard compliance requirement, log a product feature request with your DataHub account team so it can be prioritized.

6. Data Egress Verification

By default in DataHub Cloud, usage event data is not forwarded outside the DataHub environment. The following integrations are not enabled by default and would need to be explicitly configured to export data externally:

Optional EventBridge audit export hook
ALB access log shipping to external storage (e.g., S3)
Log aggregation sidecars (e.g., Fluentbit, Datadog agent, CloudWatch agent, Sumo Logic)

Application stdout request logs are subject to standard kubelet log rotation and are not durably retained beyond that. Verify your specific deployment configuration with your DataHub account team if you need to confirm the egress posture of your environment.

Summary Table

| Data Type                  | Storage Location        | Retention (Default)          | Configurable? |
|----------------------------|-------------------------|------------------------------|---------------|
| Usage events               | OpenSearch              | Indefinite (no delete phase) | Via Support   |
| corpuser profile aspects   | MySQL/PostgreSQL        | 10 days after soft-delete    | Not currently |
| Application stdout logs    | Kubelet (node-local)    | Standard kubelet rotation    | N/A           |

Additional Notes

If your IdP is configured to use email address as the username claim (a common default), then actorUrn values in usage events will contain employee email addresses in plain text (e.g., urn:li:corpuser:user@example.com). This is an important consideration for GDPR data subject access requests and right-to-erasure requests. Additionally, note that some sourceIP values in usage events may originate from internal system actors (e.g., urn:li:corpuser:__datahub_system) performing automated writes such as IdP-driven user provisioning; these reflect internal cluster IP addresses rather than end-user IPs and do not represent employee personal data. The browserId field functions as a persistent cross-session device identifier and may be considered personal data under certain regulatory frameworks. This article reflects the behavior of DataHub Cloud as of early 2026. Self-hosted deployments may differ depending on infrastructure configuration. Always validate retention behavior against your specific deployment with your DataHub account team before making compliance representations.