Duplicate User Accounts and Group Membership Discrepancies from Mixed LDAP and SSO Provisioning – DataHub Customer Support Portal

Area: Deployment Issues

Sub-Area: User Identity and Group Membership — Mixed Provisioning Sources

Issue

When an organization uses both an LDAP-based ingestion source (such as an Active Directory sync) and an SSO/SCIM-based identity provider (such as Okta) to provision users and groups in DataHub simultaneously, duplicate user entities are created under different URNs. The LDAP ingestion typically generates URNs keyed by a short username (e.g., urn:li:corpuser:username), while SSO login generates URNs keyed by email address (e.g., urn:li:corpuser:username@example.com). Because group memberships, roles, and permissions are stored against a specific URN, the account a user actually logs in with may carry no group memberships and therefore inherit no roles — even though the correct memberships and roles exist on the other duplicate account. This results in users being unexpectedly blocked from editing, permissions appearing inconsistent between the UI and GraphQL queries, and group membership counts that differ depending on which relationship type is queried.

Error Messages

Something went wrong. (UI error when a user attempts to view or edit an entity they should have access to)
User shows as member of a group via IsMemberOfGroup query but cannot exercise the role that group grants

You Might Be Asking

Why does the UI show a different group member count than my GraphQL query returns?
Why does a user appear as a member of a group but still cannot edit entities in DataHub?
Why does querying a user's group memberships return fewer groups than expected, or not return a group at all?
Why do some users have two separate accounts in DataHub?
How do I consolidate duplicate user URNs caused by LDAP and SSO provisioning?

Solution

Step 1 — Understand Why Member Counts Differ Between the UI and GraphQL

DataHub tracks group membership using two distinct relationship types:

IsMemberOfGroup — created when membership is ingested from an external source (e.g., LDAP/AD ingestion). This is what a standard GraphQL query using types: ["IsMemberOfGroup"] returns.
IsMemberOfNativeGroup — created when membership is added manually inside DataHub via the UI.

The group members page in the DataHub UI queries both relationship types together, so the count it shows is the union of both sets. A GraphQL query that specifies only IsMemberOfGroup will return fewer results. To get a full count programmatically, query both relationship types or use count: 500 and verify the total:

query getGroupMembers {
  corpGroup(urn: "urn:li:corpGroup:<your-group-name>") {
    urn
    properties {
      displayName
    }
    relationships(
      input: {
        types: ["IsMemberOfGroup", "IsMemberOfNativeGroup"]
        direction: INCOMING
        start: 0
        count: 500
      }
    ) {
      total
      relationships {
        entity {
          ... on CorpUser {
            urn
            username
            properties {
              displayName
              email
            }
          }
        }
      }
    }
  }
}

Important: Native memberships (added via the UI) are not managed by your external ingestion source. If a user is later removed from the AD group, the ingestion pipeline will drop their IsMemberOfGroup relationship, but the IsMemberOfNativeGroup relationship will remain, and they will continue to inherit any roles the group grants. If your external directory is intended to be the single source of truth for access, remove all native group memberships and let ingestion manage the group exclusively.

Step 2 — Understand Why a User's Groups Tab Shows Fewer Groups Than Expected

The DataHub UI groups tab on a user's profile page historically filtered out groups that did not exist as full group entities in the database, even if the user had valid membership references to those groups. This could suppress pagination and cause groups beyond the first 20 to not be shown. This is a known UI display behavior and does not affect actual permissions. A fix was implemented to remove this filter so all groups are shown regardless of whether a full group entity exists.

Additionally, GraphQL relationship queries default to returning only the first 100 results if no count parameter is supplied. Always pass an explicit count when querying users with large group memberships:

query GetUserGroupMemberships(
  $userUrn: String!
  $start: Int = 0
  $count: Int = 500
) {
  corpUser(urn: $userUrn) {
    urn
    username
    relationships(
      input: {
        types: ["IsMemberOfGroup"]
        direction: OUTGOING
        start: $start
        count: $count
      }
    ) {
      start
      count
      total
      relationships {
        type
        entity {
          urn
          type
          ... on CorpGroup {
            name
            properties {
              displayName
            }
          }
        }
      }
    }
  }
}

Step 3 — Identify Duplicate User URNs

When LDAP and SSO provisioning are both active, a single person typically ends up with two DataHub user entities:

urn:li:corpuser:<shortUsername> — created by LDAP ingestion; carries all group memberships and roles
urn:li:corpuser:<shortUsername@example.com> — created when the user logs in via SSO with their email address; carries no group memberships

Because DataHub's authorization engine reads the membership aspects stored on whichever URN is the active login session, if the user logs in via SSO and lands on the email-keyed URN, they will inherit no roles — even though all the correct group memberships and roles exist on the other URN. To confirm which URN a user is actually logging in with, check the account with recent product activity recorded against it. The account with no product activity is the one not being used for login.

Use the following query to inspect a specific user URN's group memberships:

query GetUserGroupMemberships {
  corpUser(urn: "urn:li:corpuser:<username>") {
    urn
    username
    properties {
      displayName
      fullName
      email
    }
    relationships(
      input: {
        types: ["IsMemberOfGroup"]
        direction: OUTGOING
        start: 0
        count: 500
      }
    ) {
      total
      relationships {
        type
        entity {
          urn
          type
          ... on CorpGroup {
            name
            properties {
              displayName
            }
          }
        }
      }
    }
  }
}

Step 4 — Immediate Remediation for Affected Users

If users are immediately blocked from editing and you need to restore access while the root cause is addressed:

Assign the required role (e.g., EDITOR) directly to the login URN (the email-keyed account) via the DataHub UI or the assignRole GraphQL mutation.
Ensure the user's account status is set to active. Inactive accounts will not receive permissions even if roles are correctly assigned. Contact DataHub Cloud support to set account active status directly if needed.
Have the affected user fully clear browser cookies, log out, and log back in to pick up the updated session and permissions.

As an interim measure, you can also use the addGroupMembers GraphQL mutation to manually add a user to a native group, which directly writes to the NativeGroupMembership aspect on the user entity and takes effect without a service restart:

mutation addGroupMember {
  addGroupMembers(input: {
    groupUrn: "urn:li:corpGroup:<your-group-name>",
    userUrns: ["urn:li:corpuser:<login-username>"]
  })
}

Step 5 — Long-Term Remediation: Consolidate to a Single Provisioning Source

The root cause of all of the above symptoms is running two identity provisioning systems concurrently. The recommended long-term resolution is to consolidate to a single source of truth. The preferred approach for organizations using Okta (or another OIDC provider) for login is:

Set up SCIM provisioning from your identity provider (e.g., Okta) to DataHub. SCIM performs a bidirectional, keyed sync that ensures user and group URNs match the login identity — eliminating the mismatch between LDAP-keyed and SSO-keyed URNs. Configure and verify SCIM is syncing users and groups correctly before proceeding.
Disable the LDAP ingestion source once SCIM is confirmed working. Do not remove the LDAP source and SCIM simultaneously; complete SCIM verification first so no one loses access during the transition.
Clean up duplicate accounts and groups created by the LDAP ingestion. This includes the large volume of users and groups that were ingested from the full directory but are not actively used in DataHub. DataHub Cloud support can assist with bulk cleanup. Coordinate this step after SCIM is operational.

The SCIM integration syncs only the users and groups that are actually assigned to the DataHub application in your identity provider, avoiding the problem of ingesting your entire directory (potentially tens of thousands of users and thousands of groups) when only a subset are active DataHub users.

Step 6 — Verify Role Inheritance Is Working After Consolidation

After consolidation, confirm that role inheritance is functioning correctly by checking that the login URN for each user carries the correct group memberships via the GraphQL query in Step 3. Role inheritance via group membership reads the GroupMembership and NativeGroupMembership aspects stored on the user entity for the active login session. The UI group member count and the graph relationships are informational and do not directly drive authorization decisions.

Additional Notes

The UI display bug that filtered out groups not present as full entities in the database (causing the groups tab to show fewer groups than expected and suppressing pagination) was fixed in a subsequent DataHub release. If you are on an older version, upgrading will resolve the display issue, but the underlying duplicate URN problem requires the provisioning consolidation steps above.
Native group memberships (IsMemberOfNativeGroup) added via the UI are not managed by external ingestion sources. They will persist even if the user is removed from the corresponding AD group. If your AD/LDAP sync is the intended source of truth, audit and remove native memberships to prevent stale access.
When querying relationships via GraphQL, always specify a count parameter (e.g., count: 500). The default page size is 100, and users with many group memberships will appear to be missing groups if you do not paginate correctly.
The SCIM provisioning approach is strongly recommended over LDAP ingestion for organizations that use an OIDC-based SSO provider, as it ensures URN consistency between provisioning and login identity.
If you prefer to keep an LDAP-derived approach for enriching user metadata (e.g., custom properties, team assignments), consider a csv-enricher ingestion source that reads from a CSV exported from your LDAP system, rather than running a full LDAP ingestion source that provisions users and groups.