Multi-Tenancy and Data Isolation

Organizations running DataHub for multiple teams or customers need to isolate metadata, control access, and separate data between tenants. Understanding multi-tenancy approaches is important for shared deployments.

You Might Be Asking:

• Can DataHub support multiple tenants?
• How do I isolate metadata between teams?
• What's the best way to implement multi-tenancy?

1. Use Domains for logical separation:

# Organize data by business domain/team
# Create domains for each tenant
domains:
  - Team A (Marketing)
  - Team B (Finance)
  - Team C (Engineering)

# Assign assets to domains during ingestion
transformers:
  - type: "pattern_add_dataset_domain"
    config:
      domain_urn: "urn:li:domain:TeamA"
      pattern:
        rules:
          "marketing.*": "urn:li:domain:TeamA"
          "finance.*": "urn:li:domain:TeamB"

2. Implement domain-based access control:

# Create policy restricting cross-domain access
mutation createDomainPolicy {
  createPolicy(
    input: {
      type: METADATA
      name: "Domain Isolation - Team A"
      description: "Team A can only see their domain"
      state: ACTIVE
      resources: {
        filter: {
          criteria: [
            {
              field: "DOMAIN"
              values: ["urn:li:domain:TeamA"]
            }
          ]
        }
      }
      privileges: ["VIEW_DATASET_PROFILE", "EDIT_DATASET"]
      actors: {
        groups: ["urn:li:corpGroup:team-a"]
      }
    }
  )
}

3. Use platform instances for physical separation:

Example for Snowflake multi-tenancy. This approach can be adapted for any data platform.

# Separate metadata by platform instance
# Tenant 1
source:
  type: snowflake
  config:
    platform_instance: "tenant1_snowflake"
    account: "tenant1.snowflakecomputing.com"

# Tenant 2
source:
  type: snowflake
  config:
    platform_instance: "tenant2_snowflake"
    account: "tenant2.snowflakecomputing.com"

# This creates separate URN namespaces:
# urn:li:dataset:(urn:li:dataPlatform:snowflake,tenant1_snowflake.db.table,PROD)
# urn:li:dataset:(urn:li:dataPlatform:snowflake,tenant2_snowflake.db.table,PROD)

4. Implement row-level security equivalent:

# Custom middleware for tenant filtering
def filter_search_results_by_tenant(user, search_results):
    """
    Filter search results based on user's tenant
    """
    user_tenant = get_user_tenant(user)

    filtered_results = []
    for result in search_results:
        # Check if entity belongs to user's tenant
        entity_domain = get_entity_domain(result['urn'])
        entity_platform_instance = get_platform_instance(result['urn'])

        # Allow access if matches tenant
        if (entity_domain == user_tenant or 
            entity_platform_instance == user_tenant):
            filtered_results.append(result)

    return filtered_results

5. Deploy separate DataHub instances:

# For strict isolation, deploy separate DataHub instances
# values-tenant1.yaml
global:
  namespace: datahub-tenant1

datahub:
  database:
    name: datahub_tenant1

elasticsearch:
  indexPrefix: "datahub_tenant1"

---
# values-tenant2.yaml
global:
  namespace: datahub-tenant2

datahub:
  database:
    name: datahub_tenant2

elasticsearch:
    indexPrefix: "datahub_tenant2"

# Deploy with Helm
helm install datahub-tenant1 datahub/datahub -f values-tenant1.yaml
helm install datahub-tenant2 datahub/datahub -f values-tenant2.yaml

6. Best practices for multi-tenancy:

Multi-Tenancy Approaches (in order of isolation):

1. Logical Separation (Softest):
   - Use Domains and Groups
   - RBAC policies for access control
   - Pros: Single deployment, easy management
   - Cons: Shared resources, potential data leakage

2. Platform Instance Separation (Medium):
   - Use platform_instance for namespacing
   - Separate metadata by instance
   - Pros: Clear separation, single deployment
   - Cons: Still shares infrastructure

3. Separate Deployments (Strongest):
   - Deploy separate DataHub instances
   - Completely isolated databases and search indices
   - Pros: Complete isolation, independent scaling
   - Cons: Higher operational overhead

Choose based on:
- Compliance requirements
- Performance needs
- Operational complexity
- Cost constraints

True multi-tenancy with complete data isolation requires careful architecture. Consider compliance requirements (GDPR, HIPAA) when choosing approach. Domains provide logical separation but share underlying infrastructure. Separate deployments provide strongest isolation but highest operational overhead.

• Domains Guide
• Access Policies