Structured Properties Validation and Type Enforcement

Structured properties are not enforcing type validation, allowing invalid values, or validation rules are not working as expected. Understanding how to properly configure structured property definitions and validation is key to maintaining data quality.

You Might Be Asking:

• How do I enforce specific values for a property?
• Can I make structured properties required?
• How do I validate property values?

1. Create structured property with validation:

mutation createStructuredProperty {
  createStructuredProperty(
    input: {
      id: "io.company.dataRetentionPeriod"
      qualifiedName: "Data Retention Period"
      displayName: "Retention Period"
      description: "How long data must be retained"

      # Type enforcement
      valueType: "STRING"

      # Restrict to specific values
      allowedValues: [
        {value: "30_days", description: "30 days"}
        {value: "90_days", description: "90 days"}
        {value: "1_year", description: "1 year"}
        {value: "7_years", description: "7 years"}
        {value: "indefinite", description: "No expiration"}
      ]

      # Apply to specific entity types
      entityTypes: ["dataset", "chart", "dashboard"]

      # Cardinality
      cardinality: "SINGLE"  # or "MULTIPLE"
    }
  }
}

2. Create number-type property with range validation:

mutation createNumericProperty {
  createStructuredProperty(
    input: {
      id: "io.company.dataQualityScore"
      qualifiedName: "Data Quality Score"
      valueType: "NUMBER"
      description: "Quality score from 0-100"

      # While GraphQL doesn't directly support min/max in this mutation,
      # validation can be enforced in the backend

      entityTypes: ["dataset"]
      cardinality: "SINGLE"
    }
  )
}

3. Apply structured property with validation:

from datahub.emitter.mce_builder import make_dataset_urn
from datahub.metadata.schema_classes import StructuredPropertiesClass
from datahub.emitter.rest_emitter import DatahubRestEmitter

emitter = DatahubRestEmitter("http://localhost:8080")

# This will fail if value not in allowedValues
try:
    structured_props = StructuredPropertiesClass(
        properties=[
            {
                "propertyUrn": "urn:li:structuredProperty:io.company.dataRetentionPeriod",
                "values": ["7_years"]  # Must match an allowed value
            }
        ]
    )

    emitter.emit_mcp(
        entity_urn=make_dataset_urn("snowflake", "db.schema.table"),
        aspect_name="structuredProperties",
        aspect=structured_props
    )
except Exception as e:
    print(f"Validation error: {e}")

4. Create multi-value property:

mutation createMultiValueProperty {
  createStructuredProperty(
    input: {
      id: "io.company.dataCategories"
      qualifiedName: "Data Categories"
      valueType: "STRING"
      cardinality: "MULTIPLE"  # Allow multiple values

      allowedValues: [
        {value: "PII", description: "Personal Information"}
        {value: "Financial", description: "Financial Data"}
        {value: "Health", description: "Health Records"}
        {value: "Anonymous", description: "Anonymous Data"}
      ]

      entityTypes: ["dataset"]
    }
  )
}

# Apply multiple values
properties: [
  {
    propertyUrn: "urn:li:structuredProperty:io.company.dataCategories",
    values: ["PII", "Financial"]  # Multiple values allowed
  }
]

5. Query structured properties with validation:

query {
  structuredProperty(urn: "urn:li:structuredProperty:io.company.dataRetentionPeriod") {
    qualifiedName
    valueType
    cardinality
    allowedValues {
      value
      description
    }
    entityTypes
  }
}

Structured Properties provide strong typing and validation compared to custom properties. Use them for critical metadata that requires consistency and validation. Consider creating a governance process for creating and managing structured property definitions.

• Structured Properties
• Property Types