Best Practices for AI-Ready Documentation in DataHub: AI Doc Generation and Context Documents – DataHub Customer Support Portal

Area: Best Practices

Sub-Area: AI-Powered Documentation and Knowledge Management

Issue

Organizations adopting AI-powered features in DataHub — such as Data Concierge, knowledge retrieval, and agentic workflows — often need guidance on how to structure and maintain documentation so that it is optimally accessible to AI agents and end users. Without a clear framework, teams may produce inconsistent, incomplete, or AI-inaccessible documentation that limits the effectiveness of these features.

You Might Be Asking

How should we structure documentation in DataHub to support AI use cases like Data Concierge or agentic workflows?
What is the recommended approach for asset-level versus high-level process documentation?
How does DataHub's AI Documentation Generation feature work, and how do we customize it for our organization?
What are Context Documents, and how do they differ from standard asset descriptions?
Is AI-generated documentation kept private within our environment, or is it shared with third-party LLMs?

Solution

DataHub provides two complementary features for building AI-ready documentation: AI Documentation Generation for asset- and column-level metadata, and Context Documents for broader, unstructured knowledge. Using both in combination represents the current best practice for organizations looking to maximize the effectiveness of AI-powered features in DataHub Cloud.

Use AI Documentation Generation for Asset- and Column-Level Docs
AI Documentation Generation automatically creates comprehensive, context-aware documentation for tables and columns by analyzing schema, lineage, sample values, and related metadata. To enable and configure this feature:
- Ensure the appropriate permissions are granted to your user or service account in DataHub.
- Navigate to the asset you want to document and trigger AI Documentation Generation from the asset's documentation panel.
- Customize the generation with organization-specific instructions to enforce internal naming conventions, terminology, or style standards. For example:
```
Organization Instructions:
- Always describe columns in business-friendly language, avoiding technical jargon.
- Use the format: "This field represents [business concept] and is used for [purpose]."
- Flag columns containing PII with the note: "Contains personally identifiable information."
```
- All AI-generated documentation is flagged for human review before it is published. Ensure your team has a review workflow in place to validate and approve generated content.
Privacy note: AI Documentation Generation uses AWS Bedrock models and operates entirely within the DataHub Cloud environment. No customer data or metadata is shared with third-party LLMs.
Use Context Documents for High-Level and Process Documentation
Context Documents are designed to capture broader knowledge that does not fit neatly into asset-level descriptions — such as runbooks, FAQs, data domain overviews, architectural summaries, or onboarding guides. These documents are indexed for semantic search and made accessible to AI agents and users via Ask DataHub.
- Create Context Documents natively within DataHub, or ingest them from external knowledge sources such as Notion or Confluence.
- Link Context Documents to relevant data assets so that AI agents can surface them in response to related queries.
- Structure Context Documents clearly with headings, numbered lists, and concise summaries to improve semantic retrieval accuracy. Example structure:
```
# [Domain or Process Name] Overview

## Purpose
Describe the business purpose of this domain or process.

## Key Datasets
List the primary datasets involved and their roles.

## Common Use Cases
Enumerate the top use cases for this domain.

## Known Limitations
Document any caveats, data quality issues, or known gaps.

## Contact and Ownership
Identify the responsible team or point of contact (use role titles, not personal names).
```
Establish a Regular Review and Maintenance Cadence
AI-ready documentation is only as useful as it is accurate and current. Implement the following practices:
- Schedule periodic reviews of both AI-generated asset documentation and Context Documents — at minimum quarterly, or whenever significant schema or process changes occur.
- Assign clear documentation ownership at the domain or team level so that review responsibilities are unambiguous.
- Use DataHub's metadata change notifications or lineage alerts to trigger documentation review when upstream assets change.

Summary: Recommended Documentation Layers

The following table summarizes which tool to use for each documentation type:

| Documentation Type                        | Recommended Tool                  |
|-------------------------------------------|-----------------------------------|
| Table and column descriptions             | AI Documentation Generation       |
| Data domain or subject area overviews     | Context Documents                 |
| Runbooks and operational procedures       | Context Documents                 |
| FAQs and tribal knowledge                 | Context Documents                 |
| Architectural or lineage explanations     | Context Documents                 |
| Glossary terms and business definitions   | DataHub Business Glossary         |

Additional Notes

Both AI Documentation Generation and Context Documents are currently in public beta as of the time of writing. Feature availability and behavior may evolve. AI Documentation Generation is available to DataHub Cloud customers and requires appropriate permissions to be configured by an administrator. Context Document ingestion from external sources (Notion, Confluence) depends on the relevant source connectors being configured in your environment. Always verify feature availability with your DataHub account team or by reviewing the latest release notes.

Issue

You Might Be Asking

Solution

Additional Notes

Related Documentation