DataHub

DataHub is an open-source metadata platform for data discovery, observability, and federated governance, originally developed at LinkedIn and now commercially backed by Acryl Data. In this DataHub review, we examine how the platform provides a unified catalog of data assets across the organization, and how it compares to alternatives like OpenMetadata, Atlan, and Collibra.

Overview

DataHub was created at LinkedIn to manage metadata across their massive data ecosystem and was open-sourced in 2020. It's now trusted by over 3,000 organizations according to Acryl Data, the commercial company behind DataHub Cloud. The platform has evolved beyond a traditional data catalog into what Acryl calls "enterprise context management" — providing trusted context for both humans and AI agents.

The platform uses a stream-first architecture built on Kafka for real-time metadata ingestion, with Elasticsearch for search and a graph database for relationship queries. DataHub supports 50+ integrations covering data warehouses (Snowflake, BigQuery, Redshift), databases (PostgreSQL, MySQL, MongoDB), BI tools (Tableau, Looker, Power BI), orchestrators (Airflow, Dagster), and transformation tools (dbt, Spark).

Key Features and Architecture

Metadata Ingestion Framework

DataHub ingests metadata through push-based (real-time events via Kafka) and pull-based (scheduled crawlers) mechanisms. The ingestion framework supports 50+ sources with configurable recipes that define what to extract and how often. Custom sources can be built using the Python SDK.

Data Discovery and Search

Full-text search across all data assets — tables, dashboards, pipelines, ML models, and data products — with faceted filtering by platform, domain, tags, glossary terms, and ownership. DataHub claims users can find data 10x faster compared to manual discovery methods.

Column-Level Lineage

Automatic lineage extraction from SQL queries, dbt models, Airflow DAGs, and Spark jobs traces data flow at the column level. The lineage graph shows upstream sources and downstream consumers, enabling impact analysis before schema changes and root cause analysis when data quality issues arise.

Federated Governance

DataHub supports domain-based governance where different teams own and manage metadata for their data assets independently. Glossary terms, tags, and ownership can be managed at the domain level while maintaining organization-wide consistency through shared policies and standards.

Data Observability

Built-in data quality monitoring tracks freshness, volume, schema changes, and custom assertions. Anomaly detection alerts teams when data deviates from expected patterns. This reduces the need for a separate observability tool like Monte Carlo for basic monitoring use cases.

DataHub Cloud (Acryl Data)

The managed SaaS version eliminates infrastructure management and adds enterprise features: SSO/SAML, advanced RBAC, SLA monitoring, and premium support. DataHub Cloud is positioned as the enterprise-ready version for organizations that don't want to self-host.

Ideal Use Cases

Large Data Teams Needing a Central Catalog

Organizations with hundreds of tables across multiple warehouses, databases, and BI tools use DataHub as the single source of truth for data discovery. The 50+ connectors mean most existing infrastructure can be indexed without custom development.

Organizations Building Data Mesh Architectures

Teams implementing data mesh principles use DataHub's domain-based governance to enable federated ownership while maintaining discoverability across the organization. Data products can be published, discovered, and consumed through the catalog.

AI/ML Teams Needing Context for Agents

With the rise of agentic AI, DataHub's "context management" positioning addresses the need for AI agents to discover and understand data assets programmatically. The API-first architecture enables agents to query metadata, lineage, and quality information.

Pricing and Licensing

DataHub open-source is free under the Apache 2.0 license. Acryl Data offers DataHub Cloud as a managed service:

Self-hosted infrastructure typically requires Kafka, Elasticsearch, MySQL/PostgreSQL, and the DataHub services — running $300–$800/month on AWS for a mid-sized deployment. For comparison, OpenMetadata's simpler 4-component architecture costs $200–$500/month to self-host, while commercial catalogs like Atlan start at ~$50,000/year and Collibra at ~$150,000/year.

Pros and Cons

Pros

• Open-source (Apache 2.0) — full feature set available for free self-hosted deployment with no open-core restrictions
• Stream-first architecture — Kafka-based real-time metadata ingestion enables near-instant catalog updates as data changes
• 50+ integrations — covers Snowflake, BigQuery, Redshift, Tableau, Looker, Airflow, dbt, and most major data tools
• Strong lineage — column-level lineage extraction from SQL, dbt, Airflow, and Spark with visual graph exploration
• 3,000+ organizations — large community and proven adoption across enterprises, with LinkedIn-scale battle testing
• Flexible metadata model — extensible entity types and aspects allow customization beyond what rigid catalog schemas support

Cons

• Complex infrastructure — requires Kafka, Elasticsearch, MySQL/PostgreSQL, and multiple DataHub services; more components than OpenMetadata's 4-component architecture
• Steeper learning curve — the flexible metadata model and GMS architecture require more upfront investment to understand and configure
• No dedicated support on open-source — community Slack is active but not guaranteed; enterprise support requires DataHub Cloud
• UI less polished than commercial alternatives — the open-source UI is functional but not as refined as Atlan or Collibra's interfaces
• Resource-intensive — Kafka and Elasticsearch add significant memory and compute requirements to the deployment

Alternatives and How It Compares

OpenMetadata

OpenMetadata is DataHub's closest open-source competitor with a simpler 4-component architecture that's easier to deploy and operate. OpenMetadata has built-in data quality testing (DataHub requires external tools for this), while DataHub has a more flexible metadata model and stream-first architecture. OpenMetadata has fewer integrations (100+ vs DataHub's 50+, though overlap is significant). Choose OpenMetadata for simplicity, DataHub for flexibility.

Atlan

Atlan (~$50,000+/year) is a commercial data catalog with a polished UI, embedded collaboration, and strong Slack integration. Atlan provides dedicated support and faster onboarding compared to self-hosted DataHub. Organizations with budget choose Atlan for the managed experience; cost-conscious teams choose DataHub for the open-source flexibility.

Collibra

Collibra ($150,000–$500,000+/year) is the enterprise market leader in data governance with deep policy management, stewardship workflows, and regulatory compliance features. Collibra is significantly more expensive and complex than DataHub but offers mature enterprise governance capabilities that DataHub is still developing.

Amundsen (by LF AI)

Amundsen is another open-source data catalog, originally from Lyft. It's simpler than DataHub but has a smaller community and fewer features. Development has slowed compared to DataHub's active pace. Most organizations evaluating open-source catalogs now choose between DataHub and OpenMetadata rather than Amundsen.