Apache Hudi: product and architecture
This Apache Hudi review covers the transactional data lake platform that pioneered incremental processing and streaming upserts on cloud object storage. Hudi (Hadoop Upserts Deletes and Incrementals, pronounced "hoodie") originated at Uber in 2016 to solve a problem that traditional Hive tables couldn't: updating and deleting individual records on HDFS and S3 efficiently. In 2026 Hudi is one of three major open table formats — alongside Apache Iceberg and Delta Lake — with a specific architectural niche around streaming ingestion and CDC workloads. We evaluated it to answer the real question: when does Hudi's streaming-first design justify choosing it over Iceberg or Delta Lake?
Overview
Apache Hudi is an open-source transactional data lake platform under Apache 2.0, sitting in the Data Warehouse category alongside Iceberg and Delta Lake. Hudi brings transactions, record-level upserts, deletes, and incremental reads to Parquet files on object storage (S3, GCS, Azure Blob, HDFS). Unlike Iceberg, which leans analytics-first, Hudi is designed from the ground up for streaming ingestion — it's the format you'd choose if your primary workload is CDC from operational databases or event streams landing in a lakehouse.
The platform matters because it's solved a specific problem that Iceberg and Delta Lake handle less efficiently: continuous row-level writes at high throughput. Uber built Hudi to ingest billions of records per day from their operational systems; the architecture reflects that origin. In 2026 Hudi is used by companies like Uber, Robinhood, and GE for streaming data pipelines where Iceberg's snapshot-per-commit model would create too many small files. Target audience: data engineering teams running CDC pipelines from operational databases, streaming-heavy organizations, and teams using Onehouse (the commercial company founded by Hudi's creators) for managed lakehouse services.
Key Features and Architecture
Hudi's architecture splits writes into two storage modes: Copy-On-Write (CoW) and Merge-On-Read (MoR). CoW rewrites entire Parquet files on update — fast reads, slower writes, similar to Iceberg's default mode. MoR writes delta log files (Avro) that are merged at read time or during background compaction — fast writes, moderate reads, but essential for high-throughput streaming ingestion. This dual-mode design is Hudi's signature feature.
Record-level indexing is another defining capability. Hudi maintains a record-key-to-file mapping (via Bloom filters, HBase, or other indexing backends) so upserts find the right file in milliseconds rather than scanning partitions. For CDC workloads where you're updating specific customer records from Kafka, this indexing makes Hudi dramatically faster than Iceberg's equivalent patterns.
Incremental reads let downstream jobs read only rows that changed since a commit timestamp — exactly-once semantics for streaming ETL. Timeline and time travel are built in: every commit is timestamped, you can query any historical state, and rollback is a single command. Concurrency control uses optimistic concurrency for writers with MVCC snapshot isolation for readers. Hudi Streamer (formerly DeltaStreamer) handles the common "ingest Kafka to Hudi" pattern out of the box, and the Flink + Spark + Presto/Trino + Hive query support covers most analytics engines.
Ideal Use Cases
Best for:
- CDC pipelines from operational databases — Debezium or similar CDC tools plus Hudi is the standard open-source pattern for near-real-time operational data warehousing.
- Streaming ingestion at high throughput — teams writing millions of records per minute from Kafka to a lakehouse. MoR mode handles this where Iceberg's default would struggle.
- Uber-style operational analytics — data platforms where the primary workload is continuous upserts rather than bulk batch writes.
- Teams using Onehouse (managed Hudi service from the creators) — the commercial ecosystem for Hudi is niche but specifically focused on streaming lakehouses.
- Organizations with record-level compliance needs (GDPR right-to-deletion) — Hudi's record-level delete support handles this cleanly.
Not suitable for:
- Analytics-first teams with batch-heavy workloads — Iceberg is simpler and faster for pure analytics query patterns. Hudi's streaming features don't pay off if you're not streaming.
- Teams committed to Databricks — Delta Lake has meaningfully deeper Databricks integration. Hudi works on Databricks but it's not the native path.
- Small teams without streaming expertise — Hudi's configuration surface (CoW vs MoR, indexing backends, concurrency modes) has real learning curve. Iceberg is friendlier for teams learning lakehouses.
- Multi-engine organizations prioritizing ecosystem breadth — Iceberg has expansive native engine support (Snowflake, BigQuery, Databricks all read Iceberg natively; Hudi support is present but often less optimized).
Pros and Cons
Pros:
- Streaming upserts at high throughput — the feature that justifies choosing Hudi over Iceberg for CDC workloads.
- Record-level indexing makes point updates dramatically faster.
- Incremental reads enable exactly-once streaming ETL downstream.
- Apache 2.0 license — no format-level lock-in.
- Hudi Streamer handles common Kafka-to-lakehouse patterns out of the box.
- Record-level deletes handle GDPR compliance cleanly.
Cons:
- Configuration complexity is real — CoW vs MoR, indexing backends, concurrency modes all require careful choices.
- An ecosystem that trails Iceberg or Delta Lake — commercial support is concentrated around Onehouse and cloud-vendor managed services.
- Less optimal for pure analytics — if you're not streaming, Iceberg is simpler and sometimes faster.
- Documentation quality varies — core concepts are well-documented, but advanced patterns sometimes require reading source code or community archives.
- Engine support is present but less mature than Iceberg's — Snowflake, BigQuery, and Databricks support Iceberg better than Hudi.
