Buying snapshot
Scroll sideways to see all columns. With a keyboard, focus this table and use the arrow keys.
| Decision point | What the project states |
|---|---|
License | Apache-2.0. |
Installation | Build from source on a Unix-like system (like Linux or Mac OS X) with Java 11 or 17, Git and Maven 3.6.0 or later, then start spark-shell with the built Spark bundle jar. The default build targets Spark 3.5.3 with Scala 2.12 and Flink 2.2. |
How it runs | Inside your Apache Spark or Apache Flink jobs, which get built-in ingestion tools. Table services run inside the Spark/Flink writers or are operated independently. |
Storage | All data and metadata stay on cloud storage in open formats. Listed lake storage includes Amazon S3, Google Cloud Storage, Azure Blob Storage, Apache Hadoop, Alibaba Cloud, IBM Cloud, Oracle Cloud, Tencent Cloud and MinIO. |
Engine versions | Spark bundles for Spark 3.3.2+ up to 4.2; Flink bundles for Flink 1.18 up to 2.2. |
Query engines and catalogs | Listed integrations include Presto, Trino, Apache Hive, AWS Athena, Apache Doris, StarRocks, Apache Impala, Amazon Redshift and ClickHouse. Catalog sync covers Apache Hive Metastore, AWS Glue, Google BigQuery, Apache XTable and more. |
Processing platforms | Listed under data processing: Apache Spark, Apache Flink, Databricks, AWS EMR, Azure HDInsight, Onehouse, Ray and Daft. dbt and Apache Airflow are listed for orchestration. |
Support | Community help through Slack, a mailing list and GitHub. |
Our take
Choose Hudi if you already run Spark or Flink and your lake needs frequent record-level updates and deletes, such as database CDC or streaming data. If your tables are mostly written in batches, compare it with Apache Iceberg or Delta Lake before you commit.
Evaluate Apache Hudi
Comparisons
Where it stands out
Record-level updates and deletes. Upserts and deletes use record-level indexes built on row-oriented file formats and bloom filters, and writes keep the indexes up to date. Hudi says this covers database CDC and high-scale streaming data, including out-of-order records, bursty traffic and data deduplication.
Read only what changed. Incremental queries return records inserted or updated since a chosen point in time. Change data capture (CDC) queries also return deletes, with before and after images for each change, so downstream jobs do not have to rescan the table.
Concurrency for streaming writers. Non-blocking concurrency control handles out-of-order and late data. Optimistic concurrency control covers Read-Modify-Write style writes, with snapshot isolation between the writer and queries.
Ingestion included. Built-in ingestion tools and a Kafka Connect sink. Listed sources include Apache Kafka, Apache Pulsar, PostgreSQL, MySQL, Debezium and Apache Flink CDC.
History and recovery. Time-travel queries, savepoints for versioning and recovery, and atomic commits with rollback/restore.
Table maintenance built in. Table services schedule clustering, compaction, cleaning, file sizing and indexing, with configurable scheduling strategies and built-in failure handling.
Product screenshot
Trade-offs to weigh
Columnar read speed depends on compaction. Hudi stores row and columnar data. Read-optimized queries give purely columnar (for example Parquet) performance only when used with a compaction policy, and streaming writers rely on asynchronous compaction from row-oriented to columnar formats. Someone has to choose and watch those policies.
Bundles are tied to engine versions. Spark 4.0, 4.1 and 4.2 bundles need Java 17. Only the Spark-related bundles build with Scala 2.13; the Flink bundle cannot. Engine upgrades mean a matching Hudi bundle.
Compression risk on older Spark. Hudi uses ZSTD as the default Parquet codec with Spark 3.5 and newer. Spark 3.3 and 3.4 keep GZIP because reading ZSTD files there can leak off-heap memory (PARQUET-2160), and the risk remains when an older Spark runtime reads ZSTD files written by another engine. Hudi advises upgrading to Spark 3.5 or newer first.
Public signals
About these signals
Verified factual signals from public sources. They indicate observable activity or interest, not total adoption, product quality, or cost.
402 GitHub commits 90d6.3k GitHub stars0 vulnerabilities across 1 package
See all signals from 6 sources
Three-month score against stable baseline terms—not search volume or adoption.
PyPI · hudi@0.5.0
Alternatives
Apache Iceberg. The Hudi website compares the two for streaming ingestion, for CDC workloads and on benchmarks, and has a guide to migrating from Iceberg to Hudi. These are the Hudi project's own articles, so test both formats on your own workload.
Delta Lake. The Hudi website compares the two for write-heavy workloads and has a guide to migrating from Delta Lake to Hudi. Databricks is also in Hudi's integration list.
Apache Hive tables or plain Parquet. Hudi publishes guides to migrating from both.
Alternatives to Apache Hudi
The reviewed substitutes for Apache Hudi among the open table formats, and what would make each one the better answer.
Direct alternatives
Reviewed substitutes: products bought for the same job, where a team picks one.
- Delta Lake
- Apache Hudi and Delta Lake are open table formats that add ACID transactions and time travel to files on a data lake. Hudi leans toward streaming ingestion, with upserts, incremental processing and record-level indexes. Delta Lake started at Databricks and is the native format there, while engines such as Spark, Trino and Flink can also read it.Applies to: Choosing between two products of the same kind for one job.
- Apache Iceberg
- Apache Iceberg and Apache Hudi each turn files on cloud storage into reliable tables with transactions and schema changes. Iceberg puts the weight on engine-neutral tables that Spark, Trino, Flink and Snowflake can all read and write, with partition evolution. Hudi puts the weight on fast upserts and incremental processing for streaming pipelines.Applies to: Choosing between two products of the same kind for one job.
Other approaches
A different approach to the same problem. Each substitutes only for the workload named beside it.
- Amazon Redshift
- Choose Redshift when you want managed cluster-based SQL with streaming ingest; Hudi plus EMR wins on cost at scale.