Apache Hudi is an open table format built around incremental processing — upserts, deletes and change streams over data lake tables. Where the other open table formats optimise for snapshot consistency, Hudi’s distinguishing feature is treating a table as a stream of changes that downstream consumers can subscribe to.
Apache Hudi
Apache Hudi brought record-level upserts and deletes to the data lake, and with them an incremental query model: instead of rescanning a table, a consumer asks what changed since the last commit. That makes it the most stream-shaped of the three open table formats.
- Upserts and deletes - Record-level mutation on object storage, which is what GDPR erasure and CDC ingestion both require.
- Incremental queries - Consumers pull only the changes since a commit, rather than re-reading the table.
- Copy-on-write and merge-on-read - Two storage layouts trading write cost against read cost, chosen per table.
- Built-in table services - Compaction, clustering and cleaning run as managed operations rather than bespoke jobs.
Hudi’s change-stream model is the closest the analytical storage layer gets to the event surface that The State of Data & Analytics APIs found missing from the market’s published contracts: the table knows what changed, and almost nobody describes that to an API consumer.
Referenced in API Evangelist papers
This standard shows up in my published research. These reports read the machine-readable evidence provider by provider — and put this standard in the context of a real sector.
The State of Data & Analytics APIs
The change-stream table format, in a market where 11.8% of the leaders describe an event surface at all.
