Apache Iceberg is an open table format for very large analytic datasets sitting on object storage. It adds a metadata layer above the data files — snapshots, schema evolution, hidden partitioning and ACID commits — so many engines can read and write the same tables safely and consistently. Along with Delta Lake and Hudi it is one of the three open table formats that decoupled analytical storage from the engine querying it.
Apache Iceberg
Apache Iceberg is an open table format for analytic datasets held on object storage. The files underneath are usually Parquet; what Iceberg adds is the metadata that turns a pile of files into a table — a snapshot history, an evolvable schema, partitioning the reader does not have to know about, and atomic commits that let concurrent writers work without corrupting each other.
- Snapshot isolation - Every write produces a new snapshot, so readers see a consistent table and time travel comes free.
- Schema and partition evolution - Columns and partitioning can change without rewriting history or breaking existing queries.
- Engine independence - Spark, Trino, Flink, DuckDB, Snowflake and the cloud warehouses read the same tables, which is the point.
- Hidden partitioning - The layout is a property of the table rather than something every query has to encode by hand.
Iceberg matters to API operations for a reason that is easy to miss: it is the most consequential interoperability event in the data and analytics market of the last five years, and it is not an API standard at all. It is a file and metadata layout. Because it decoupled storage from engine, the warehouses now compete on the query layer rather than on lock-in — which is why the analytical stores in The State of Data & Analytics APIs score best on published contracts while their data plane still travels over drivers rather than HTTP.
Referenced in API Evangelist papers
This standard shows up in my published research. These reports read the machine-readable evidence provider by provider — and put this standard in the context of a real sector.
The State of Data & Analytics APIs
The open table formats settled how analytical data sits on object storage — the most consequential interoperability event in this market, and a file layout rather than an API standard.
