Apache Iceberg is an open table format for very large analytic datasets sitting on object storage. It adds a metadata layer above the data files — snapshots, schema evolution, hidden partitioning and ACID commits — so many engines can read and write the same tables safely and consistently. Along with Delta Lake and Hudi it is one of the three open table formats that decoupled analytical storage from the engine querying it.
Apache Iceberg
Apache Iceberg is an open table format for analytic datasets held on object storage. The files underneath are usually Parquet; what Iceberg adds is the metadata that turns a pile of files into a table — a snapshot history, an evolvable schema, partitioning the reader does not have to know about, and atomic commits that let concurrent writers work without corrupting each other.
- Snapshot isolation - Every write produces a new snapshot, so readers see a consistent table and time travel comes free.
- Schema and partition evolution - Columns and partitioning can change without rewriting history or breaking existing queries.
- Engine independence - Spark, Trino, Flink, DuckDB, Snowflake and the cloud warehouses read the same tables, which is the point.
- Hidden partitioning - The layout is a property of the table rather than something every query has to encode by hand.
Iceberg matters to API operations for a reason that is easy to miss: it is the most consequential interoperability event in the data and analytics market of the last five years, and it is not an API standard at all. It is a file and metadata layout. Because it decoupled storage from engine, the warehouses now compete on the query layer rather than on lock-in — which is why the analytical stores in The State of Data & Analytics APIs score best on published contracts while their data plane still travels over drivers rather than HTTP.
Industry: Data Engineering
Also in the data model catalog
This standard shows up on both sides of the house — as a standard applied in API operations here, and as a data model defining what information exists. Same standard, two lenses.
See it as a data model →