An open table format defines how large analytic datasets are organized as tables on top of files in object storage, adding a metadata layer that tracks schema, partitions, and snapshots so that engines can read and write the same data reliably with ACID guarantees.
Table Format
Table Format, in the modern data stack, refers to an open table format — a metadata layer that turns a pile of files in object storage into a real, queryable table. Formats like Apache Iceberg, Delta Lake, and Apache Hudi sit above physical file formats such as Parquet and describe schema, partitioning, and point-in-time snapshots, so that many engines can read and write the same dataset without stepping on each other. It is the difference between “a folder of files” and “a table with transactions.”
- Metadata over files - Tracks which files belong to a table at each version, layered on top of columnar file formats like Parquet and Arrow.
- ACID and time travel - Snapshots give atomic commits, safe concurrent writes, and the ability to query data as of a past version.
- Schema and partition evolution - Columns and partitioning can change over time without rewriting the whole dataset.
- Engine-agnostic - A single table can be read by Spark, Trino, Flink, and others, which is what makes it a genuine interoperability standard rather than a vendor feature.
In real data and API operations, the table format is the contract that lets a lakehouse serve both analytics and the APIs built on top of it without duplicating data or forking pipelines. Choosing a format like Iceberg or Delta is increasingly a governance decision — it determines who can safely write, how versions are audited, and how downstream services and agents get consistent reads. As data products get exposed through APIs, the open table format underneath is what keeps those products reproducible and trustworthy.
