iceberg — Apache Iceberg is a high-performance open table format for huge analytic datasets, bringing SQL-like reliability and simplicity to big data environments.

Apache Iceberg is a high-performance open table format for huge analytic datasets, bringing SQL-like reliability and simplicity to big data environments.

About this project

Apache Iceberg is a high-performance table format designed for large-scale analytic tables. It enables multiple processing engines—including Apache Spark, Trino, Apache Flink, Presto, Apache Hive, and Impala—to safely and concurrently work with the same tables. The project provides a stable format specification and a reference implementation in Java. The Java library is organized into several modules: - Core API and Implementation: `iceberg-api` and `iceberg-core` provide the public API and core logic, including support for Avro data files. - File Format Support: Optional modules for Parquet (`iceberg-parquet`), ORC (`iceberg-orc`), and Arrow (`iceberg-arrow`). - Integration Modules: Specific implementations for Spark (`iceberg-spark`), Flink (`iceberg-flink`), and Hive (`iceberg-mr`), as well as a Hive metastore implementation (`iceberg-hive-metastore`). - Application Support: `iceberg-data` allows JVM applications to work with tables directly. Beyond the Java implementation, the Iceberg ecosystem includes implementations in Go, Python (PyIceberg), Rust, and C++.