About this project

Ibis is an open-source Python dataframe library designed for portability across execution engines. Rather than being tied to one engine, it provides a common Python API for data manipulation and compiles expressions into the backend's native language, typically SQL. Key capabilities described in the README: - Fast local dataframes, using DuckDB by default. - Lazy dataframe expressions, plus an interactive mode for iterative exploration. - Mixing Python dataframe code with SQL: expressions can be compiled to SQL via ibis.to_sql, and SQL can be embedded in a pipeline with t.sql(...). - One API across more than 20 backends, including Apache DataFusion, Apache Druid, Apache Flink, Apache Impala, Apache PySpark, Athena, BigQuery, ClickHouse, Databricks, DuckDB, Exasol, Materialize, MySQL/MariaDB, Oracle, Polars, PostgreSQL, RisingWave, SingleStoreDB, SQL Server, SQLite, Snowflake and Trino. - Portability: switching backends via ibis.set_backend or by creating a connection object (for example ibis.duckdb.connect, ibis.polars.connect, ibis.datafusion.connect), then listing and reading tables or files such as CSV and Apache Parquet. - Two broad backend categories: SQL-generating backends and dataframe-generating backends. Installation is via pip, for example pip install 'ibis-framework[duckdb,examples]'. The project is Apache-licensed, independently governed, and welcomes community contributions through GitHub and Zulip. Documentation, tutorials and installation guides are hosted at ibis-project.org.