About this project

Kedro is an open-source Python framework for building production-ready data engineering and data science pipelines. It applies software engineering practices to help teams create pipelines that are reproducible, maintainable and modular. The project is hosted by the LF AI & Data Foundation and released under the Apache 2.0 license. Main features described in the README: - Project Template: a standard, modifiable project template based on Cookiecutter Data Science. - Data Catalog: lightweight data connectors for saving and loading data across many file formats and file systems, including local and network file systems, cloud object stores and HDFS. It also includes data and model versioning for file-based systems. - Pipeline Abstraction: automatic resolution of dependencies between pure Python functions, with pipeline visualisation via Kedro-Viz. - Coding Standards: test-driven development with pytest, documentation with Sphinx, linting support for ruff, and use of the standard Python logging library. - Flexible Deployment: single-machine or distributed deployment, with additional support for Argo, Prefect, Kubeflow, AWS Batch and Databricks. Installation is available from PyPI (uv pip install kedro) or conda-forge, and the latest unreleased version can be installed from the main branch. The README links to a Get Started guide, a spaceflights tutorial, documentation on working with Jupyter notebooks, and an API reference. The project states it was created to address shortcomings of Jupyter notebooks, one-off scripts and glue code, to improve team collaboration where members have varied software engineering exposure, and to encourage reusable analytics code through modularity and separation of concerns. Kedro supports actively maintained Python versions; support is dropped when a Python version reaches end of life. The separate kedro-datasets package follows the NEP 29 policy and generally drops Python versions earlier. Community resources include Slack, a Linen archive, technical FAQs, an awesome-kedro repository, and a recurring public Kedro Coffee Chat.