metaflow

Metaflow is an open-source, human-centric Python framework for building and managing real-life AI and ML systems. Originally developed at Netflix and now supported by Outerbounds, it aims to streamline the entire development lifecycle, from rapid prototyping in notebooks to reliable production deployments. Key capabilities described in the README: - A simple, friendly Pythonic API covering foundational AI/ML needs. - Rapid local prototyping, notebook support, and built-in experiment tracking, versioning and visualization. - Horizontal and vertical scaling in the cloud using CPUs and GPUs, with fast data access for massively parallel and gang-scheduled workloads, plus reliability and checkpointing features. - Dependency management and one-click deployment to highly available production orchestrators, with support for reactive, event-triggered orchestration. Installation is available via PyPI (`pip install metaflow`) or conda-forge. The project provides tutorials, an API reference, release notes, and a Slack community for support. The README notes adoption across companies including Amazon, DoorDash, Dyson, Goldman Sachs and Ramp, and states that at Netflix it supports over 3000 AI/ML projects. It also points to a guide for deploying the supporting infrastructure in your own cloud. Contributions are welcomed through the project's contribution guide.