dvc

DVC (Data Version Control) is a command-line tool and VS Code extension designed to make machine learning projects reproducible. It brings Git-like version control to data and models: data artifacts are stored in a cache outside of Git (in cloud or on-premise storage) while lightweight meta-files in the Git repository track their versions. This allows teams to share large datasets and models without checking them directly into Git. Key capabilities include: - Data and model versioning: Store data in S3, Azure, Google Cloud, SSH, or other remotes, while keeping version info in Git. - Lightweight ML pipelines: Define computational graphs connecting code, data, and outputs. When changes occur, only impacted steps re-run. - Local experiment tracking: Run, filter, and compare experiments by hyperparameters and metrics, with visualization plots, all without a dedicated server. - Collaboration: Share and automatically reproduce experiments through existing Git hosting platforms. DVC can be installed via pip, conda, brew, choco, snap, or platform-specific packages. Optional dependencies provide support for specific remote storage backends (S3, Azure, GCS, OSS, SSH, Google Drive). The VS Code extension offers a GUI for experiment tracking and data management. The project is distributed under the Apache License 2.0.