bacalhau

Bacalhau is a community-developed, Apache-2.0-licensed framework for distributed Compute over Data (CoD). Its main approach is to schedule computation close to where data resides rather than copying large datasets to a central compute cluster. The project presents this as a way to reduce network transfer, latency, and ingress/egress costs while supporting data-sovereignty and cross-organizational processing scenarios. The software is distributed as a single binary that can operate as a client, orchestrator, or compute node. The orchestrator is responsible for job scheduling and coordination, while compute nodes execute tasks. Its modular architecture supports multiple execution engines, specifically Docker and WebAssembly, along with pluggable storage and publisher integrations. Documented storage inputs include S3, HTTP/HTTPS, IPFS, and local storage. Results can be published to local volumes, S3, or other supported storage backends. Bacalhau supports several job types—batch, ops, daemon, and service—and allows jobs to be submitted imperatively through the CLI or declaratively with YAML. The README lists use cases such as distributed log processing, regional data warehousing, fleet management, distributed machine learning, and edge computing. A Linux/macOS CLI installation script is provided, along with links to documentation for installation, submitting first jobs, and publishing results. The project also provides community channels and a contribution guide.