kestra — Kestra is an open-source, event-driven orchestration platform designed for data, AI, and infrastructure workflows, utilizing a declarative YAML interface.

Kestra is an open-source, event-driven orchestration platform designed for data, AI, and infrastructure workflows, utilizing a declarative YAML interface.

About this project

Kestra is an orchestration platform that unifies scheduled and event-driven automation. It allows users to build reliable workflows using a declarative YAML interface, which can be managed via a built-in UI code editor or integrated with Git for Infrastructure as Code (IaC) practices. Key capabilities include: - Workflow Management: Supports namespaces, labels, subflows, retries, timeouts, conditional branching, and dynamic tasks. - Execution & Scaling: Designed for high availability and fault tolerance, capable of handling millions of workflows. It supports local execution, remote servers via SSH, and serverless containers via Task Runners, including Docker and Kubernetes. - Plugin Ecosystem: Provides hundreds of built-in plugins for database extraction, cloud storage, and API interactions. It supports scripting in multiple languages including Python, Node.js, R, Go, and Shell. - Event-Driven Triggers: Can react in real-time to external events such as file arrivals or messages from Kafka, Redis, Pulsar, AMQP, MQTT, NATS, AWS SQS, Google Pub/Sub, and Azure Event Hubs. - Visual Tools: Features a Topology Editor for drag-and-drop task arrangement, real-time syntax validation, auto-completion, and a live DAG (Directed Acyclic Graph) view. - Integrations: Offers an official Terraform provider and supports CI/CD pipeline integration for flow deployment.