go-flow

Flow is a modern, lightweight ETL execution engine and workflow orchestrator designed as a single Go binary with zero external runtime dependencies. It serves as an alternative to legacy tools like SSIS and complex code-heavy orchestrators like Apache Airflow. Flow combines declarative XML pipelines with an embedded visual HTMX web builder, allowing users to design, validate, and execute data processing workflows without heavy infrastructure. Key capabilities include native support for multiple heterogeneous database engines (PostgreSQL, SQL Server, Oracle, MySQL, SQLite, and Key-Value stores) with automatic dialect abstraction for SQL placeholders. The engine supports advanced orchestration primitives such as parallel execution, conditional logic, iterative loops, and preflight checks. It integrates interpreted Go scripts and SQL queries within the same pipeline. Flow provides built-in observability through automatic audit telemetry, writing execution status, durations, and row counts to `pipeline_runs` and `pipeline_events` tables. It also supports self-documenting pipelines by generating Mermaid.js visual flowcharts and offering XSLT 3.0 transformation capabilities for custom reporting. Configuration is Git-friendly, supporting hierarchical overrides via environment-specific XML files and runtime CLI variables.