OPEN SOURCE, OPEN TO EVERYONE

Open source. Open possibilities.

Discover quality open-source projects, submit projects anonymously, and claim and edit your own project.

Human-curated · Discover open source4105discovered

A little curiosity. A world of open source.

THE FIRST COLLECTION
Topic: data-engineering清除
delta-explaincdelmonte-zg
ADDED

delta-explain visualizes Delta Lake file pruning. It reads transaction logs to show partition pruning and statistics‑based skipping for a given predicate, with CLI, Python API, CI gating, and JSON reporting.

Developer toolsAutomationCLI & terminal
go-flowetl-madness
ADDED

Flow is a single-binary, high-performance ETL engine and workflow orchestrator written in Go. It features an embedded visual web builder, declarative XML pipelines, cross-database support, and built-in audit telemetry.

Data & databasesAutomationData integration

A runnable local real-time analytics pipeline using Kafka-compatible broker (Redpanda) and PySpark Structured Streaming. Simulates clickstream events, performs windowed aggregation with watermarking, all via docker-compose.

Developer toolsAutomationBackend & APIs
kestrakestra-io
ADDED

Kestra is an open-source, event-driven orchestration platform designed for data, AI, and infrastructure workflows, utilizing a declarative YAML interface.

AutomationDeveloper toolsWorkflow automation
ADDED

SolarisPKN-Stats is an automated system for collecting, storing, and updating project statistics from multiple platforms into structured JSON files.

AutomationData & databasesData automation
airflowapache
ADDED

Apache Airflow is a platform to programmatically author, schedule, and monitor workflows as code, commonly used for data pipelines and AI/ML orchestration.

AutomationData & databasesWorkflow automation