OPEN SOURCE, OPEN TO EVERYONE

Open source. Open possibilities.

Discover quality open-source projects, submit projects anonymously, and claim and edit your own project.

Human-curated · Discover open source4109discovered

A little curiosity. A world of open source.

THE FIRST COLLECTION
Topic: data-engineering清除
made-with-mlGokuMohandas
ADDED

Made With ML is an open-source course and codebase teaching how to design, develop, deploy and iterate on production-grade machine learning applications, covering MLOps, testing, serving, CI/CD and continual learning.

AI & MLDeveloper toolsTraining & evaluation

A runnable local real-time analytics pipeline using Kafka-compatible broker (Redpanda) and PySpark Structured Streaming. Simulates clickstream events, performs windowed aggregation with watermarking, all via docker-compose.

Developer toolsAutomationBackend & APIs
kestrakestra-io
ADDED

Kestra is an open-source, event-driven orchestration platform designed for data, AI, and infrastructure workflows, utilizing a declarative YAML interface.

AutomationDeveloper toolsWorkflow automation
xonshxonsh
ADDED

Xonsh is a full-featured, cross-platform shell that uses Python 3 as its language, seamlessly integrating shell commands with Python. It runs on Linux, macOS, Windows, BSD, Jupyter, Android, Raspberry Pi, and Nix, with an extension system and AI-friendly features.

Developer toolsCLI & terminal
ADDED

Great Expectations (GX Core) is an open-source Python library for data quality, providing a framework for creating unit tests for data to ensure consistency and reliability.

Data & databasesDeveloper toolsData integration
semanticasemantica-agi
ADDED

Semantica is a graph-native infrastructure designed to provide structured context, deterministic reasoning, and full decision provenance for AI systems, serving as a semantic layer for LLMs and agents.

AI & MLDeveloper toolsRAG & knowledge
delta-explaincdelmonte-zg
ADDED

delta-explain visualizes Delta Lake file pruning. It reads transaction logs to show partition pruning and statistics‑based skipping for a given predicate, with CLI, Python API, CI gating, and JSON reporting.

Developer toolsAutomationCLI & terminal