polymo

Polymo allows users to define API ingestion pipelines through a UI and export them as standalone Lakeflow Declarative Pipelines. It generates plain Python code utilizing requests, the standard library, and pyspark, ensuring that polymo itself is not a runtime dependency of the generated scripts. Key capabilities include: - UI-based pipeline definition with local previews of DataFrames, raw records, and API responses. - Generation of specialized code for pagination, incremental sync, partitioning, filter pushdown, error handling, and XML responses. - Integration with Databricks Asset Bundles for bootstrapping, deploying, and running pipelines via the Databricks CLI. - Support for secure credential management using Databricks secret scopes or Unity Catalog service credentials to avoid hardcoding secrets. - Ability to launch via pip, uvx, or Docker.