OPEN SOURCE, OPEN TO EVERYONE

Open source. Open possibilities.

Discover quality open-source projects, submit projects anonymously, and claim and edit your own project.

Human-curated · Discover open source4109discovered

A little curiosity. A world of open source.

THE FIRST COLLECTION
Topic: web-scraping清除

AI-powered template that clones any website into a clean Next.js app via a single command. Supports Claude Code, Cursor, Codex, Gemini, and more.

Developer toolsAI & MLTemplates & starters
web-coren24q02m
ADDED

web-core is a Python package providing shared web infrastructure: SearXNG search with retry, deduplication and domain filtering; a multi-strategy scraping agent with automatic escalation; an SSRF-safe HTTP client with DNS pinning; stealth and remote browser rendering; and typed Google Drive and MangaDex adapters.

Developer toolsAutomationBackend & APIs
node-sdkknownagents
ADDED

The @knownagents/sdk is a Node.js library that lets server‑side applications track AI agent traffic, record LLM referrals, generate up‑to‑date robots.txt files, and identify bots via the Agent Identification API. It supports batch event flushing, MCP call tracking, and integration with ACP/UCP commerce flows.

AI & MLDeveloper toolsAI agents
design-extractManavarya09
ADDED

designlang is a Playwright-based CLI that reads a website's live DOM and extracts its design system: DTCG tokens, Tailwind config, Figma variables, shadcn theme, component anatomy, motion tokens, brand voice, WCAG contrast audits, and multi-platform emitters, plus MCP server and Chrome extension.

DesignDeveloper toolsDesign systems
firecrawlfirecrawl
ADDED

Firecrawl is an open-source web data API for searching, scraping, crawling, and interacting with websites, turning pages into clean Markdown, JSON, or screenshots for LLM-ready use by AI agents and applications.

AI & MLDeveloper toolsAI agents
ADDED

Crawlee is a Python web scraping and browser automation library for building reliable crawlers. It supports HTTP and headless browser crawling, automatic retries, proxy rotation, and data storage for AI/LLM applications.

Developer toolsAI & MLBackend & APIs
ferretMontFerret
ADDED

Ferret is a declarative data automation language and Go runtime for structured extraction workflows, supporting embedding, capability-based access, and cross-source querying.

AutomationDeveloper toolsData automation
ADDED

An open-source tool for monitoring website changes and receiving real-time alerts via various notification channels.

AutomationDeveloper toolsData automation