ms-job-watcher

ms-job-watcher is an automated job-alert pipeline that runs unattended on GitHub Actions. It polls six major company career APIs (Microsoft, NVIDIA, Amazon, Goldman Sachs, IBM, Oracle) and approximately 1,200 curated ATS job boards across Greenhouse, Lever, SmartRecruiters, Workday, and Ashby platforms. Jobs are filtered for US-based entry and mid-level software engineering roles using rule-based classification that hard-excludes internships, QA, sales, marketing, clearance, citizenship-required, and non-sponsoring positions, while soft-excluding DevOps/ops roles unless paired with a strong SWE signal. Results are deduplicated against persisted JSON state, with new matches emailed as digests via Gmail SMTP. The pipeline runs two independent modes: a main pipeline checking company APIs every ~10 minutes and a boards pipeline sweeping ATS boards every ~30 minutes in cursor-based batches of 200 to complete full sweeps within GitHub Actions timeouts. Boards are fetched concurrently using ThreadPoolExecutor with per-platform semaphores. Dead boards returning 404/410 are automatically skipped. A Flask dashboard provides browsing of seen jobs and run history. Tech stack: Python 3.11, requests, BeautifulSoup4, GitHub Actions, Gmail SMTP, Flask. State persists as atomic JSON files committed back to the repo to survive ephemeral runners. Production scheduling uses cron-job.org hitting the workflow_dispatch API for reliable short intervals.