anime

AnimeLek Scraper is a Python-based tool designed to scrape the animelek.vip website. It systematically crawls all 83 pages of the anime catalogue, collecting metadata for each anime (title, poster, status, genres, description, studios, year), along with episode lists, watch server URLs (data-ep-url), and download links per episode. The scraper outputs structured JSON files: a comprehensive `data/animes.json` containing an array of anime objects, and individual JSON files for each episode under `data/episodes/`. It also maintains a `state.json` file to track progress, enabling resumable runs—users can safely interrupt with Ctrl+C and restart without re-scraping completed items. A rolling log file (`scraper.log`) records activity. For local usage, the tool requires Python and dependencies listed in `requirements.txt`. It uses `cloudscraper` and `BeautifulSoup` to fetch public pages, with no paid APIs involved. Anti-detection measures include random User-Agent rotation via `fake-useragent`, random delays between requests (1.5–3.5 seconds), exponential backoff on 429/5xx errors, and up to 4 retries per URL. The project also includes a GitHub Actions workflow that runs the scraper approximately every 5.5 hours (combining two cron schedules). It uses the built-in `GITHUB_TOKEN` for authentication, requiring the repository to have 'Read and write permissions' enabled for Actions. This allows automated, scheduled scraping without manual secrets. The README provides detailed JSON schemas for both anime and episode files, illustrating the structure with examples. Overall, this tool is useful for anyone needing to archive or analyze anime data from animelek.vip, offering robust scraping with resume capability and automation support.