About this project

The Open-Source Log File & Bot Traffic Analyzer is a client-side tool designed for Technical SEOs, developers, and sysadmins to audit server access logs without uploading sensitive data to a third-party server. It processes JSON, JSONL, and NDJSON logs locally in the browser, supporting formats from Cloudflare, Nginx, Apache, AWS ALB, and Varnish. Key capabilities include: - Bot Classification: Identifies over 50 known search engine, AI scraper (e.g., GPTBot, ClaudeBot), and monitoring tool signatures. - Crawl Analysis: Detects crawl traps, pagination loops, and low-value directory paths. - Cost Estimation: Calculates approximate infrastructure egress overhead based on CDN pricing models (e.g., AWS CloudFront). - Security Detection: Flags automated path traversal attempts and vulnerability scanning patterns. - Edge Rule Generation: Produces ready-to-use syntax for Cloudflare WAF, Fastly VCL, and AWS WAF to block or rate-limit specific user-agents. The tool operates entirely offline after the initial load, ensuring that log data never leaves the user's machine.