bitcurrents-log-analyzer

BitCurrents Log Analyzer is a real-time Nginx log analysis tool built as a single-page web application with a Python/Flask backend and dynamic JavaScript frontend. It transforms raw Nginx access and error logs into interactive dashboards focused on security intelligence, performance monitoring, and data visualization. The application offers three distinct analysis modes: **General Mode** — Provides an at-a-glance traffic overview with KPI cards (total hits, unique IPs, impersonator and suspicious counts), charts for traffic breakdown (human vs bots vs crawlers), device types, OS and browser distribution, HTTP status codes, top countries, and most-accessed pages. Includes filtering to hide noisy IPs and toggle between unique IPs and all hits views. **Security Mode** — Enables proactive threat hunting with dynamic threat categorization (config exposure, webshells, WordPress probes) using a configurable `security_patterns.json`. Features targeted visualizations for top attack categories and attacking IPs, plus automatic bot impersonator detection that flags traffic claiming to be legitimate crawlers but originating from unverified IP ranges. **Error Mode** — Diagnoses server issues by intelligently parsing common Nginx `error.log` messages (e.g., "Backend Unreachable," "Index Forbidden"). Visualizes error types and severity levels, with an in-app troubleshooting guide for common errors. The project leverages the IP2Location.io API for high-fidelity geolocation (country, city, ISP data) and advanced bot detection beyond simple user-agent string matching. It maintains an IP cache file to conserve API credits. ### Setup and Usage Prerequisites include pyenv and Python 3.13.0. Clone the repository, set up a virtual environment, and install dependencies from `requirements.txt`. A free IP2Location API key must be configured via a `.env` file. The backend runs with `python app.py`, and the frontend is opened directly in a browser as a local file. Users point the app to their log directory, list available files, optionally specify paths to track, and click Analyze. ### Nginx Configuration For optimal results, the Nginx access log format should include `$http_referer` and `$http_user_agent` variables. Sample configuration is provided in the README. Concept by Ray Kooyenga (@deadflowers @rkooyenga). Created for the IP2Location 2025 Hackathon Programming Contest.