turkeybite

TurkeyBite is an open-source domain and host context analysis pipeline designed to analyze client network traffic and categorize the domains requested by clients, covering categories such as adult content, gambling, shopping and more. It is explicitly built as a non-intrusive analysis tool that does not block client requests, so it can be run alongside blocking tools like Pi-hole if needed, with no impact on end user network experience. The pipeline relies on a Docker-based technology stack including Python 3, Valkey (Redis-compatible in-memory data store), Bind9 DNS server, Packetbeat and/or Browserbeat for traffic collection, OpenSearch for data indexing and storage, and OpenSearch Dashboards for visualization. It aggregates domain and host classification lists from multiple public sources to enrich analysis results. In operation, Packetbeat captures DNS traffic from supported DNS servers (tested in production with Bind9 and Microsoft DNS servers, and compatible with any DNS server that can run Packetbeat) and sends the data to Valkey. Browserbeat can be used to collect browser history data from supported browsers. The TurkeyBite core and worker processes process the ingested data, enrich it with domain category context, and index the results to OpenSearch for querying and visualization via OpenSearch Dashboards. Deployment is managed via Docker Compose, with a setup script that guides users through configuration for different deployment scales (development, small scale, full scale), including DNS lookup settings for client IPs, output options (OpenSearch and/or Syslog), and service credentials. The setup script automatically generates required configuration files (.env, config.yaml, docker-compose.yml) and secret files for service authentication. For distributed deployments across multiple nodes, the setup script can be run on each node with appropriate configuration, and Valkey authentication secrets need to be copied to core and worker nodes if Valkey runs on a separate dedicated server. For data collection configuration, Packetbeat needs to be set to capture DNS traffic on port 53 and output to Valkey, with careful matching of the datatype and key settings to the selected TurkeyBite pipeline mode: the "consume" mode uses Valkey lists for at-least-once delivery with persistent backlog tracking, while the "rq" mode uses Valkey pub/sub which is lossy by design. Browserbeat is configured separately following its own installation instructions to collect browser history data. Routine maintenance includes checking logs stored in the vols/logs directory, and domain lists stored in vols/lists are updated automatically according to user-configured refresh intervals. Common troubleshooting steps include checking container logs via docker compose logs, restarting individual services, and verifying connectivity between containers in the Docker network.