agentleak

AgentLeak is an open-source privacy testing framework designed specifically for AI agents. It addresses the critical security gap where an agent's final output appears clean, but sensitive data leaks through internal channels such as tool calls, shared memory, inter-agent messages, and logs. The tool operates entirely locally, ensuring no data leaves the user's machine, and requires no cloud services or LLM dependencies for its core analysis. Key features include: **Multi-Channel Detection:** AgentLeak inspects eight normalized channels: `user_input`, `final_output`, `inter_agent_message`, `shared_memory`, `tool_call`, `tool_response`, `log`, and `generated_file`. This comprehensive coverage allows it to detect leaks that traditional output-only audits miss. **Deterministic Scoring (AgentRisk):** Leakage is scored using the AgentRisk framework, a severity-weighted, density-normalized Risk Index grounded in GDPR Article 9 and Québec Law 25. The system grades secrets on a four-tier taxonomy (L1-L4) and calculates a Risk Index (RI) between 0 and 1, providing both global and per-channel metrics. **Integrations & Interfaces:** The tool offers a Python SDK, CLI, and local web UI (built with React and shadcn/ui). It supports a wide range of agent frameworks including LangChain, LangGraph, CrewAI, AutoGen, OpenAI Swarm, LlamaIndex, and more. Integration is simplified via a `agentleak.watch()` context manager or specific framework callbacks. **Red Teaming & Compliance:** AgentLeak includes a Red Team module with executable plugins for adversarial testing, mapping attacks to observable classes. It also provides compliance mapping to 14 major frameworks such as GDPR, HIPAA, PCI-DSS, and the EU AI Act, helping users identify which controls are at risk during a run. **Privacy Guarantees:** Detection and scoring are 100% local. Reports show masked values by default, and raw traces are not stored unless explicitly opted in. The tool supports optional detectors like Presidio for NER and LLM-judge for contextual leaks, but these are optional and can be configured to maintain strict local privacy.