ifixai

iFixAi provides a framework for the independent auditing of AI agents, focusing on operational assurance and adversarial testing. Unlike traditional observability tools that focus on technical metrics like latency or token efficiency, iFixAi assesses if an agent is performing its intended business role. The tool evaluates agents across five core pillars: Fabrication, Manipulation, Deception, Unpredictability, and Opacity. Based on these inspections, it assigns a letter grade (A–F). It also includes a premium tier of 13 additional categories (such as sabotage and systemic risk) for deeper analysis. Key features include: - Multiple execution modes: A guided CLI wizard, explicit CLI flags for CI/CD automation, and native plugins/skills for agents like Claude Code, Cursor, and VS Code. - Independent Judging: Supports "citable" grades where a second, independent AI provider acts as the judge to avoid self-grading bias. - Flexible Integration: Can test bare model APIs or real deployed agents via OpenAI-compatible HTTP endpoints. - Customizable Test Suites: Offers various suite levels including smoke, strategic, core, extended, and all. - Comprehensive Reporting: Generates results in JSON and Markdown formats, including a detailed scorecard.