Proving-Ground

Proving Ground provides a standardized methodology for grading AI agents based on behavioral evidence rather than marketing claims. It evaluates agents across twelve dimensions, including security, task success, grounding, safety, bias, privacy, instruction following, conversational quality, honesty, robustness, memory, and latency. Key technical features include: - Adapters: Supports black-box interaction via REST API and Socket.IO, with a browser-based WidgetAdapter planned. - Grading Pipeline: Uses a combination of deterministic checks for binary facts (e.g., secret leaks, latency) and an LLM-as-judge ensemble (utilizing Claude) for qualitative assessment. - Anti-Gaming Measures: Employs a public practice set for vendor self-testing and a private, held-out set for final certification to prevent over-tuning. - Certification Tiers: Assigns Standard, Premium, or Elite tiers based on composite scores and minimum performance floors across dimensions. - Verification: Includes tool-execution grading to verify that agents actually perform actions (e.g., booking or emailing) rather than just claiming to do so.