opik

Opik is an Apache-2.0 licensed, open-source platform for LLM observability, evaluation, and AI agent tracing, built by Comet. It covers the full LLM application lifecycle, from development tracing to production monitoring. Key capabilities include deep tracing of LLM calls and multi-step agent workflows, datasets and experiments for evaluation, LLM-as-a-judge metrics for hallucination detection and RAG assessment, prompt management, and production-ready dashboards with online evaluation rules. The platform is framework-agnostic and supports Python and TypeScript SDKs, OpenTelemetry, and numerous third-party integrations including LangChain, Anthropic, Bedrock, CrewAI, and Autogen. Opik can be self-hosted using Docker Compose for local development or Kubernetes with Helm for scalable deployments, with the full server backend and web application included under the open-source license. A PyTest integration enables CI/CD evaluation of LLM pipelines, and an MCP server allows coding agents like Claude Code and Cursor to access traces and run evaluations.