agents

LiveKit Agents is an open-source framework for building realtime, programmable participants that run on servers. Its focus is conversational, multimodal voice agents that can see, hear and understand, and it is distributed as a Python package (a JS/TS sibling library, AgentsJS, is referenced separately). Key capabilities described in the README: - Flexible integrations: mix and match STT, LLM, TTS and Realtime API providers for a given use case. - Integrated job scheduling: built-in task scheduling and distribution with dispatch APIs to connect end users to agents. - WebRTC clients: client applications can be built with LiveKit's open-source SDK ecosystem across major platforms. - Telephony: works with LiveKit's telephony stack so agents can make or receive phone calls. - Data exchange: RPCs and other Data APIs allow agents and clients to exchange data. - Semantic turn detection: a transformer model detects when a user has finished speaking, which the project says helps reduce interruptions. - MCP support: tools from MCP servers can be integrated with one line of code. - Built-in test framework: tests and judges can be written to check agent behavior. - Open source: the whole stack can be self-hosted, including the LiveKit server, described as a widely used WebRTC media server. Core concepts are Agent (an LLM-based application with instructions), AgentSession (a container managing interactions with end users), entrypoint (the starting point for an interactive session, similar to a request handler), and AgentServer (the main process coordinating job scheduling and launching agents). Usage examples show a simple voice agent wiring VAD, STT, LLM and TTS through LiveKit Inference or direct provider plugins, plus a multi-agent handoff example where an intro agent gathers user details and passes control to a story agent that can switch to a Realtime API model. A testing example uses pytest with an AgentSession, runs a user input, and asserts on events such as function calls and assistant messages, including an LLM judge for intent. Running modes: `python myagent.py console` for local terminal audio testing without external servers, `dev` for a hot-reloading agent server that connects to LiveKit Cloud or a self-hosted server, and `start` for production-ready optimizations. Environment variables LIVEKIT_URL, LIVEKIT_API_KEY and LIVEKIT_API_SECRET are required for server-connected modes. The repository includes an examples directory covering a starter voice agent, an outbound caller, MCP support, a multi-user transcriber, video avatars with third-party providers, and a Gemini Live vision demo. Development uses uv for package management, ruff for formatting and linting, pytest for unit tests, and pdoc for local documentation generation. The framework is licensed under Apache-2.0, while the LiveKit turn detection models use a separate LiveKit Model License. Contributions are welcomed via issues, pull requests or the community Slack.