speech-to-speech

Speech To Speech is a fully modular framework for building voice agents using open-source models. It organizes the voice interaction pipeline into four interchangeable stages: Voice Activity Detection (VAD), Speech-to-Text (STT), Large Language Model (LLM), and Text-to-Speech (TTS). Key capabilities include: - **Modular Architecture**: Every component is swappable. It supports various backends including MLX for Apple Silicon, CUDA for NVIDIA GPUs, and OpenAI-compatible APIs for hosted or self-hosted servers (e.g., vLLM, llama.cpp). - **Realtime API**: Exposes the core OpenAI Realtime GA event set via WebSocket and WebRTC, enabling low-latency turn-taking and live transcription. - **Flexible Deployment**: Can be run fully locally (on Mac or Linux), in a hybrid mode (local speech with hosted LLM), or as a server for external clients. - **Broad Model Support**: - STT: Parakeet TDT, Whisper, Faster Whisper, Paraformer, and Qwen3-ASR. - LLM: Transformers, mlx-lm, and various OpenAI-compatible providers. - TTS: Qwen3-TTS, Kokoro-82M, Pocket TTS, ChatTTS, and OmniVoice. - **Tool Integration**: The packaged Python client supports local tool calling via a specific module contract. - **LLM Proxy**: Optionally exposes the configured remote LLM as a standard OpenAI-compatible endpoint for concurrent side tasks.