semantic-router

Semantic Router is an open-source Python library designed to provide a superfast decision-making layer for Large Language Models (LLMs) and AI agents. Instead of relying on slow LLM generations to make tool-use or routing decisions, it utilizes semantic vector spaces to route user queries to the appropriate routes or tools based on meaning. Key features include: - **Semantic Routing**: Define routes with sample utterances, and the router quickly classifies incoming queries into the correct category using embedding models. - **Encoder Support**: Integrates with various embedding providers like Cohere, OpenAI, HuggingFace, and FastEmbed, enabling both cloud and fully local execution. - **Vector Database Integration**: Supports syncing embeddings with Pinecone and Qdrant for scalable indexing and retrieval. - **Dynamic Routes**: Enables parameter generation and function calls based on user queries. - **Multi-Modal Routing**: Capable of processing and routing multi-modal data (e.g., images). - **Threshold Optimization**: Allows training and tuning route thresholds to optimize classification performance. - **Integrations**: Easily integrates with LangChain agents and other LLM frameworks. The library is available in two release lines: the stable 0.x line and the 1.x breaking rewrite currently in development on the main branch.