route67

Route67 is a lightweight LLM routing system designed for OpenAI-compatible chat completion APIs. It enables users to define a routing table mapping natural language queries to target models (e.g., 'Prove this theorem' → 'strong_model'). At runtime, incoming queries are matched against the routing table using cosine similarity computed by a sentence transformer model. If no match exceeds a configurable threshold, a designated 'weak model' acts as a gatekeeper: it either answers directly or escalates to a stronger model based on its judgment. The system supports any OpenAI-compatible provider (e.g., OpenRouter) by injecting a custom client. Model names in the routing configuration are passed through unchanged, allowing flexible backend switching without modifying routing logic. Provider-specific request options (like extra_body) and response fields (like reasoning_details) are preserved. Routing table entries only reference "weak_model" or "strong_model"; actual model identifiers live in ModelSpec objects. Usage notes from ModelSpec are injected into the weak model’s system prompt to guide escalation behavior. The embedding model (minishlab/potion-base-8M) is lazily downloaded on first use. The repository includes benchmarking tools to evaluate routing accuracy and threshold sensitivity using synthetic datasets. These are intended for development and tuning, not part of the standard PyPI install. Installation requires Python 3.10+ and can be done via pip or uv. Example code demonstrates minimal setup with environment variables and basic routing logic.