llm-router

LLM Router is a server designed to route OpenAI-compatible API requests across different LLM providers. It provides a unified interface and handles the conversion of requests and responses between OpenAI, Anthropic, and Gemini formats. Key capabilities include: - Multi-provider support: Compatible with OpenAI, Anthropic, and Gemini API endpoints. - Request/Response Conversion: Automatically translates data formats between the supported API standards. - Advanced Model Selection: Supports model selection using full jq expression syntax, allowing for conditional routing based on the request body. - Routing Strategies: Implements round-robin, random, and least-connection strategies with weight-based distribution. - Error Handling: Automatically reduces a model's weight upon failure to manage availability. - Configuration: Managed via a YAML file where users can define model lists, API keys, base URLs, and routing groups.