llm-router

LLM-Router is a production-ready gateway designed to consolidate various Large Language Model (LLM) providers into a unified, OpenAI-compatible API. It allows developers to switch between providers such as OpenAI, DeepSeek, and Anthropic, or local models like Ollama, without changing their client-side code. The system is built on an asynchronous architecture featuring connection pooling, request caching, and real-time response streaming via Server-Sent Events (SSE). Key enterprise features include API key authentication, rate limiting, request validation, and dynamic configuration hot-reloading. For monitoring and maintenance, it offers built-in health checks, Prometheus metrics, and structured logging. Deployment is streamlined through Docker and Kubernetes manifests, making it suitable for high-load production environments.