llm-router

This is a local deterministic LLM routing proxy that accepts OpenAI and Anthropic API requests and routes them to the optimal available model based on configurable, rule-based scoring — no randomness or LLM-in-the-loop decisions. It supports multiple backends including OpenRouter (cloud meta-provider), oMLX (local Apple Silicon models), Ollama, and direct cloud providers like OpenAI, Groq, Gemini, etc. The system evaluates each request using a four-phase expert system: profile resolution, hard filtering (context, modalities, privacy, price caps), weighted scoring (cost, latency, context headroom, preference, quality), and provider-specific flag injection. Users can select from six built-in profiles (e.g., cheap, fast, private, smart, local) or define custom ones via config. Per-request steering is possible via headers or body fields. The proxy includes a web interface for chat playground, model catalog, routing explain, usage stats, and logs; plus a native macOS menu-bar admin app for GUI configuration. It handles upstream errors by falling back down the ranked list and streams SSE responses directly. Features include automatic OpenRouter key rotation, secret redaction (DLP), profile pinning to API keys, and quality scoring via Artificial Analysis. Docker support is provided with persistent config and SQLite history. No hot-reload — restart required after config changes. Built in Rust with modular crates for config, core logic, providers, and admin tools.