llm-router

SJMN is a self-hosted router for LLM API calls designed to ensure high availability. It allows users to register multiple OpenAI-compatible connections (such as OpenAI, Azure OpenAI, OpenRouter, Groq, or local instances like Ollama and LM Studio) and assign them a priority order. If a primary connection fails due to a timeout or error, the router automatically fails over to the next available connection in the priority list. Key features include: - Admin Dashboard: A React-based interface to manage connections, generate API keys, and inspect request/response logs. - Routing Logic: Automatic failover based on priority numbers (lower numbers are tried first). - Playground: An integrated chat interface to test specific connections or the general fallback logic. - Health Monitoring: Tools to ping connections and mark them as online or offline, with status updates triggered by live traffic. - Security: JWT-protected admin access and API key authentication for the router endpoint. The system is built with a Node.js/Express backend using SQLite for data persistence and a React/Vite frontend. It can be deployed via Docker Compose or run locally.