llm-gateway

Inja LLM Gateway is a lightweight, dependency-free API gateway for large language model providers. It accepts requests in OpenAI Chat Completions, Anthropic Messages, or native Gemini dialects and routes them to any configured upstream provider—including OpenAI, Anthropic, Google, DeepSeek, xAI, Moonshot, OpenRouter, vLLM, and others. Same-dialect traffic is passed through near-verbatim; cross-dialect requests are translated automatically, supporting structured outputs, tool use, and thinking blocks. The gateway is fully stateless with no database, sessions, or sticky routing, making it trivial to scale identical replicas. It ships as a single static binary for macOS, Linux, and Windows, and includes Docker and Kubernetes deployment manifests. The only runtime dependency is gopkg.in/yaml.v3. Key capabilities include multi-provider egress with four provider kinds (openai, openai_compat, anthropic, google), model aliases and routing, optional edge authentication with shared secrets, OAuth2 and service-account token sources, and a built-in operator dashboard at /ui. Usage metering emits one event per proxied request via JSONL, async webhook, optional SQLite, or an in-process Go hook. The gateway also supports embeddings, image generation, video jobs, text-to-speech, speech-to-text, OpenAI Responses, Files, Batches, Moderations, and Realtime WebSocket sessions (same-protocol passthrough only). Configuration is a single YAML file with environment-variable overrides. Health checks, Prometheus metrics, body-size limits, and graceful shutdown are built in. The project is licensed under AGPL-3.0.