ai-gateway

AI Gateway is a lightweight, self-hostable API gateway designed to be compatible with OpenAI’s interface. It allows users to define routes mapped to specific model names used by their applications, then attempts requests across configured providers in sequence until one succeeds — helping reduce failures due to rate limits or outages. Supports Docker deployment, systemd services, and local binary builds. Configuration is managed via YAML with environment variable substitution, enabling flexible provider setup (e.g., Cerebras, OpenRouter) and per-route timeouts. Includes request adapters for compatibility fixes (like tool_choice relaxation or JSON schema injection), endpoint rotation on transient errors, and diagnostic endpoints. Authentication uses API keys; telemetry via OTLP/HTTP is optional. Security defaults include non-root execution and redacted logs. Ideal for developers integrating LLMs into apps like n8n while maintaining resilience and control over provider fallbacks.