cognigate

CogniGate is an open-source, self-hosted LLM gateway designed to sit between applications and various AI model providers. It allows applications to use a single OpenAI-compatible API key while the actual provider credentials remain securely within the deployment. Key capabilities include: - OpenAI-Compatible API: Integrates with existing SDKs without requiring code changes. - Capability Aliases: Uses aliases like `fast`, `balanced`, and `best` to resolve to specific models, allowing clients to remain agnostic of specific model versions. - Fallback Chains & Circuit Breakers: Implements ordered cascades for model rotation and exponential backoff for 429/5xx errors to ensure reliability. - Per-Tenant Quotas: Supports token and spend caps with an 'observe' mode for sizing. - Durable Usage Metering: Records all requests in PostgreSQL for analytics per tenant, model, or provider. The system architecture consists of a Go-based Gateway for request handling and a Java/Spring Boot Analytics engine for durable metering. It supports any provider that implements the OpenAI wire format, including Together, Groq, Fireworks, vLLM, Ollama, and Azure OpenAI.