About this project

OBEY API Gateway is a high-performance, single-binary AI gateway designed to sit between applications and multiple AI providers. It acts as a drop-in replacement for the OpenAI API, allowing users to point their existing SDKs to the gateway to gain advanced routing and reliability features without changing application code. Key capabilities include: - **Multi-Provider Support**: Compatible with OpenAI, Ollama, AWS Bedrock, Groq, Together AI, NVIDIA NIM, vLLM, and LM Studio. - **Reliability & Routing**: Features automatic failover with circuit breakers, exponential backoff, and smart rate-limit handling (honoring Retry-After headers). It supports priority, cost-aware, and complexity-aware routing. - **Advanced AI Features**: - **Response Caching**: Two-tier caching with an in-memory exact-match tier and an optional semantic tier via Qdrant. - **Token Management**: Includes 9 compression strategies for tokens and a 12-stage pipeline for tool definition compression. - **Guardrails**: Pre- and post-call policy enforcement including PII redaction, regex scanning, and moderation. - **Agent Loop Detection**: Detects repetitive agent behavior and escalates through warning and throttling levels. - **OpenAI Integration**: Supports OpenAI OAuth login for ChatGPT Plus/Pro subscriptions and provides a native /v1/responses endpoint. - **Developer Tools**: Includes a built-in admin panel/dashboard, Prometheus metrics, SQLite-based structured logging, and virtual key management for per-caller budgeting and rate limiting. - **Deployment**: Available as a Windows installer, Docker image, or can be built from source using Rust.