ai-proxy

This project is a fork of the ai-proxy service, designed to act as a secure proxy for various Large Language Model (LLM) providers. It provides a unified API endpoint, allowing users to interact with multiple AI services without modifying application code for each provider. Key capabilities include: - Unified API Access: Supports providers such as OpenAI, Google Gemini, Groq, OpenRouter, Cloudflare, Cohere, and Sberbank GigaChat. - Request Management: Implements rate limiting (per minute/hour/day), intelligent routing, and structured error handling. - Performance Optimizations: Utilizes buffer pooling via sync.Pool for memory efficiency, optimized connection pooling for HTTP clients, and a streaming retry mechanism with exponential backoff and jitter. - Security: Features proxy access token authentication and secure configuration management via environment variables. - Streaming Support: Supports server-sent events (SSE) for providers that implement streaming responses. - Configuration: Uses a YAML-based setup to define models, priorities, and API keys.