azure-ai-proxy

Azure AI Proxy is a lightweight, modular server that exposes an Azure OpenAI–compatible API interface while routing requests through LiteLLM to backend models from providers including DeepSeek, MIMO, Anthropic, Ollama, and custom endpoints. It enables developers to use non-Azure LLMs within tools and IDEs (like JetBrains with GitHub Copilot) that expect the Azure OpenAI format. The proxy supports chat completions, embeddings, legacy completions, model listing, and deployment management. Key features include multi-model configuration via YAML, model identity emulation for UI compatibility, enhanced streaming with keepalive and timeout handling, tool call sanitization, graceful error mapping, debug logging, and optional API key authentication. Installation options include pip, one-liner scripts for Windows/macOS/Linux, or Docker. Configuration is managed through a simple config.yaml file defining host, port, timeouts, logging, and per-model parameters including API keys, base URLs, and token limits. The project structure is cleanly modularized with handlers for each endpoint type and middleware for auth, CORS, and logging. It requires only aiohttp, litellm, and pyyaml dependencies. The proxy is licensed under GPL v3 and maintained by CarmJos, with support encouraged via GitHub Sponsors or ifdian.net.