litellm — LiteLLM is an open-source AI Gateway and Python SDK that provides a unified OpenAI-compatible interface for calling over 100 different LLM providers.

LiteLLM is an open-source AI Gateway and Python SDK that provides a unified OpenAI-compatible interface for calling over 100 different LLM providers.

About this project

LiteLLM simplifies the integration of multiple Large Language Models by providing a single interface to interact with providers such as OpenAI, Anthropic, Gemini, Bedrock, Azure, and many others. It can be utilized as a Python SDK for direct library integration or deployed as a centralized AI Gateway (Proxy Server) for organizational use. Key capabilities include: - Unified API: A single interface for 100+ LLMs, eliminating the need for provider-specific SDKs. - OpenAI Compatibility: Allows swapping LLM providers without rewriting application code. - Production Gateway Features: Includes support for virtual keys, spend tracking, guardrails, load balancing, and an admin dashboard. - A2A Agent Support: Ability to invoke agents from providers like LangGraph, Vertex AI Agent Engine, and Azure AI Foundry. - MCP Tools Integration: Connects Model Context Protocol (MCP) servers to any LLM via the SDK or Gateway. - Broad Endpoint Support: Supports various endpoints including /chat/completions, /embeddings, /images, /audio, and /rerank.