private-gpt

PrivateGPT describes itself as an open-source API layer for building private AI applications on top of local models. It does not run models itself; instead it connects to any OpenAI-compatible inference server (for example Ollama, llama.cpp, or vLLM) through OPENAI_API_BASE and exposes an API modeled after the Claude API. The project began as a popular 2023 proof of concept for chatting with documents offline, and version 1.0 was rebuilt from the ground up as a broader application backend. Key capabilities include a standard messages API with streaming, async processing and token counting; file and artifact ingestion; retrieval with citations; agentic RAG; built-in tools such as web search, web fetch and code execution; custom tools and MCP connectors; structured access to databases and CSVs; embeddings; and model selection from the connected inference server. The built-in workbench UI, available at /ui, is intended for testing, demos and local pilots rather than as the core product. The actual product is the API. Installation options include Homebrew on macOS, uv tool installation on Linux and Windows, and Docker (covered in the full quickstart guide). The README lists integrations with Claude Code, Claude Desktop/Cowork, Claude for Microsoft 365, OpenCode, and compatibility with tools such as n8n, VS Code, Cline, and other clients that can target a local OpenAI-compatible provider. Compared with inference servers like Ollama, LM Studio, LocalAI, vLLM, and llama.cpp, PrivateGPT positions itself as the layer above local model inference. Compared with app-first products like Onyx and Open WebUI, it describes itself as API-first, providing a standardized local backend for building self-hosted AI applications. PrivateGPT is maintained by Zylon, which also offers a separate enterprise AI infrastructure product built around PrivateGPT. The project has an active community and accepts contributions via GitHub.