mithril

Mithril is an open-source multi-model orchestration engine written in Rust (with a Python CLI wrapper) that combines multiple LLM providers into a single, unified API endpoint. The project's core value proposition is eliminating the need to manage separate integrations for each AI provider: users define a "fellowship" — a team of AI models — in a YAML configuration file, then point any Ollama-compatible or OpenAI-compatible client at Mithril's API. The orchestration engine supports three provider types: local GGUF models (via llama.cpp, free and private), cloud API providers (Gemini, OpenAI, Anthropic, Groq), and CLI tools (Kiro, Junie, GitHub Copilot, or any CLI with its own model access). A local GGUF model serves as the default controller/router, classifying incoming requests and delegating them to the appropriate agent based on the fellowship configuration. Agents communicate via a NEXT/TASK protocol, allowing them to delegate work to each other — for example, a coder agent can hand off to a reviewer agent for code review. The system exposes three API protocols simultaneously: Ollama API (for tools like Junie, OpenCode, Open WebUI), OpenAI-compatible API (for LangChain, LlamaIndex, and custom Python clients), and MCP (Model Context Protocol) for Claude Desktop integration. This means existing AI tooling can connect to Mithril without modification — simply change the base URL. Mithril ships with 24 built-in tools spanning file operations (read, write, edit, delete, apply_patch), terminal execution (sandboxed), file discovery (list, grep, find, glob, stats), Git integration (status, log, diff, blame, branch), web access (search, fetch), code analysis (symbol search, document outline), knowledge management (lore write/read), and interaction (todo, question). Agents can be granted access to specific tools or all tools via the fellowship YAML. Installation is flexible: a one-liner shell script for Linux/macOS, pip install for Python/Jupyter environments, Homebrew tap for macOS/Linux, standalone pre-built binaries for all major platforms (Linux x86_64/ARM64, macOS Apple Silicon/Intel, Windows x86_64), Docker (single container or Docker Compose), and building from source with Cargo. API keys are stored encrypted with Argon2id + AES-256-GCM in a local config file, or passed via environment variables for Docker/CI deployments. The CLI includes an interactive REPL with tab completion, a full-screen TUI mode, non-interactive execution for CI/scripts, file expansion (@file), agent routing (#agent), plan/build modes, undo/redo, session persistence, custom commands, and hooks. A Telegram bot mode is also available for chatting with the same fellowship via Telegram. Use cases include serving as a backend for AI coding assistants (Junie, OpenCode), a shared orchestration backend for teams via Docker, a Python data science integration via pip, and an MCP server for Claude Desktop. The architecture is designed for extensibility — new providers, tools, and agent types can be added through the fellowship configuration system.