runanywhere-sdks

RunAnywhere provides eight SDKs (Swift, Kotlin, Flutter, React Native, Web, Electron, Python, and a CLI) built over a single C++ core. It enables local, offline AI inference across diverse hardware, routing calls to the best available engine on each device. **Capabilities:** - LLM chat with token streaming, multi-turn history, and LoRA adapters (Llama, Qwen, Gemma, Phi, DeepSeek, etc.) - Vision language models (VLM) for image understanding and camera Q&A - Speech-to-Text (Whisper, Moonshine) and Text-to-Speech (Piper, Kokoro, MeloTTS) - Voice agent pipeline (VAD + STT + LLM + TTS) - Embeddings and RAG with local document ingestion and streaming - Image generation via Stable Diffusion (Core ML) and inpainting (Hexagon NPU) - Structured JSON output and tool calling with agent loops - Computer-use action parser (CUA) **Inference Engines:** - QHexRT: RunAnywhere's runtime for Qualcomm Hexagon NPUs (v75/v79/v81), running LLMs, VLMs, speech, and TTS directly on the NPU, including MoE and 1-bit models. - MLX: Apple-native inference for LLM, VLM, STT, TTS, and embeddings. - llama.cpp: Runs everywhere via Metal (Apple), CUDA (NVIDIA, opt-in), WebGPU/WASM (browser), and CPU (NEON/AVX). - sherpa + ONNX: For speech, VAD, and embeddings across all platforms. - Core ML: For diffusion-based image generation on iOS/macOS. **Additional Features:** - OpenAI-compatible server mode via Python SDK and CLI, exposing endpoints for chat completions, embeddings, audio transcription/speech, and models. - RunAnywhere Console: A control plane for deploying models over the air and collecting telemetry across device fleets. - Connect (LAN): Allows a macOS host to share a loaded model with iOS/Android clients over a trusted local network. - A capabilities discovery API (capabilities()) lets apps query what the current device and installed engines can actually execute. SDK maturity varies: Swift, Kotlin, and rcli are stable; Flutter, React Native, and Web are in beta; Electron is in preview; and Python is in alpha.