sie

SIE (Superlinked Inference Engine) is an open-source inference server and production cluster designed to run all the models an AI agent needs from a single, self-hosted system. It replaces the need for separate model servers per task by providing one unified API that serves over 100 models, loading them on demand. Key features include: - **OpenAI-compatible API**: Drop-in migration with endpoints like `/v1/embeddings`, `/v1/chat/completions`, `/v1/completions`, and `/v1/responses`. - **Pre-configured model catalog**: Includes models like Stella, SPLADE, Qwen3, GLiNER, SigLIP, and more, with embedding and retrieval models benchmarked on MTEB. - **Multi-model serving**: Serves multiple models simultaneously with on-demand loading and LRU eviction. - **Production-ready**: Ships Kubernetes and Helm deployment configs, including a load-balancing gateway, KEDA autoscaling, and Grafana dashboards. - **Integrations**: Works with LangChain, LlamaIndex, Haystack, DSPy, CrewAI, Chroma, Qdrant, Weaviate, and LanceDB. SIE handles a wide range of tasks, including search (embedding, matching, reranking), document-to-markdown conversion (PDFs, Office files, scans), structured output (schema-valid JSON extraction), decision-making (choice, yes/no, score answers), classification, content safety, agent loop execution, translation (400+ languages), image captioning and object detection, and audio transcription. Quickstart options include native installation via pip for macOS/Linux, or Docker images for NVIDIA GPU (with bundle-specific images for different model families) and CPU. The SDK is available for Python (`sie-sdk`) and TypeScript (`@superlinked/sie-sdk`). For production, SIE offers Terraform modules for Alibaba Cloud ACK, AWS EKS, Azure AKS, and Google GKE, along with Helm charts for reproducible cluster deployments. The project is Apache 2.0 licensed and collects anonymous usage telemetry (disablable via environment variables).