haystack

Haystack is a modular AI orchestration framework designed for creating scalable LLM applications. It allows developers to design pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. The framework is model- and vendor-agnostic, supporting integrations with OpenAI, Mistral, Anthropic, Cohere, Hugging Face, Google, Azure OpenAI, AWS Bedrock, and local models. Key capabilities include: - Production-ready Agents: Features lifecycle hooks for guardrails, monitoring for token usage and tool calls, and skill discovery via SkillToolset. - Context Engineering: Tools for retrieving, ranking, filtering, and routing information before it reaches the model. - Native Async Support: Pipelines can run synchronously or asynchronously with token-by-token streaming, and agents support concurrent tool calls. - Modular Architecture: Includes built-in components for indexing, retrieval, and evaluation, with the ability to create custom components. - Versatile Use Cases: Supports the development of Retrieval-Augmented Generation (RAG) systems, multimodal applications, semantic search, and question-answering systems.