needle

Needle is a specialized foundation model architecture designed for deployment on tiny devices, including microcontrollers, robots, smart home hardware, and mobile platforms. The model utilizes a 'Laddered Simple Attention Network' architecture, enabling it to function effectively with a footprint of only 8-29 MB. It is optimized for three primary tasks: tool calling, structured data extraction, and text embedding. Key features include: - Tool Calling: The model can parse natural language requests to trigger specific functions with correct arguments, returning empty lists rather than guesses for unsupported queries. - Structured Extraction: Converts unstructured text into typed JSON fields, useful for processing invoices, forms, or notifications. - Text Embedding: Provides vector representations for local search and routing tasks. - Scalability: The architecture supports a 'ladder' of depths (2 to 20 layers), allowing developers to select the optimal model size for their specific hardware constraints. - Fine-tuning: Supports both local LoRA-based fine-tuning and platform-based full-model fine-tuning to improve performance on domain-specific tasks. - Deployment: Includes a lightweight engine (under 1 MB) for various platforms, including macOS, Linux, and WASI, with support for air-gapped environments.