minimind

MiniMind provides a complete lightweight large language model training pipeline, aiming to enable individual developers to quickly reproduce the entire LLM process. The core algorithms of the project are implemented using native PyTorch without relying on high-level abstract interfaces, offering significant educational and practical value. Main Capabilities and Features: 1. Complete Training Pipeline: Covers the entire process from data cleaning, Pretrain, Supervised Fine-Tuning (SFT), LoRA, and RLHF (DPO) to RLAIF (PPO/GRPO/CISPO). 2. Model Structure: The main structure is aligned with the Qwen3 ecosystem, supporting both Dense and MoE architectures. It provides various model scale versions, such as 64M (Dense) and 198M (MoE). 3. Functional Extensions: Supports Tool Use, Agentic RL, adaptive Reasoning, and model distillation. It also extends to vision (MiniMind-V) and multimodal (MiniMind-O) models. 4. Ecosystem Compatibility: Compatible with mainstream frameworks such as transformers, trl, and peft, and supports inference engines like llama.cpp, vllm, and ollama. 5. Practical Tools: Provides an OpenAI API-compatible server and a Streamlit-based WebUI chat interface. 6. Datasets: Open-sources high-quality datasets covering the pre-training, SFT, and RL stages.