reef

Reef is the first open-source infrastructure for continual self-improving agents. It connects agent inference, feedback, learning, and versioned delivery. Use it to train model weights with Slime and SGLang, or improve an agent's harness, including its prompts, rules, and skills. ## When to use Reef Use Reef when you want your agent to keep improving simply by learning from how you interact with your agent. It supports three learning paths: - Model weight training: for a stronger model designed for you, requiring a trainable model, GPU stack, and feedback. - Harness optimization: for self-improving harness (prompts, rules, skills), requiring a model endpoint, tasks, and evaluator; no local training GPUs. - Scientific discoveries: test-time training with an execution environment, correctness checker, and measurable objective. ## How it works Reef processes each learning cycle in four steps: 1. **Serve**: Serve agent requests and record interactions. 2. **Observe**: Match feedback to recorded interactions. 3. **Grow**: Produce an update from eligible records. 4. **Commit**: Apply selection policy and publish accepted updates. ## Installation Install from PyPI with `uv pip install reef-infra` or from source. Requires `git-lfs` for artifact and checkpoint functionality. ## Usage Start Reef as a pure inference server: `uv run reef serve --inference.model-path Qwen/Qwen2.5-1.5B-Instruct`. For weight-training deployments, use the SAO example. Send inference requests via OpenAI/Anthropic-compatible endpoints, report feedback with scores and receipts, and watch the model learn and update weights without restart. For harness-evolving deployments, use the built-in Reefine recipe. It refines a coding harness from plain-language asks using a model API. Install the harness with `reef-pi` and evolve it with commands like `reef-pi evolve "..."`. ## Recipes and examples Reef includes recipes for scientific discovery (TTT-Discover, Guidance-TTT), continual learning on task streams (SAO, Meta-Harness, GEPA), and learning from usage (OpenClaw-RL, SkillClaw, Reefine). Each has guides, code, examples, and measured benchmarks. ## Architecture The architecture diagram shows harness requests flowing through a scenario to inference, receipt-linked feedback feeding records and recipe training, and artifact evaluation selecting updates for versioned publication. ## Learn more Documentation covers quickstart, HTTP API, writing recipes, evolving harness or model, recipe catalog, core loop, and glossary. ## Community Join Discord, WeChat, GitHub Discussions, contribute via guide, propose RFCs, report vulnerabilities. ## Team Listed alphabetically, including Wenhao Chai, Shuangrui Ding, Shiyi Zoe Du, Hao He, Haoze He, Chonghe Jiang, Nan Jiang, Xuan Jiang, Xiaochen Li, Paul Liang, Bo Liu, Boyuan Long, Qiuyang Mang, Zhenting Qi, Ao Qu, Mingruo Qu, Zhaokai Wang, Xuezhi Yan, Hanfei Yu, Haofei Yu, Simon Yu, Han Zheng, Kaichen Zhou, Zijian Zhou, Jiacheng Zhu, Dingyi Zhuang, Xinkai Zou. ## Acknowledgements Grateful to SGLang, slime, and cordis for powering important parts of Reef.