autoresearch

autoresearch is an autonomous AI research framework created by Andrej Karpathy that lets AI agents independently experiment with LLM training on a single GPU. The system is built around a simplified single-GPU implementation of nanochat, where an AI agent autonomously modifies the training code (train.py), runs experiments with a fixed 5-minute wall-clock budget, evaluates results using validation bits per byte (val_bpb), and iterates—keeping improvements and discarding regressions. The project is deliberately minimal, consisting of three key files: prepare.py (fixed data preparation and utilities, not modified), train.py (the single file the agent edits, containing the full GPT model, optimizer, and training loop), and program.md (baseline instructions for the agent, edited by the human). The human's role shifts from writing Python code to programming the program.md Markdown files that define the autonomous research organization. The default program.md is intentionally kept as a bare-bones baseline, though it is obvious how one would iterate on it over time to find the 'research org code' that achieves the fastest research progress, or add more agents to the mix. Training runs for exactly 5 minutes regardless of compute details, enabling approximately 12 experiments per hour and around 100 experiments overnight. This fixed time budget makes experiments directly comparable regardless of what the agent changes (model size, batch size, architecture, etc.) and ensures the system finds the optimal model for the given platform within that constraint. The metric val_bpb is vocab-size-independent, allowing fair comparison across architectural changes. Requirements include a single NVIDIA GPU (tested on H100), Python 3.10+, and uv. The project is self-contained with no external dependencies beyond PyTorch and a few small packages—no distributed training or complex configurations. Notable forks exist for macOS, Windows, and AMD platforms. The project is licensed under MIT.