peft — PEFT is a library providing state-of-the-art Parameter-Efficient Fine-Tuning methods to adapt large pretrained models with minimal computational and storage costs.

PEFT is a library providing state-of-the-art Parameter-Efficient Fine-Tuning methods to adapt large pretrained models with minimal computational and storage costs.

About this project

PEFT (Parameter-Efficient Fine-Tuning) enables the adaptation of large pretrained models to downstream tasks by fine-tuning only a small subset of model parameters rather than the entire network. This approach significantly reduces GPU memory requirements and storage space for checkpoints. Key capabilities include: - Implementation of various PEFT methods such as LoRA, Soft prompts, and IA3. - Integration with the Hugging Face ecosystem, including Transformers for training/inference, Diffusers for adapter management, and Accelerate for distributed computing. - Support for quantization (e.g., QLoRA) to further reduce memory footprints, allowing large models to be trained on consumer-grade hardware. - Ability to save and load lightweight adapters (often only a few MBs) instead of full model weights. - Compatibility with TRL for RLHF (Reinforcement Learning from Human Feedback) workflows, including Direct Preference Optimization (DPO).