insightface

InsightFace is a comprehensive open-source project for 2D and 3D face analysis, maintained by Jia Guo and Jiankang Deng. It provides a rich set of state-of-the-art algorithms for face recognition, face detection, face alignment, face swapping, and 3D face reconstruction, all optimized for both training and deployment. The project is built primarily on PyTorch and MXNet, supporting Python 3.x with PyTorch 1.6+ and/or MXNet 1.6-1.8. Key modules include: 1. Face Recognition: Implements ArcFace (CVPR 2019), SubCenter ArcFace (ECCV 2020), PartialFC (CVPR 2022), VPL (CVPR 2021), and more. Supports multiple backbones including IResNet, MobileFaceNet, MobileNet, InceptionResNet_v2, and DenseNet. Includes training data (cleaned MS1M, VGG2, CASIA-Webface), evaluation pipelines (IJB, MegaFace), and pretrained models. 2. Face Detection: Implements RetinaFace (CVPR 2020) and SCRFD (Arxiv 2021), providing training code, datasets, pretrained models, and evaluation scripts. 3. Face Alignment: Implements SDUNets (BMVC 2018) for heatmap-based alignment and SimpleRegression for lightweight coordinate regression. 4. 3D Face Reconstruction: Includes methods for monocular 3D face reconstruction, facial reflectance reconstruction (CVPR 2024), and gaze estimation. 5. Face Swapping: Provides inswapper models for facial identity swapping. 6. Liveness Detection: Optional RGB liveness detection addon with normal/observe modes, per-face scores, and input-rejection reasons. InsightFace 2.0 introduces PrivateFrame for local video face blur and mosaic, reference-photo selection, and editable analysis JSON through desktop, CLI, and Python workflows, with a default Fast mode targeting 15 analysis FPS. The project also provides: - A Python package for easy detection, recognition, and alignment on input images, with automatic CoreML/CUDA/CPU provider selection and reusable CoreML compilation caches. - InsightFace Server: a self-hosted Web UI with snake_case REST API and Python client for detection, optional liveness checks, comparison, registration, and person search. Runs on a single Linux x86_64 CPU or NVIDIA GPU container with local ONNX Runtime inference and SQLite, using operator-supplied manifest-verified models. Offers accuracy-preserving INT8 embedding quantization and 50M+ image search on one RTX 5090 GPU. - InspireFace: a cross-platform C/C++ face recognition SDK. - A cross-platform desktop GUI Demo for face recognition, enterprise evaluation, reports, and face swap trials. The code is released under the MIT License with no limitations for academic and commercial usage. Training data and models are available for non-commercial research purposes only. The project has achieved notable results including Rank-1st on the NIST-FRVT 1:1 VISA track and 1st place on the ECCV-2022 WCPA Workshop Perspective Projection Based Monocular 3D Face Reconstruction Challenge.