aicr

AI Cluster Runtime (AICR) is a toolset from NVIDIA for standing up GPU-accelerated Kubernetes clusters. It captures known-good combinations of drivers, operators, kernels, and system configurations and publishes them as version-locked recipes. These recipes are reproducible artifacts for Helm, Argo CD, Flux, and Helmfile. Key features include: - `aicr` CLI: single binary for snapshot, recipe, bundle, validate, verify, diff, and trust management. - API Server (`aicrd`): REST API for CI/CD integration or air-gapped environments. - Go Library: stable SDK for in-process consumers. - Snapshot Agent: Kubernetes Job that captures live cluster state into a ConfigMap. - Multi-Deployer Bundles: render recipes into Helm, Argo CD, Flux, or Helmfile artifacts. - Multi-Phase Validation: deployment, performance, and conformance phases. - Drift Detection: `aicr diff` compares snapshots. - Supply Chain Security: SLSA Build Level 3 provenance, signed SBOMs, and attestations. Supported components include GPU stack (GPU Operator, DRA GPU Driver, Network Operator, NFD, NVSentinel), cloud integration (AWS EFA, AWS EBS CSI, GKE NCCL TCPXO), node tuning, observability, training platforms (Kubeflow Trainer, Slinky Slurm Operator, KAI Scheduler, Kueue), and inference platforms (Dynamo, Grove, NIM Operator, Agent Gateway). Supported environments cover services like AKS, EKS, GKE, OCP, RKE2, etc.; accelerators like A100, H100, B200, GB200, etc.; operating systems like Ubuntu, RHEL, Talos, etc.; workload intents (Inference, Training); and platforms (Dynamo, Kubeflow, NIM, Run:ai, Slurm). AICR is not a Kubernetes distribution, cluster provisioner, managed control plane, or generic configuration management platform. It is a cluster configuration generator that produces runtime configuration artifacts for your deployment tooling and can validate that the configuration was correctly materialized. Documentation is available at docs.nvidia.com/aicr. The project is under Apache 2.0 license and welcomes contributions.