valkey-conductress

Conductress is a framework for benchmarking Valkey, designed to queue and execute performance, memory, and replication tests. It collects and records benchmark data for analysis. The tool provides a text-based user interface (TUI) for interactive monitoring and task queuing, and a command-line interface (CLI) for scripted automation. Users can queue tasks with configurable parameters including test types (e.g., get, set, mget), value sizes, IO thread counts, pipelining levels, warmup and test durations, and repetitions. The CLI computes the Cartesian product of multi-valued parameters, generating separate tasks for each combination. Benchmarks can be run on remote servers (via SSH) or localhost. Setup involves configuring servers and installing dependencies. The framework supports key-size benchmarking by generating padded keys and insertion-only SET tasks that measure new-key path performance. Results are aggregated when repetitions are specified, computing mean requests per second and 95% confidence intervals. The analysis module compares two specifiers (branches, tags, or commits) using Welch's t-test, presenting deltas, p-values, and measurement precision. Output includes a summary of significant changes and detectable effect sizes. Additional features include hardware performance counter collection, flame graph generation, and crash logging.