finddupes

findDupes is a command-line utility written in Go that locates duplicate files by content hash rather than by filename, so copies with different names are still detected. It searches one or more directories and hashes concurrently. Scanning proceeds in stages to avoid reading every file in full. First it recursively walks the supplied roots and collects the path and size of each regular file; files whose size is unique among the discovered files are discarded, since they cannot have an identical copy. The remaining candidates are hashed over their first 4 KiB by a worker pool, and only files sharing that prefix hash are read again and hashed in full with SHA-256. Files with the same full hash are reported as duplicates. The size and prefix checks act purely as filters, so files with different sizes or prefix hashes are never fully hashed against one another. Root handling is deliberately careful. Roots are converted to absolute, symlink-resolved paths before scanning; repeated or overlapping roots (such as the same directory given twice, or a directory and its subdirectory) are deduplicated, including different spellings or paths resolving through the same symlink. Symlinks encountered inside a scanned directory are not followed, only regular files are collected, and multiple paths to the same hard-linked file are collected once. A file that cannot be read during hashing is logged and skipped; if a root cannot be resolved or walked, files found under other roots are still processed, but the command returns an error after printing results. Ctrl-C and SIGTERM cancel an active scan, with work stopping between filesystem reads. Duplicate groups are printed with their SHA-256 hash, file size and paths; paths within each group and the groups themselves are sorted for repeatable output. Flags include -v/--verbose for progress while scanning, -w/--workers for the number of concurrent hashing workers (default: number of CPUs), --json for newline-delimited JSON objects with hash, size and paths fields, --stats for scan statistics on stderr, and --version. Statistics cover discovered files, size-filtered files, content-hash candidates, duplicate groups and duplicate files; writing them to stderr keeps JSON output safe to pipe into tools such as jq. Installation requires Go 1.25 or later and can be done with go install github.com/mojotx/findDupes@latest, or by cloning the repository and running go install -v ./.... The repository includes CI and CodeQL workflows, and its test matrix runs the race-enabled suite on Linux, macOS and Windows using the Go version declared in go.mod.