About this project

Rhubarb Lip Sync is a command-line tool that analyzes audio recordings and automatically generates 2D mouth animation data. It works on Windows, macOS, and Linux, and can process WAV and Ogg Vorbis audio files. **How it works** The tool uses speech recognition to determine what is being said at each point in the recording, then maps the sounds to appropriate mouth shapes. Two recognizers are available: PocketSphinx (default, for English speech) and a language-independent phonetic recognizer (for non-English recordings). Users can also supply a dialog text file to improve recognition accuracy, especially when the recorded performance deviates from the script. **Mouth shapes** Rhubarb Lip Sync uses between 6 and 9 mouth positions. The 6 basic shapes (A through F) follow the Hanna-Barbera standard, covering common consonants and vowels. Three extended shapes (G, H, X) are optional and cover upper-teeth-lower-lip, tongue-raised 'L' sounds, and the idle/rest position respectively. **Output formats** The tool can export animation data as TSV (tab-separated timestamps and shape names), XML, or JSON. For Moho and OpenToonz integration, it can generate .dat switch data files. The output specifies which mouth shape to display at each point in time during the recording. **Integrations** - Adobe After Effects: animate dialog directly within the compositing workflow - Moho and OpenToonz: produce .dat switch data files, with support for Preston Blair naming conventions - Spine by Esoteric Software: a graphical tool for importing projects, running lip sync, and re-importing results - Vegas Pro by Magix: two plugin scripts for direct video editing integration **Command-line usage** A basic invocation looks like: ``` rhubarb -o output.txt my-recording.wav ``` Key options include `--recognizer` to choose between pocketSphinx and phonetic modes, `--exportFormat` for the output format, `--dialogFile` to supply reference text, `--extendedShapes` to control which optional mouth shapes are used, and `--quiet` / `--machineReadable` for scripting and automation. The tool supports multithreading and can emit structured JSON status messages for integration with other applications. **Building from source** Requires CMake 3.10+, a C++17-capable compiler, and Boost. The project includes CI configuration and build instructions for common platforms.