neuralnote

NeuralNote is an advanced audio plugin (VST3/AU/Standalone) designed for Digital Audio Workstations (DAWs) that performs state-of-the-art audio-to-MIDI transcription using deep learning. It utilizes the MuScriptor model (from Kyutai and Mirelo), ported to a C++/ggml engine (`muscriptor.cpp`), allowing for fully local, privacy-preserving transcription on macOS and Windows (Linux support is upcoming). Key Features: - Multi-Instrument Transcription: Transcribe full musical mixes with notes grouped by instrument, or use the automatic instrument detection. - Fully Local Processing: All audio processing runs locally on your machine; audio never leaves your device. - Model Options: Comes in three sizes (small, medium, large) trading off quality for speed. Models are downloaded as GGUF files from Hugging Face. - Live & File Recording: Record live or drop audio files (WAV, AIFF, FLAC, MP3, OGG) directly into the plugin. - Background Transcription & Preview: Transcribes in the background while letting you preview the decoded MIDI in real-time. Drag and drop the resulting MIDI onto a track or save it directly. Build from Source: NeuralNote v2 is currently a work in progress and requires building from source. Requirements include git, CMake, Python 3, and a C++23 compiler (Clang is preferred). Windows GPU support requires the Vulkan SDK at build time, while macOS GPU acceleration utilizes Metal. License: The source code is released under the Apache-2.0 license, while the underlying MuScriptor model weights are released under CC BY-NC 4.0 (non-commercial use only).