premove-itn

Premove ITN is an open-source, context-aware inverse text normalization (ITN) library designed for English voice-agent transcripts. It converts spoken ASR (Automatic Speech Recognition) output into canonical written forms for structured values such as phone numbers, email addresses, identifiers, dates, times, money, measurements, and alphanumeric codes. The core challenge Premove ITN addresses is the ambiguity of spoken language. For example, 'one oh five' could mean the identifier 105, the time 1:05, or something else entirely. Traditional rule-based systems cannot always determine which interpretation the speaker intended. Premove ITN solves this by using context-aware scoring to select the correct written form. The architecture follows a three-step pipeline: generate, score, and decode. First, a Rust-based realization layer enumerates all valid written forms for spans of spoken text. Second, a DeBERTa-v3-large contextual scorer (435.6M parameters) encodes the transcript once and scores each candidate based on contextual source span, candidate kind labels, and proposed replacement. Third, an exact dynamic programming decoder finds the highest-scoring compatible path through the transcript, handling overlapping candidates. Supported forms include numbers, digit sequences, times, dates, money, decimals, measurements, ordinals, phone numbers, email and URLs, versions and identifiers, punctuation, and abbreviations. The tool is English-only and does not provide first-class normalization for non-English speech, street addresses, free-form rewriting, or arbitrary application-specific formats. On a frozen benchmark of 1,500 rows with a dedicated 400-row voice-agent subset, Premove ITN achieves 99.50% semantic accuracy on the voice-agent subset, compared with 68.25% for text-processing-rs and 67.00% for NVIDIA Thutmose. The benchmark is a synthetic stress suite held out from training and checkpoint selection. The tool is distributed as a Python package (premove-itn) on PyPI, with model weights hosted on Hugging Face (premove-ai/premove-itn). The current release is v0.2.0. Model loading is expensive (multi-second initialization, ~1.6 GB first download), but warm normalization calls are fast (56.49 ms mean warm latency on Apple M4 MacBook Air). Limitations include English-only support, bounded normalization by deterministic Rust realizers, rejection of inputs longer than 512 DeBERTa tokens, and unvalidated support for CUDA, Windows, macOS Intel, Linux ARM64, and other accelerators. The source code and model weights are MIT licensed. The contextual scorer uses microsoft/deberta-v3-large as its encoder backbone, and the Rust realization layer uses text-processing-rs (Apache-2.0 licensed).