PhonoNoise: Reshaping Diffusion Geometry via Phonology-Aware Non-isotropic Diffusion for Sign Language Production

Sign Language Production (SLP) aims to generate semantically faithful and articulatorily coherent sign motion from spoken-language text. Since the hands carry much of a sign's lexical content, errors in handshape, inter-hand configuration, and movement directly impair intelligibility, yet existing methods still often produce distorted fingers, stretched hand bones, and inaccurate trajectories. We attribute this partly to the forward process of diffusion-based methods, which corrupts all joints with isotropic noise and thus leaves hand-joint dependencies to be learned implicitly from data. Motivated by sign phonology, where handshape and inter-hand arrangement describe spatial hand relations and movement describes how the hands travel over time, we propose a phonology-aware diffusion framework with two components. PhonoNoise derives a phonology-induced covariance from intra-hand topology and bilateral joint correspondence for non-isotropic hand diffusion. PhonoContext extracts movement context from noisy wrist trajectories and injects it into the denoiser. Neither requires phonological annotations. Experiments on PHOENIX-2014T and CSL-Daily show state-of-the-art back-translation performance, with BLEU-4 scores of 17.11 and 6.21, respectively, while substantially reducing limb stretching. Code and more visualizations will be released on our project page.