Zero-Shot Singing Style Transfer via a Structured Lyrics-Melody Bottleneck

Zero-shot singing style transfer aims to reproduce a reference singer’s vocal identity and expressive delivery while preserving a song’s lyrics and melody. Existing conversion systems can retain unwanted traces of the original singer's style: the pitch trajectories and frame-level features used to preserve the song also encode how it was performed. We introduce $S^3$, a score-conditioned, singer-prompted singing framework built around a structured lyrics–melody bottleneck. This symbolic score represents timed notes, their lyric assignments, and groups of notes sung on the same syllable (melismas), separating score-level constraints from acoustic realization. Joint lyrics–melody transcription recovers this structure from full-mix audio (vocals with accompaniment) and supplied lyrics, allowing lyric evidence to guide note selection. A reference-conditioned generator explicitly encodes this structure and uses reference audio to guide pronunciation timing and vocal expression. The framework supports both synthesis from supplied scores and conversion from recovered scores without singer-specific fine-tuning. Experiments show higher mean style-similarity and naturalness ratings for both tasks, alongside improved note transcription, note–lyric assignment, conversion intelligibility, and predicted singing quality. Audio examples are available at https://scube2027iclr.github.io/demo/.