- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces DelexWav, a codec trained to convert recordings into playable waveforms that aim to remove lexical content while preserving prosody. Extraction requires neither text nor phoneme alignment. The waveform can be edited locally and used with text to condition a separately trained speech generator. In same-text reconstruction, it achieves lower word error rates and more accurate phoneme timing than F0+Energy and FACodec, but some prosody metrics are weaker, cross-text generation is unreliable, and editing increases recognition errors.
Critical strengths
-
Extraction requires only a recording, without text or phoneme alignment. In Table 1, energy correlation is 0.85 versus 0.67 for MBROLA m, and combined WPT correlation at the 0.125-second scale is 0.62 versus 0.32. The English-trained model also achieves F0 correlations of approximately 0.95 on Mandarin and German, supporting some cross-lingual prosody preservation.
-
With the same backbone, data, and training schedule, same-text reconstruction achieves a WER of 7.18%, compared with 35.52% for F0+Energy and 65.91% for FACodec, with lower phoneme boundary error as well.
Critical weaknesses
(1) The downstream utility of the proposed representation remains insufficiently established. The same-text reconstruction results in Table 4 demonstrate useful improvements in phone timing and word accuracy, but primarily validate reconstruction fidelity rather than establish a practical use case for the representation. A particularly relevant application would be transferring the prosody of utterance A to a different target text B. However, the exploratory diagnostic in Appendix C.5 reports a target-text WER of 103.97% across 144 outputs from 48 A/B pairs and three seeds; only 59 outputs are closer to B than to A, while 75 are closer to A. These results indicate a severe failure to synthesize the intended target content. Although the authors acknowledge this limitation, it substantially restricts the demonstrated usefulness of DelexWav as reusable prosodic conditioning. The paper therefore leaves unclear which downstream application would materially benefit from the proposed audible representation beyond reconstructing the source utterance.
(2) The editing experiments do not establish a compelling advantage over simpler alternatives, and the edits substantially degrade content accuracy. The demonstrated operations are limited to local pitch shifting and time stretching, both of which can be implemented using explicit acoustic controls or conventional waveform processing. The paper’s own comparisons do not establish an advantage for DelexWav on these operations: Table 10 reports WERs of 20.61% and 24.82% after time scaling by 1.25 and 0.80, compared with 11.54% and 15.00% for conventional waveform stretching, which also better preserves pitch and energy contours. Furthermore, Table 5(b) shows that modest pitch shifts of +2 and −2 semitones increase WER from 10.35% to 29.95% and 46.29%, respectively. Thus, a directional response to an edit does not by itself demonstrate practically useful control. Without evidence of a distinct editing capability or a better trade-off between control and content preservation, the practical significance of introducing a learned audible waveform remains limited.
My main concern is that the practical value of the proposed audible prosody representation remains unclear. Same-text reconstruction mainly serves as a validation experiment, while the demonstrated pitch and duration edits are achievable with other existing simpler methods and introduce substantial content errors. Cross-text prosody transfer would be a more compelling application, but the reported results are highly unreliable, leaving the downstream usefulness insufficiently established.
Questions for authors
-
What is the primary downstream use case for DelexWav beyond same-text reconstruction and local pitch/duration editing? Which reported results demonstrate a benefit of its audible waveform formulation over explicit acoustic features or conventional waveform processing?
-
What causes the severe target-text fidelity failure in Appendix C.5: source-specific timing encoded in DelexWav, the generator’s same-text reconstruction training, or another limitation? Are controlled results available showing how the same generator handles text B without A’s DelexWav condition, or with B’s own condition, to distinguish these explanations?
-
Given the stronger contour preservation and lower WER of conventional waveform stretching in Table 10, what practical advantage does editing DelexWav provide? Are comparable pitch-editing baseline results available to establish whether the audible representation improves the control–fidelity trade-off?
-
Why do pitch changes of only ±2 semitones cause the large WER increases in Table 5(b)? Does the available failure analysis identify a setting in which local prosody can be meaningfully modified while maintaining content accuracy close to the unedited output?