- Rating
- 3(−1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SPM-TTS, a structured prosody modeling framework for long-form text-to-speech synthesis. The method combines Segmental Information-Guided Generation (SIG), Semantic-conditioned Prosody Consistency Regularization (S-PCR), and neural codec representations to improve local prosodic transitions and global consistency. Experiments on AISHELL-3 and long-form Chinese texts demonstrate improvements in speech quality, intelligibility, and prosodic stability compared with several representative baselines.
Critical strengths
The paper addresses prosodic inconsistency in long-form TTS, an important challenge in speech synthesis. The distinction between local segment-level modeling and global prosodic consistency is well motivated.
Critical weaknesses
Several important aspects of S-PCR remain unclear, including the construction of the global prosodic anchor, the definitions of the smoothing and variance losses, and the computation of differentiable prosodic statistics from predicted codec representations. These omissions make it difficult to fully assess and reproduce the proposed method.
Questions for authors
no
Response to Reviewer kwG3: Clarification of S-PCR Optimization and Long-form Evaluation
We sincerely thank the reviewer for the thoughtful and constructive feedback, and for recognizing the value of long-form prosodic consistency and linguistic-segmentation-based regularization. We address all concerns below and acknowledge areas requiring clearer disclosure.
1. Long-form Evaluation and Reference Alignment
We first clarify our two evaluation settings: AISHELL-3 is used for standard objective evaluation, while long-form testing uses a separate held-out set of 40 Chinese passages, split into four length bins (200–500, 500–1000, 1000–1500, 1500–2000 words, 10 passages each) with manually verified coherent discourse structure.
We agree the manuscript insufficiently documents paired reference recordings, their provenance, and the temporal alignment pipeline for reference-based F0 and energy metrics. We will explicitly clarify reference sources, text-recording correspondence, alignment procedures, and duration mismatch handling, and distinguish reference-dependent acoustic errors from reference-free long-range prosodic stability measures.
2. S-PCR Optimization and Semantic-conditioned Offsets
We agree the S-PCR optimization path requires more precise explanation, especially with discrete codec tokens.
A key detail of Eq. (5) is that the semantic offset is predicted solely from text-side representations: δi=MLP([hi;hg;hi−hg]). It does not take generated segment prosodic statistics (Fi) as input. The shared semantic mapping captures systematic, semantically appropriate prosodic deviations, producing a semantic-conditioned target rather than uniform relaxation of the global constraint.
We acknowledge the optimization behavior and offset magnitude are under-explained. We will clarify the full optimization procedure, stop-gradient role, differentiable prosody computation (distinguished from offline WORLD/pyworld evaluation), and the gradient path to the acoustic generator, with component-wise ablations comparing zero offsets, semantic offsets, and global moment matching.
3. SIG, Existing Components, and Ablation Design
We do not claim SpeechTokenizer as a novel neural codec; it provides the acoustic representation for generation.
SIG’s contribution lies in integrating semantic segmentation, explicit boundary markers, segmental position encoding, and segment-conditioned acoustic prediction to organize local generation by linguistic structure, complementing S-PCR’s cross-segment consistency constraint.
We agree comparisons to existing segmentation and global prosody regularization lack detail. We will clarify Table 3 configurations (independent vs. progressive components) and add isolated and combined ablations of SIG, S-PCR, and the codec, including conventional global moment matching baselines, to disentangle individual effects.
4. Evaluation Metrics, MOS, and Reproducibility
We agree PESQ, STOI, and F0 metrics alone cannot fully capture pause placement, rhythmic continuity, or long-range prosodic drift. We will supplement evaluation with segment- and boundary-level measurements: pause detection, pause duration consistency, and long-range F0/energy drift, alongside length-stratified analysis across the four passage groups.
For MOS: 24 native Mandarin listeners provided 600 ratings across 25 unique samples (5 per system variant). We acknowledge limited stimulus diversity restricts the strength of perceptual conclusions. We will explicitly state this limitation, add confidence intervals and appropriate statistical analysis, and expand stimulus coverage where feasible.
We will also fully document training configurations: model architecture, semantic encoder, segmentation procedure, tokenizer settings, optimization hyperparameters, and all regularization terms (including smoothness and variance loss definitions).
5. Closing Remarks
We thank the reviewer for identifying these critical issues. The core motivation of SPM-TTS remains valid: local prosody should be semantic-conditioned, while long-form generation should maintain appropriate global consistency.
We acknowledge the original submission lacks sufficient implementation detail and targeted evaluation to fully validate this mechanism. We will improve methodological transparency, clarify the boundaries of current empirical evidence, and strengthen controlled evaluations accordingly.
We hope these clarifications address the reviewer’s concerns and demonstrate the promise of semantic-conditioned cross-scale prosody modeling for long-form speech synthesis.