W1. Noticeable artifacts, degraded speech naturalness, and inconsistent emotion controllability
After listening to the provided audio samples, I observed noticeable artifacts and degradation in speech naturalness across multiple examples.
For instance:
Sad (Chinese): "带女友出去好好吃上一顿。" contains noticeable artifacts in the segmented-α result.
ESD-English: "How suspicious this fox." exhibits audible artifacts, particularly around "fox".
Sad: "The end part is fish mouth." contains noticeable artifacts toward the end of the utterance.
Several generated samples exhibit artifacts that are absent from their corresponding neutral references, raising concerns about pronunciation fidelity and content preservation.
Furthermore, emotion intensity control appears inconsistent across samples. For example, in the ESD-English sample "How suspicious this fox.", the perceptual differences across α values are relatively subtle and difficult to distinguish.
Overall, the qualitative results do not fully support the claimed preservation of speech naturalness and reliable emotion intensity controllability.
W2. Limited methodological novelty and insufficient justification for post-hoc WORLD interpolation
Although the paper proposes semantic-emotion disentanglement through twin flow-matching networks, the actual emotion intensity control is achieved through a heuristic, post-hoc interpolation of WORLD acoustic features extracted from fully generated neutral and emotional speech.
It is unclear whether the proposed velocity decomposition is essential for achieving such controllability. In principle, a similar interpolation strategy could be applied to neutral and emotional speech generated by existing emotion-controllable TTS models.
Furthermore, since the model explicitly learns an emotion-residual velocity field, direct modulation of this component during flow-matching integration would be a more natural use of the proposed decomposition. Instead, the method requires generating two complete speech candidates before performing acoustic interpolation.
This design appears computationally inefficient and leaves the benefit of the proposed velocity decomposition insufficiently justified.
More broadly, the paper does not clearly establish how the proposed decomposition differs from existing residual-based or disentangled flow-matching approaches, beyond its specific application to emotional speech synthesis.
W3. Insufficient ablation studies to justify the proposed Twin-DiT and conditioning architecture
The proposed architecture employs two independently trainable DiTs, each conditioned on separate adapter outputs, μ_A and μ_Δ. However, the paper does not sufficiently investigate whether these components are necessary for the claimed semantic-emotion disentanglement.
In particular, there is no ablation examining the contribution of the individual conditioning signals, or whether using the same conditioning representation for both branches would yield comparable results.
Moreover, the two DiTs contain approximately 666M trainable parameters in total. Without a comparison against a simpler single-DiT architecture with different neutral/emotional conditions, it is difficult to determine whether the additional architectural complexity is justified.
Although Table 4 investigates the orthogonality coefficient, the model still exhibits substantial emotion intensity controllability without the orthogonality regularizer. This further raises questions about which components are responsible for the observed performance.
W4. Insufficient analysis of the time-dependent velocity fields
The paper lacks a detailed analysis of how the semantic and emotion velocity fields behave across flow-matching timesteps.
According to Section 3, both branches share a noised state Y_t derived from emotional speech, while the neutral branch is supervised with a velocity target corresponding to neutral speech.
This raises a question about the consistency between the training state and the target velocity for neutral generation. In particular, at later timesteps, when the shared state approaches the emotional speech, predicting a velocity corresponding to neutral generation may become challenging.
A timestep-wise analysis of the prediction errors, velocity magnitudes, and relationship between the two velocity fields would help clarify how the intended decomposition is learned throughout the generative process.
W5. Insufficient quantitative evaluation of localized emotion control
Localized emotion intensity control is a central claim of the paper. However, the experimental validation is largely limited to a few qualitative demonstrations and mel-spectrogram visualizations.
There is no systematic quantitative evaluation of temporal localization accuracy, preservation of unaffected regions, or perceptual naturalness at emotion transition boundaries.
This limitation is particularly concerning given the audible artifacts observed in some segmented-intensity samples.
W6. Limited training scalability due to paired-data requirements
The proposed training framework relies on paired neutral and emotional recordings with identical linguistic content and speaker identity, followed by explicit temporal alignment.
Such paired recordings can be expensive to collect and may not be readily available for large-scale or in-the-wild emotional speech datasets.
Although the authors evaluate generalization on out-of-domain datasets, the reliance on paired data during training remains a practical limitation of the proposed framework.
Minor Weakness
Presentation quality of the architecture overview
Figure 1 could be improved, particularly in terms of arrow organization and the relationships between the conditioning modules, the two velocity branches, and the subsequent inference pipeline.
Response to Reviewer xSkk
Thanks to you the reviewer to spend efforts to provide the suggestions. We have respond below point by point to clarify potential misunderstandings.
W1. Effect of orthogonalityEmoScale decomposes the flow-matching velocity field itself into a semantic and an emotion component, learned jointly from paired data and kept frame-synchronous, so that emotion becomes an additive, per-frame quantity that can be scaled globally or locally. the velocity decomposition provides frame-synchronous, interpretable emotion and semantic components, while WORLD is used only to remove the comb-like artefacts of direct mixing. Replacing our endpoints with independently generated ones under the same WORLD interpolation yields a flat intensity axis and breaks the content at intermediate α, which isolates the contribution of the decomposition.
To verify this, we replaced our endpoints with neutral and emotional versions generated independently by an existing emotional TTS model (CosyVoice3-instruct, with the real neutral recording as the speaker prompt), aligned them by DTW, and applied exactly the same pitch-aligned interpolation (Table below). Independently generated endpoints do not provide a clean emotional direction: the generated emotional end is not more emotional than the neutral end, the intensity axis is flat. They are also not time-synchronous (duration ratio 1.55), so the content breaks down at intermediate α (CER 0.26 at α = 0.5), and segmented α does not move the intensity in the intended direction. EmoScale, in contrast, yields a monotonic intensity axis and correct localised control.
Endpoint-source ablation on the RAVDESS utterances of Table 3, using identical pitch-aligned interpolation.
W2. Limited training scalabilityWe view data efficiency as a strength of the framework rather than a limitation. EmoScale is trained on only neutral–emotional pairs from ESD, yet it generalises to three out-of-domain datasets (RAVDESS, CREMA-D and IEMOCAP) with different speakers, recording conditions and emotion distributions (Table 1). The current scale is therefore sufficient for the task, and scaling to large in-the-wild corpora is beyond the scope of this paper.
Q1. Disentanglement verificationYes. In EmoScale, orthogonality is the constraint that realises the semantic–emotion separation in training, and the separation is also evidenced empirically. (i) Figure 2 shows that the emotion velocities cluster by emotion rather than by speaker or sentence, while the semantic velocities remain close to each other. (ii) Generating with the semantic component alone yields natural neutral speech, and adding the emotion component yields emotional speech whose intensity increases with α (Table 1, Figure 3).
Q2. Clarification on restruction processThe final waveform is produced by the vocoder, as shown in Figure 1. The “waveform” at Line 268 refers to an intermediate step: WORLD resynthesises the two endpoints (acquired by flow-matching with two velocity) with a shared F0 contour; the resynthesised signals are then converted back to mel-spectrograms and mixed according to α, and the mixed mel-spectrogram is passed to the vocoder to produce the final audio. We will make this clear in Section 3 in camera-ready version.
In conclusion, we sincerely thank the reviewer again for their constructive feedback and thoughtful consideration. We hope that our clarifications have satisfactorily addressed the concerns raised and resolved the misunderstandings. If these clarifications have changed the reviewer's initial assessment, we would greatly appreciate your consideration of an upward revision of the overall score.