- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper investigates whether pretrained flow-matching and hybrid TTS models can be directly repurposed for training-free speech emotion editing. The authors first analyze the editability of pretrained speech flows and study where along the generative trajectory emotion editing is effective, as well as whether source-to-target velocity transport also carries non-emotional attributes. Based on these observations, the paper proposes SEmoEdit, which repeatedly computes a target-conditioned versus source-conditioned velocity difference during sampling and uses this difference to update an edited speech state. The framework supports emotion replacement, erasure, and intensity control, with additional speaker alignment and an emotion-bridging procedure for interpolation. The paper also introduces SEmoEditBench, a 600-case evaluation benchmark, and evaluates the method on F5-TTS, CosyVoice 2, and IndexTTS2 against training-based and activation-steering baselines. The paper claims that dynamic velocity transport provides stronger emotion editing than existing alternatives. The paper explicitly positions the contribution as the first training-free formulation of speech emotion editing via dynamic velocity transport.
Critical strengths
Clear presentation and broad experimental coverage
The paper is well written and easy to follow. The proposed approach is evaluated across multiple pretrained TTS backbones and emotion-editing tasks. Beyond the main results, the authors provide analyses of trajectory dependence, speaker leakage, and interpolation behavior, offering additional insight into the method.
A useful benchmark and diagnostic setup
SEmoEditBench covers emotion replacement, erasure, and intensity control across different speakers and datasets. This provides a useful testbed for examining emotion-editing behavior in pretrained speech models.
Critical weaknesses
Insufficient evidence isolating the contribution of dynamic velocity transport
The reported improvements are difficult to attribute specifically to the proposed state-dependent velocity transport. The strongest SEmoEdit results depend on explicit target-emotion audio references and speaker-converted target conditioning, while interpolation additionally requires reference retrieval and a second editing pass. The paper also reports a substantial performance drop when audio conditioning is replaced with text conditioning.
These factors make it unclear how much of the improvement comes from dynamic transport rather than the additional target information and preprocessing. A controlled comparison that retains the same target references and preprocessing but replaces dynamic transport with a simpler static transport rule would help isolate its contribution. This is particularly important given the similarity between the proposed source-to-target velocity-difference formulation and existing inversion-free flow-editing approaches.
Limited evidence for speech fidelity beyond emotion-editing performance
The evaluation focuses primarily on emotion-editing success, with less convincing evidence for intelligibility, speaker preservation, and overall speech quality. Several emotion metrics, including E-SIM, DES, and EIC-Emb, rely on emotion2vec representations. Since the interpolation pipeline also uses emotion2vec for bridge-reference retrieval, the intensity-control evaluation is not fully independent of the representation used by the method.
Furthermore, the main comparisons report relative WER/CER without absolute recognition error rates, making it difficult to assess the intelligibility of the edited speech. This concern is reinforced by the interpolation analysis, where the absolute WER/CER reaches 0.603 after bridging for IndexTTS2. SEmoEdit also shows relatively low speaker-similarity and UTMOS scores in several settings.
Taken together, the results demonstrate effective emotion modification, but provide insufficient support for broader claims of robust, high-fidelity emotion editing.
Questions for authors
-
Can the authors compare static and dynamic transport while keeping the backbone, target reference, speaker alignment, and preprocessing fixed?
-
How does the proposed transport formulation differ from prior inversion-free flow editing? An equation-level comparison would clarify the methodological novelty.
-
Can the authors evaluate intensity control using metrics independent of emotion2vec, which is also used for bridge retrieval?
-
Can the authors report absolute source and edited WER/CER alongside relative changes, particularly given the high errors observed after bridging?
-
How were the activation-steering baselines tuned, including steering strength, layer/token selection, and backbone-specific settings? Were official implementations used?
5 comments
Response to Reviewer bxsv (2/2)
W2/Q4 — Intelligibility, the reported 0.603 WER, and speech fidelity
We identified an English normalization error that penalized equivalent transcriptions (e.g., "eleven" vs. "11"). We corrected this using Whisper's standard English normalizer across all 11 systems, without changing audio or ASR predictions.
Crucially, the reviewer-cited 0.603 WER decreases to 0.000 on the same 450 IndexTTS2 bridged outputs. The main intensity-control source/edited WER similarly changes from 20.00%/61.11% to 0%/0%. These subsets contain only three distinct short sentences, so zero WER does not imply perfect perceptual fidelity.
We additionally report absolute source/edited WER/CER. Across three backbones, corrected English WER decreases from 16.35% to 7.53–11.25% for replacement and from 12.64% to 3.57–6.22% for erasure.
Regarding preservation, our matched-TEP evaluation demonstrates higher S-SIM than dots.tts.edit (Table 6). Human evaluation further shows competitive naturalness (IndexTTS2 N-MOS 3.88/4.15) alongside strong emotion-editing performance (ES-MOS 4.22/4.40). CosyVoice 2/IndexTTS2 replacement UTMOS also remains close to CoCoEmo (gaps of 0.077/0.081).
These corrected and complementary results strengthen the evidence for intelligibility and competitive speech quality while accounting for emotion-transfer strength.
Revision: Secs. 4.1–4.3, Tables 2–4 and 6, and Appendix E.3–E.4 provide the corrected scores, absolute error rates, and preservation analyses.
Q5 — Baseline implementations and tuning fairness
CoCoEmo uses the authors' official implementation and steering vectors, retaining their selected layers and default strength 3 without test-set retuning. Our EmoSteer-inspired comparison uses a mean-pooled activation-steering adaptation, with its differences from the original method explicitly documented.
We further evaluated CoCoEmo strengths 0–5 on 24 common inputs (5 calibration / 19 evaluation). Across this predefined sweep, the maximum observed CoCoEmo TEP on the held-out evaluation inputs is 0.176/0.211 for CosyVoice 2/IndexTTS2, compared with 0.637/0.724 for SEmoEdit on the same inputs.
SEmoEdit's emotion-transfer advantage therefore persists beyond CoCoEmo's default strength, supporting the robustness of our comparison across the tested settings.
Revision: Appendix E.5 and Table 30 provide implementation details, layer/token settings, and the complete strength sweep.
Dear Reviewer bxsv,
As the discussion period draws to a close, we would greatly appreciate it if you could revisit our responses and updated results, and share your thoughts on whether they have adequately addressed your original concerns.
Thank you for your time and consideration.
Response to Reviewer bxsv (1/2)
We thank the reviewer for recognizing our broad experimental coverage, trajectory analysis, and benchmark. We conducted additional controlled experiments and independent evaluations to further substantiate our contributions.
W1/Q1 — Contribution of dynamic velocity transport
First, we clarify that our main evaluation (Table 2) uses the same-speaker setting, where speaker alignment reduces to an identity mapping, requiring no voice conversion. Moreover, even with text conditioning (without target audio), SEmoEdit outperforms activation-steering baselines in emotion transfer (TEP 0.274/0.354 vs. 0.025–0.082).
To directly isolate dynamic transport, we conducted a controlled static-vs-dynamic comparison on 94 speaker-disjoint inputs per backbone, keeping source representations, target references, preprocessing, noise draws, decoders, and branch-query budgets identical. Static transport fixes the velocity-difference direction at a calibration-selected timestep. Neither variant uses additional voice conversion or emotion bridging.
At α=1.0, results are reported as Static / Dynamic:
Dynamic transport substantially improves emotion transfer and English intelligibility across all three backbones, while also improving speaker preservation for both hybrid backbones. These results establish its contribution independently of reference information and preprocessing.
Revision: Sec. 4.3, Table 5, and Appendix E.4 provide results at α=0.5/1.0 and detailed protocols.
W1/Q2 — Relationship to FlowEdit in image domain and methodological novelty
Our formulation builds on FlowEdit, as explicitly acknowledged in Sec. 3.1. Under the time transformation s=1−t and v(q,t;c)=−V(q,1−t;c), its update becomes:
qs=(1−t)ϵ+tx,qt=qs+Et−x,
E˙t=Eϵ[v(qt,t;ctgt)−v(qs,t;csrc)],
which corresponds to our formulation at α=1,τ=0.
Our contribution lies in systematically investigating and enabling inversion-free editing for pretrained speech models, rather than proposing a fundamentally new ODE. Specifically, we identify and address three speech-specific challenges:
Together, these findings establish practical training-free emotion editing across distinct speech architectures.
W2/Q3 — Independent intensity-control evaluation
We added an evaluation using the fixed SUPERB wav2vec2 emotion recognizer, independent of emotion2vec and unused in generation, retrieval, or tuning. All eight systems were evaluated on 90 common RAVDESS/CREMA-D curves per system. We separately measure monotonic ordering (M↑) and endpoint distance to the target emotion distribution (TV↓).
SEmoEdit achieves better target proximity than CoCoEmo on both backbones, with a statistically supported improvement for CosyVoice 2. Although monotonic ordering is more mixed, independent human evaluation also favors SEmoEdit (EIC-MOS 2.80–3.65 vs. 1.65 for CoCoEmo).
These complementary evaluations support improved target-emotion control beyond emotion2vec-based metrics.
Revision: Sec. 4.2, Appendix E.2, and Tables 22–23 provide all eight systems, endpoint accuracy, and confidence intervals.