- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper try to simulate transferring emotional classification model to seen tonal languages. To do that, this model utilizes source model as trained with English dataset (MELD dataset), and transfering it into Mandarin (Emotiontalk dataset).
Critical strengths
None.
Critical weaknesses
-
While the paper mentioned that "it does not imply that Mandarin is generally a low-resource language or reproduce the absence of transcripts and pretrained models in many low-resource settings.", but I am not sure it is adoptable to other languages. From the beginning, ToneNet get "syllable" as an income. However, for low-resource languages, there is no meaningful syllable extractor. Also, after syllable extraction, it calculates surface tone by sandhi processing, but it also might not eligible to low-resource languages. In addition, they mentioned about low-resource language, but their "unsupervised dataset" is larger than English dataset.
-
Why ANCHORED FUSED GROMOV–WASSERSTEIN ALIGNMENT is utilized? In section 4.3, it just mentioned AFGW, and did not mention why it is meaningful target. Also few variables, such as , are not defined.
-
The figures appears to heavily AI-generated and does not provide any additional visual insights. Figures 1, 2, and 3 all appear to be heavily AI-generated.
-
Results are too weak to demonstrate. All results shows low accuracy than "Always neutral". Meanwhile, all other models' accuracy is lower than 40% for 7-class classification, I am concerning that showing weighted F1 and macro F1 is meaningful. In addition, the proposed method does not have meaningful differences with DANN or MMD (citation is needed).
-
Missing confusion matrix. While many previous studies on emotion classification/generation present confusion matrix, this paper does not include one.
-
Writing issue The method section is largely diffult to follow, as the authors describe the methods without sufficiently explaining the motivation behind their design choices.
Questions for authors
Please refer to the Weakness section.
3 comments
Author Response to Reviewer qmYs
Question 5:
Answer: Thank you, this is a good suggestion. We added confusion matrices (row-normalized, pooled over three seeds) in Appendix Table 6: for Mandarin under Protocol A (source-only, MMD and TD-AFGW) and for Vietnamese (source-only and TD-AFGW). They make the failure easy to see: the Mandarin TD-AFGW model predicts neutral for 98.8–99.8% of the clips of every class, while MMD spreads its predictions much more evenly. They are referenced in Section 5.2 (“Classes”).
Question 6:
Answer: Thank you, we accept this criticism. Within the page limit we made targeted changes: a motivation for anchored FGW (Section 4.3), definitions of α, ε and β1–β3 (Sections 4.3 and 4.5), a clearer description of how the tone-swap contour is built (Section 4.4), and a rewritten paragraph on the decision rule (Section 4.5, “Decision rule”). Most of the revision effort went into the evaluation, because the new results changed what the paper can claim. Section 5.3 now explains, component by component, what each part was meant to do and what we actually measured. We will keep improving the clarity of the method section for the final version. Thank you again for your time and for the detailed review.
Response to the author's comments
Dear Authors, Thank you for your revisions and responses to my comments. However, I still have several concerns that have not been adequately addressed
- Thank you for including additional tonal languages, but the results on the Vietnamese data do not appear to demonstrate meaningful improvements from the proposed method.
- I am still struggling to understand the motivation for applying AFGW.
- Whether the figures were created using Tikz or AI-generated tools was not the main point of my comment. My primary concern is that these figures do not "provide additional visual insights" with the pattern of heavily AI generated figures. I recommend to revise all figures from ground for next submission. 4/5. As shown in table 6, 99% of the proposed model's predictions are biased to the neutral. Given this severe class imbalance in the predictions , I do not believe the reported results demonstrate a meaningful improvement from the proposed method.
Therefore, I will maintain my current score.
Author Response to Reviewer qmYs
Question 1:
Answer: Thank you for raising this. We agree that it is a real limitation, and you are right that our “low-resource” framing was too strong. • Syllables and tone sandhi. ToneNet does need syllable-level text processing. For Mandarin we use pinyin with rule-based sandhi, which matches the human-annotated surface tones of AISHELL-3 on 96.8% of syllables (94.1% without the sandhi rules) (Appendix A). Many low-resource languages do not have such tools, so the method in its current form is not directly applicable to them. Section 3 now states that the method still relies on transcripts, linguistic tone processing, speech alignment and neutral target-language speech, and that a setting without these resources is outside what we evaluated. • A second tonal language. To check whether the idea carries over at all, we repeated the full pipeline on Vietnamese, where syllables are written separately and tones are marked in the spelling, so no syllable extractor or sandhi rules are needed (Section 5.4; Appendix Tables 16 and 17). The method did not help there either: it is 1.44 W-F1 below a fairly selected source-only model. • Dataset size. You are right that the unlabeled EmotionTalk pool (12,551 clips after removing the holdout) is larger than the MELD training set (9,966 utterances), and Mandarin is not a low-resource language. We added a label-budget experiment: fine-tuning with only 1% of the target labels (126 clips) already reaches 53.22 W-F1 on test, far above every zero-label adaptation method, and zero-shot emotion2vec+ reaches 46.85 (Section 5.5; Appendix Table 18). Section 6 now states clearly that, on this benchmark, a small labeled budget or an off-the-shelf model is the better practical choice.
Question 2:
Answer: Thank you, this is a fair point. We added a short motivation at the start of Section 4.3: Gromov–Wasserstein matching can compare the structure inside each domain without a shared feature space, but it is ambiguous up to symmetries, while a few fixed cues that are assumed comparable across languages can break those symmetries. We also now define α (the weight of the relational term against the anchor term) and ε (the entropic regularization) right after Eq. (6), and β1–β3 right after Eq. (12). The variable name was cut off in the review text, so if there is another symbol we missed, we would be glad to fix it. We also want to be honest that the new experiments do not support this motivation in practice. The coupling carries almost no class information (its soft labels are correct for 35.3% of development clips, against 35.4% for always predicting neutral), and the anchors are only weakly comparable between MELD and EmotionTalk (Section 5.3, “Coupling soft labels carry no class information”; Appendix Tables 4 and 15).
Question 3:
Answer: Thank you for the honest feedback. Figures 1–3 are schematic diagrams of the method, drawn as TikZ code; as stated in our AI use statement, generative AI helped with the figure code and layout. We agree that they mostly restate the text. In this revision we kept them as method overviews, but we added figures that carry evidence and are drawn directly from the run outputs: Figure 4 (development W-F1 during training, showing the decline in Stage 2 and the collapse in Stage 3) and a re-drawn Figure 5 (ablation results with reference lines), both in Appendix C. If you feel that Figures 1–3 do not earn their space, we are happy to simplify or remove them in the final version.
Question 4:
Answer: Thank you. You were right, and the re-run shows it even more clearly. Under the corrected protocols the method does not beat simple baselines. • Accuracy against always-neutral. Always-neutral reaches 41.52% accuracy on the test set. Without target labels, TD-AFGW reaches 41.47% only because it predicts neutral for more than 99% of clips (Appendix Table 6). With development-based selection it reaches 45.01%, against 42.37% for a source-only model selected the same way and 45.34% for MMD. • Metrics. Because accuracy is dominated by the neutral class (41.5% of the test set), we report weighted and macro F1 next to two simple references, an always-neutral predictor and a class-prior random predictor (Table 1). We also give per-class F1 (Appendix Table 5), confusion matrices (Appendix Table 6) and paired bootstrap intervals over clips and over actors (Appendix Table 7). • DANN and MMD. All baselines were re-trained under the new protocols with three seeds. TD-AFGW is not better than them: under Protocol B, MMD reaches 46.09 W-F1 on test against 43.76 for TD-AFGW (difference −2.33, clip-bootstrap interval [−3.86, −0.85]), and DANN is on par (43.13). We added the citations for DANN, MMD and OT-based adaptation where the baselines are introduced (Section 5.1, “Baselines and metrics”). We removed all claims of improvement from the Abstract, Section 1 and Section 7.