- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The authors propose a fine-tuning technique (Joint-State Relevance Learning - JSRL) to help speech retrieval models (models trained to match a pair of utterances based on a given criterion) to be less affected by co-occurent similarities in the given utterances. Tested on MOSS-Audio-4B-Instruct, compared to the original implementation as well as a set of Speech-aware LLMs and SSL models, on diverse set of Speech Emotion Recognition datasets, they show that most retrieval models choose by similarity, thus being easily confunded by co-occuring properties of a speech utterance. Their implementation show almost no lasting impact by co-occuring proprties of the speech.
Critical strengths
The augmentation technique is quite straightforward, well detailed, with obvious results.
Critical weaknesses
None
Questions for authors
With the test set being a composite of 2 acted and 1 naturalistic speech emotion recognition datasets, some differences mayb be expected in behaviour. Could you add details on the balancing operations that were performed between those sets, or provide dataset-specific results?
2 comments
Thank you for the per-dataset additions
I aknowledge the per-dataset additions in the appendix, which ease the issue with the blind dataset mixing during the evaluation, and thank the authors for their answer.
We sincerely thank you for your time and positive assessment. We have uploaded a revised PDF, with text changes highlighted in blue.
Q: Balancing across datasets and dataset-specific results.
We balance the three datasets at the aggregation stage. Each metric is first computed within each dataset, weighting all queries equally, and then averaged with equal weight over the three datasets, so each dataset contributes equally to the aggregate regardless of its size (App. A.7). On individual datasets, absolute scores differ, but JSRL achieves the highest Conflict and SwitchExact scores for all three criterion pairs on every dataset (Appendix Tables 17–19). In multi-candidate retrieval, JSRL also achieves the highest mAP, NDCG@5, and Recall@5 on all three datasets, outperforming the strongest baseline, WAVE (Dynamic), in mAP by 1.6 (CREMA-D), 7.6 (MSP-IMPROV), and 3.2 (RAVDESS) percentage points. These results have been added to Appendix E.2 (Table 20).