- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors of the article propose a method to measure the disantenglement of speech tasks by a given SSL network. This measure is operated at the frame level for ASR, SER and SID tasks, on the IEMOCAP dataset, by leveraging a Linear layer classifier head from a single layer. Networks leveraged include Wav2Vec2.0 and WavLM. Experiments show the frame-level separability is low for all tasks, showing a non-independence between the tasks. Additional experiments show the proposed separability metric can be improved with finetuning, but no experiment show the degradation on the base performances of the foundational models.
Critical strengths
The experimental process is explained in enough details to be reproducible, using a known dataset and known models.
Critical weaknesses
Some claims of the authors should be verified and provided adequate citations, like: "Most real-world data is believed to be generated based on a mixture of multiple latent factors. One of the working assumptions is that the changes among them are independent." Does this imply variations in emotions are independent from the linguistic content? or that the speaker identifying information would be independent from the dialect of a given speaker? Citations or clarification are needed.
The evaluation of a given metric on a dataset as small as IEMOCAP (limited set of utterances, limited set of speakers) critically affects the results presented.
The frame-wise evaluation of the separability is limited for the selected tasks. At least for the SID task, the speaker prosody plays an important part in speaker recognition tasks, and cannot be guessed from a single frame, which is why in standard benchmarks studying SSL models capabilities (like SUPERB), not all classifier heads are linear layers.
In general, the novelty of the technique is limited, but as I didn't find a specific reference showing a paper doing the exact same thing, I would put that as a minor weakness.
Questions for authors
Could you please adress the main weaknesses mentionned early?