- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper investigates feature adaptation for unseen audio mixtures and proposes a lightweight residual module (CompAdapt) fitted on frozen speech/audio encoders to align joint mixture representations with combined class-mean prototypes.
The paper itself is hard to read, with no clear goal and little real world applications and its core empirical findings reveal that the method offers almost no benefit in realistic settings. Under proper calibration-based checkpoint selection, performance gains over a baseline identity map completely vanish. Furthermore, the paper's own control experiments reveal that random or constant targets match the proposed gains, demonstrating that any observed boost stems from simple feature shifting rather than compositional semantics. Lastly, the dataset scope is rather limited, only using the 2000-sample ESC-50 and ~20000 sample SONYC-UST-V2 dataset, even though the proposed method seems to be general enough to be used in other domains like music (stem separation). Thus, I recommend a reject.
Critical strengths
- The paper has a somewhat broad coverage using four audio encoders, two tasks and a very long and extensive appendix.
Critical weaknesses
- The study uses four audio encoder models, two of which (wav2vec2, HuBERT) have no known audio recognition/classification capability. Why are even HuBERT and wav2vec2 considered as reasonable baselines? These models are trained mainly for ASR, thus should perform bad for audio events, which the paper also shows.
- Dataset for training and evaluation is very limited. ESC-50 might be a reasonable test set for some purposes, but given the general nature of the paper, authors could also run separation experiments for Music on i.e., musdb18 and gtzan. Overall the experiments seem lack luster due to the small number of datasets as well as samples ( 2000 in ESC-50 ).
- To be honest, I read the paper multiple times over but still do not understand the goal of the work, i.e., what actual problem needs to be solved here? The proposed scenario of having "a downstream training set may contain dogs, rain and keyboards without containing their joint mixture" is very artificial.
- Proposed approach is statistically indistinguishable from identity under calibration-selected checkpoints. Also the SONYC gains vanish under sensor composition.
- The paper's own controls (Appendices F/H) show random, constant, and even rotation/mean-shift targets reproduce the fixed-endpoint gains, which lessens the impact of the paper.
- Paper is extremely dense and hard to understand with dozens of appendix tables and the novelty over the SSLAM/CARL-style baselines is limited (2-layer residual MLP + anchor).
- BEATs/EAT were pretrained on AudioSet clips, which contains mixtures of most ESC-50 events, so the encoders already know those mixtures, thus the claim of "unseen combinations" is only artificial.
Questions for authors
I still am missing a motivation for this work, i.e., what kind of problem are the authors solving? In which real-world scenario would we encounter a problem that this work would help solve?