- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Long-Term Memory-Guided Audio Enhancement (LTM-AE), which builds a category-specific PCA memory from clean audio tokens and uses a user-specified target category to select a memory at inference time. It reconstructs and interpolates audio tokens before they enter a frozen audio-language model. Experiments on 20 sound categories and three models report gains in constrained and free-form classification, as well as target-conditioned retrieval. A separate speech-transcription experiment adds a learned token-level gate and reports improved WER on Qwen2-Audio.
Critical strengths
- The method is clearly specified and does not update the audio-language model’s parameters. The rank and interpolation weight are calibrated on a separate validation set, and the paper reports results across multiple memory sizes. The authors also discuss inference cost and compact memory storage.
- The empirical study covers three model interfaces and 20 sound categories. It includes class-wise analyses, a PCA/SVD solver cross-check, and an absent-category control. Improvements in language outputs provide useful evidence beyond representation-level retrieval.
Critical weaknesses
- The main interpretive question is whether LTM-AE improves target perception or steers the representation toward the selected category. The enhancement uses that category’s PCA memory, and calibration minimizes error against paired clean target embeddings. Retrieval is then measured against clean category examples, so it should be viewed as representation-alignment evidence rather than independent confirmation of recovered perception. This does not negate the language-output gains, but the paper lacks wrong-category-memory and simpler matched controls, such as a category mean/prototype or a dimension-matched subspace. The absent-category control tests piano memory on 100 examples with Qwen2-Audio only.
- The main benchmark is a controlled synthetic setting: three-second, mono mixtures at a fixed −10 dB target-to-interference ratio, with the target category known and present. The results establish effectiveness in this setting but leave performance on natural mixtures and absent or mis-specified targets unclear. The speech result is a separate learned extension: ungated LTM-AE worsens WER, while the gate improves it in a single-model experiment.
Questions for authors
Q1: Do wrong-category memories or simpler matched category-conditioned controls produce similar gains? Results from these controls would help determine how much of the improvement is specific to the proposed PCA memory.
Q2: Is the clean retrieval gallery source-group-disjoint from the memory, validation, and evaluation recordings? For speech transcription, are training, selection, and test sets disjoint by speaker and original recording?
Follow-up on our response
Dear Reviewer,
Thank you again for your valuable comments. As the discussion period is nearing its end, we would like to kindly ask whether our responses have adequately addressed your concerns. Please let us know if any further clarification is needed. If our responses have resolved your concerns, we would sincerely appreciate your consideration in updating your evaluations accordingly.