- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces VoxMem, a benchmark for evaluating memory in large audio language models across multiple sessions. It covers semantic content, speaker identity, paralinguistic cues, and environmental sounds. The results show that models struggle with information beyond semantic content, especially as conversation histories grow longer.
Critical strengths
- Memory is an important topic, and memory in large audio language models remains underexplored.
- The benchmark goes beyond semantic content to cover speaker identity, paralinguistic cues, and environmental sounds. I appreciate this broader coverage.
Critical weaknesses
- I found the paper difficult to follow. It introduces many terms, making it hard to grasp for my first and second pass of reading.
- The data are synthesized through a complicated pipeline, which may introduce bias. It is also unclear how well the data reflect real conversations.
- The paper lacks cascaded baselines or toplines, such as various speech, audio encoders, ASR systems paired with an LLM.
Questions for authors
The authors should motivate more on how the memory operations are defined. Why these four? and why there four are important?