- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper first curates AuralSpeech, a corpus for spatially localizing and transcribing speakers. It then introduces AuralScene, a multiple-choice question-answering dataset for spoken-content understanding, spatial grounding, and relational reasoning. Unlike existing datasets that primarily study these capabilities in isolation, AuralScene explicitly couples spoken content with spatially grounded sound events to support joint scene-level reasoning. To enable its systematic construction, the paper proposes Spatial Auditory Graph, a structured representation of spatial auditory scenes that enables the generation of diverse and controllable spatial audio scenes and question-answer pairs. Finally, it develops AuralListener, a baseline spatial LALM for the proposed tasks that combines monaural and spatial audio token streams, enabling controlled analysis of their complementary roles in spatial auditory understanding.
Critical strengths
- The paper introduces spoken-content-aware spatial auditory scene understanding, jointly interpreting spoken content, sound events, and their relationships within a 3D auditory scene.
- It presents AuralSpeech and AuralScene, and proposes Spatial Auditory Graph, a structured scene representation that enables automatic generation of spatial audio scenes and question-answer pairs.
- It develops AuralListener, a spatial LALM baseline combining monaural and spatial audio token streams for the proposed tasks.
Critical weaknesses
- The model comparison on AuralScene in the main table is unfair. The authors' own AuralListener is directly fine-tuned on the 420k AuralScene training QA pairs in Stage 3, while all the other audio language models in the comparison are tested zero-shot without training on this dataset. The high score in the main table mainly comes from training on the in-domain training split, which does not prove the model architecture itself is better than others.
- All the audio in AuralScene is generated from simulated rooms in SoundSpaces 2.0 and TTS synthesized speech from GPT-SoVITS, with no real recorded binaural audio at all. Although the paper tests on STAR-Bench and BiDepth, neither of those benchmarks tests reasoning that combines spoken content with sound events, so it remains unclear whether this method works on real binaural recordings.
Questions for authors
- After AuralListener finishes Stage 2 training on SpatialSoundQA and AuralSpeech only, before fine-tuning on the AuralScene training set in Stage 3, what score does it get when tested directly on AuralScene for a fair comparison with other zero-shot models?
- Can you fine-tune at least one open-source spatial audio language model baseline on the AuralScene training set and compare under the same training condition?