- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies the problem of spatial sound understanding with audio LLMs. It proposes the addition of a multi-format (mono, binaural, FOA) spatial audio encoder to a Qwen3-Omni-based LLM backbone. It follows prior work such as Spatial-Omni by having two audio encoders, a mono semantic encoder and a multichannel spatial encoder, and interleaves each spatial token with its paired semantic audio token, using a newly proposed SMRoPE positional encoding for the interleaving.
The paper presents a multi-stage training scheme (encoder pretraining followed by projector training followed by join SFT with the LLM and then GRPO) which uses a newly proposed benchmark, SpheraQA, for evaluation. SpheraQA contains 20k questions spanning audio perception, spatial relations, and spatial reasoning. Experimental results show that the proposed model generally outperforms Spatial Omni 7B and Gemini 3.1 Pro on the proposed benchmark, as well as the SO-Bench and TWNM benchmarks.
Critical strengths
-The paper studies an important and timely problem (spatial sound understanding with LLMs)
-The paper proposes a new model for spatial sound understanding, which seems to outperform prior models
-The paper also introduces a new spatial audio encoder which can handle multiple channel formats (stereo binaural and FOA) which is a useful innovation
Critical weaknesses
-
The evaluation conditions unfairly favor the proposed SpheraAudio model. SpheraAudio is fine-tuned with in-distribution spatial CoT data from the same generator used to construct SpheraQA test, but the baseline models are applied to SpheraQA in a zero-shot fashion. Additionally, Gemini 3.1 Pro receives only the mono W channel (so it cannot access the directional cues the tasks require). A fairer comparison would include Spatial-Omni or another baseline model fine-tuned on the same SpheraQA training data, as well as other very close architectural competitors (PhaseCoder, ST-AudioLM, BAT)
-
Some ablations are invalid. For example, Table 4c switches the RoPE type at inference (rather than training) on a model trained with SMRoPE
-
Regarding the benchmark itself, the "Spatial Reasoning" category contains only 120 questions (making statistical significance testing necessary), and according to appendix B, ~12k of the questions in the benchmark (more than half) are derived from only 168 STARSS23 recordings, so the majority of the evaluation samples are not independent from one another
-
Some of the baseline performances are strange. Gemini on mono beats SpheraAudio on lateral position (44.6 vs 41.5) and motion status (73.9 vs 73.4), which should be impossible for true lateral cues unless the questions can be solved semantically, or the metrics are noisy.
-
There is only a very vague discussion of what training data was used to train the model, and the majority of the important details about this are missing.
Questions for authors
- Can you present experimental results in Table 1 for baseline models fine-tuned on identical data as SpheraAudio?
- Can you state whether any SO-Bench or TWNM training splits were used in SFT/RL for SpheraAudio?
- Can you describe what data was used to train the model? Please provide a table that details the training data used for each stage of training, including at minimum the data sources and number of examples/total duration of audio.
- What are your release plans for the training data?