- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes an invariance–equivariance learning framework for spatial audio representations, using rotations of FOA recordings as a label-free training signal. The central idea is to encourage the representation to remain invariant to spectral nuisance transformations such as noise, gain, and time-frequency masking, while changing predictably under rotations of the acoustic scene. The authors instantiate this framework with three approaches: SIE, SimCLR-EquiMod, and a predictive seq-JEPA-based method that aggregates multiple rotated views into an unsplit latent representation. All methods use the same Conformer backbone and the same FOA-based input representation, allowing the authors to focus on the effect of the learning objectives. The learned representations are evaluated through linear probing, SELD, and spatial audio-language tasks. The proposed methods encode spatial information substantially better than several supervised and self-supervised baselines in linear probing. The paper further analyzes the latent geometry and shows that the seq-JEPA representation forms smooth trajectories under continuous rotations.
Critical strengths
-
A clear and well-motivated self-supervised formulation for spatial representation learning.
The use of analytically generated FOA rotations as a label-free signal is elegant and particularly well suited to spatial audio. Rather than treating rotation merely as data augmentation, the paper explicitly asks the representation to change predictably under scene rotation while remaining stable to spectral nuisance transformations. This gives a clear inductive bias for preserving spatial information without requiring DoA annotations. -
Strong transfer to realistic downstream spatial tasks.
The learned representations show meaningful benefits beyond the synthetic single-source probing task. In particular, frozen seq-JEPA outperforms supervised EINV2, and it also performs well on ScaperSynth and TAU-NIGENS without retraining. The improvements in spatial audio-language transfer are also substantial. These results support the claim that the learned representation contains useful spatial structure that transfers to downstream tasks.
Critical weaknesses
- The demonstrated scope is primarily limited to FOA spatial audio.
The core transformation mechanism relies on the exact Wigner-D rotation operation for FOA, and all reported representation learning experiments use FOA-based inputs. This is a particularly clean setting for studying the proposed idea, but the broader claim of a general-purpose spatial audio representation is therefore somewhat stronger than what is directly demonstrated. It remains unclear whether the same learned representation principle transfers to other spatial audio formats or microphone-array recordings where the corresponding transformation is not as direct.
Questions for authors
Q1. Potential generalization beyond FOA spatial audio. How much does the proposed framework depend on the FOA representation? Since the rotation operation is particularly clean and exact for FOA, could the authors clarify whether the same formulation can be applied to higher-order ambisonics or microphone-array recordings?
2 comments
Follow up on our response.
Dear Reviewer 7h83, thank you for your feedback and the question on scope.
We clarified that our rotation is the first-order case of the Wigner-D action, extending to higher orders by adding blocks, and that array recordings can reach this pipeline through ambisonic encoding, as our STARSS23 and TAU-NIGENS results already do. We would welcome your view on whether this resolves the concern.
Rebuttal to Reviewer 7h83
We are grateful for the reviewer's careful reading, and encouraged that the central idea came through as intended: rotation used not as an augmentation but as a signal the representation must respond to predictably (S1), with benefits that carry over to realistic downstream tasks (S2). The question of scope is a fair one, and answering it has helped us state more precisely what the paper demonstrates and what the framework permits.
W1, Q1.Concern: The demonstrated scope is limited to FOA; it is unclear whether the principle transfers to higher-order ambisonics or microphone-array recordings.Action/Clarification. In our framework, only the input action DR is format-specific. The objectives, the encoder, the rotation token, and the learned latent action ρR make no assumption about the number of channels or the ambisonic order. We chose first-order ambisonics because it is the format of the available real datasets (STARSS23, TAU-NIGENS, Sphere360), not because the method requires it.
Higher-order ambisonics. The rotation we use is the first-order case of a general operator. An ambisonic signal of order N has (N+1)2 channels, and a rotation acts on it through a block-diagonal matrix with one (2ℓ+1)×(2ℓ+1) Wigner-D block per order ℓ=0,…,N. Our DR (Eq. 3) is exactly the N=1 case: the 1×1 order-0 block and the 3×3 order-1 block MRM⊤. Second order appends a 5×5 block, and the third order appends a further 7×7 block. The operator remains an exact, invertible, label-free per-sample multiply, and the objective and training are unchanged. We state this as a property of the operator; we have not yet run higher-order experiments and leave them to future work.
Microphone-array recordings. We agree raw array signals admit no closed-form rotation; in practice they are encoded to the ambisonic domain, after which the same rotation and training apply. Part of our evaluation already takes this route: STARSS23 and TAU-NIGENS are captured with a 32-channel spherical array and converted to FOA, so the real-data SELD results in Tables 2 and 3 are on array recordings through an ambisonic front end. For arrays used without encoding, permuting the microphones of a symmetric array rotates the scene exactly, though only by the coarse angles the geometry allows. We will list arrays without such symmetry as a limitation and a direction for future work.
Why FOA is the right first setting. Our experiments are on FOA, which has become an industry standard for spatial audio: it is the format of 360° video and AR/VR audio, of commercial ambisonic microphones, and of the main public spatial audio datasets, and it is independent of the recording array. A representation learned on FOA therefore already serves a large share of practical spatial audio processing. The approach is not tied to this format, however. As outlined above, it carries over to higher orders and to array recordings, because what it relies on is rotation equivariance itself, which our experiments show to be a strong signal for self-supervised spatial audio representation learning. In the revised version, we will state the FOA scope of the experiments explicitly and incorporate this discussion.
We thank the reviewer again for the supportive assessment. The revised version will also include a direct measurement of equivariance on held-out rotations and ablations isolating rotation conditioning, which make the evidence for the predictable response to rotation noted in S1 quantitative. We are happy to expand on any of these points.