- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes AURA, a unified representation for spatial audio that aims to support arbitrary input and output spatial-audio formats. The key idea is to encode multi-channel audio as an unordered set of channel-wise latent representations conditioned on the physical direction of each channel, producing a format-agnostic latent representation . A geometry-conditioned decoder can then reconstruct the representation in an arbitrary target format. The model additionally incorporates a physics-inspired regularization toward a deterministic HOA representation and a flow-matching diffusion transformer for stereo-to-soundfield generation. The authors evaluate AURA on STARSS22/23, Freesound 5.1, and 3D-MARCo. The experiments include self-reconstruction, cross-format conversion, cross-array conversion, and stereo-to-spatial generation.
Critical strengths
-
Clean unified representation formulation
AURA provides a compelling formulation of spatial audio as a geometry-conditioned set representation. Treating channels as an unordered set while explicitly conditioning on their physical directions is a natural and potentially powerful way to decouple the representation from a particular channel ordering or format. This is a meaningful system-level perspective for spatial-audio representation learning. -
Practical any-to-any conversion framework
The same latent representation can be decoded into multiple target spatial formats, providing a unified interface for cross-format conversion. The experiments on 3D-MARCo demonstrate that the approach can handle several nontrivial conversion scenarios, including cross-array conversion. This makes the work practically relevant beyond a single spatial-audio task.
Critical weaknesses
The central “any-format” generalization claim is insufficiently validated. Although the authors claim that AURA can generalize to arbitrary spatial configurations, the current experimental setup does not convincingly demonstrate this capability. In particular, the evaluation does not clearly establish whether the proposed geometry-conditioned representation can generalize to spatial configurations that are unseen during training. Several concerns regarding this claim are raised in the following section.
Questions for authors
Q1. Unseen geometries and datasets.
In Table 1, the authors list the datasets and spatial formats used for training and evaluation. However, the current experiments do not explicitly evaluate generalization to unseen datasets or spatial geometries. Could the authors provide experiments in which certain spatial configurations or geometries are held out from training and used only for evaluation? For example, the authors could remove one or more geometries from the training set and evaluate AURA on these unseen geometries. Such experiments would help distinguish the architectural capability to support arbitrary geometries from empirically demonstrated generalization to unseen spatial configurations.
Q2. Role of geometry conditioning.
In Table 5, replacing geometry conditioning with index embedding results in comparable performance, with some metrics even slightly improving. This raises a question about whether the geometry conditioning is actually necessary for the claimed generalization to arbitrary spatial configurations. Could the authors provide further analysis of the role of geometry conditioning, particularly by evaluating both variants on unseen geometries? Such an analysis would help clarify whether geometry conditioning provides a meaningful advantage for generalization beyond the spatial configurations observed during training.
Q3. Effect of the spherical-harmonic order .
In Table 5, changing from 3 to 2 results in very similar performance, with small enhancements in Reconstruction Spec. SNR and Cross Latent-SNR. Could the authors provide further analysis of the role of in the proposed representation? In particular, it would be helpful to evaluate additional values, such as and , to better understand how the spherical-harmonic order affects the model's reconstruction and cross-format generalization performance.
3 comments
Thank you for the detailed rebuttal and the additional experiments. I appreciate the authors' efforts to address my concerns and the clarifications regarding generalization to unseen geometries and the role of geometry conditioning. Nevertheless, my overall assessment remains unchanged, and I will maintain my original score.
Thank you very much for your reply. We will further optimize the algorithm in the future, aiming to enhance its generalization performance in layouts unseen before.
To Reviewer XXZm
Q1: held-out geometries. Done; the answer is mostly negative, and the paper now says so. We retrained with every Hamasaki 12-channel example removed and decoded to Hamasaki at test time.
With the layout unseen, AURA degrades gracefully:
With six training layouts it does not extrapolate. We agree with your distinction between architectural capability and demonstrated generalization. The paper now claims only the former, and §3.3, §4.5 and the conclusion are revised accordingly.
Q2: SH geometry conditioning vs. index embedding on an unseen geometry. We ran exactly this comparison:
The index embedding reconstructs the seen layouts slightly better; it has fewer constraints while memorizing six layouts. However, it transfers worse to the unseen one: |ΔE| is 1.3 dB worse, which is well beyond the ±0.18 dB seed spread. Geometry conditioning therefore helps where it should, on an unseen layout, but modestly. Its main benefit remains structural: permutation invariance, one encoder across channel counts, and a continuous query space.
Q3: SH order N_sh (including 1 and 4). Added (13 scenes, 40k steps):
All values lie within seed noise. The representation is insensitive to N_sh on seen layouts, so even N_sh = 1 (4 features per channel instead of 16) suffices there. We keep N_sh = 3 as the default because it costs almost nothing at these channel counts. We now report this insensitivity rather than claim an optimum.