acceptodds
Under review as a conference paper at ICLR 2027

OrbitRIR: Frame-Averaged Few-Shot RIR Synthesis for 3D Sound Source Localization

Abstract

Embodied spatial reasoning requires agents to infer 3D world states from partial sensory observations. Among these observations, room acoustics provides a particularly informative spatial signal, as Room Impulse Responses (RIRs) encode interactions between source–receiver geometry and the surrounding environment. Recent few-shot acoustic models predict RIRs from spatial geometry and sparse room context. However, existing few-shot acoustic generators lack architectural inductive biases that reflect the known receiver-yaw invariance of omnidirectional monaural RIRs. To this end, we introduce OrbitRIR, a geometry-constrained generative acoustic model incorporating this inductive bias through Frame Averaging over a receiver-yaw orbit. We evaluate OrbitRIR on few-shot RIR synthesis tasks and downstream sound source localization tasks through analysis-by-synthesis. On AcousticRooms, OrbitRIR outperforms its geometry-unconstrained counterpart. When inverted for localization, it also improves localization accuracy over learned and physics-based baselines.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
few-shot RIR synthesis, frame averaging, receiver-yaw invariance, generative acoustic modeling, flow matching, analysis-by-synthesis, 3D sound-source localization, spatial acoustics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_qjdYOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes OrbitRIR, a framework prtraining a generative model for RIR prediction and inversely using the generative model for 3D sound source localization task. In the generative model, they explicitly consider yaw-rotation invariance during generative model training and use panoramic depth map as the 3D room context. In the experiment, they adopt AcousticRooms dataset and compared with several other baselines like FewShot-RIR, FEM-based localization methods and random selection. Experimental results demonstrated their claimed feasibility and advantage of the OrbitRIR method.

Critical strengths
  1. The core idea is clearly-presented throughout this paper, enabling me quickly follow OrbitRIR idea while reviewing this paper.

  2. The experiment section is well-repsented, given the experimental setting. Alongside the code availability, the authors present the experiment, analysis, ablated study and qualitative visualization in detail.

Critical weaknesses
  1. Applicability in Real Scenario: The paper wants to solve 3D sound source localization which is very practical in mobile robots. My main concern comes from the setting in the paper that they rely on the accessibility of direct RIR data. In real scenario, direct RIR data rarely exist, but instead the RIR-effected audio is widely accessible. I thus think the applicability of this paper (in either RIR generation and the reverse 3D sound source localization) is largely limited. A more applicable way is to use real audio (convolved with RIR) to implicitly model RIR for the task (for example, Deep Neural Room Acoustics Primitive, ICML24), so that your method can be applied in real-world setting.

  2. Localization Setting: In the localization module, RIROrbits requires a finite set of potential 3D positions, then compare the generated RIR at each potential position with the recorded RIR. In other words, the accuracy of localization depends on the finite potential 3D potions, requiring well-prepared 3D positions candidates.

Questions for authors
  1. Can OrbitRIR handle multi-room setting? In complex room setting, for example, the sound source and receiver are placed in different rooms (for example, soundspaces-v2 supported matterport3D data), the panoramic depth map thus can't capture the whole scene, does the method still work or how much performance degradation will be observed?

  2. Generalization regarding Few-Shot RIR synthesis. Based on my knowledge, RIR is extremely sensitive to either sound source or receiver position. A small position to them can lead to huge RIR difference. Can the author run experiments to explicitly test OrbitRIR's generalization and robustness regarding source-receiver placement?

1 comment
AuthorsOpenReview ↗

Received-Audio Localization, Grid Sensitivity, and Generalization

We address practical applicability, grid sensitivity, and generalization through received-audio, measured-HAA, and multi-room MP3D evaluations.

Table Q1. Known-excitation received-audio localization on MP3D (0.5 m grid). We evaluate 128 held-out source–receiver pairs across four buildings, including 60 same-room and 68 cross-room pairs. All four models are adapted to MP3D with matched training budgets. Results are averaged over three inference seeds.

MethodMedian (m) ↓SR@1m ↑Same-room SR@1m ↑Cross-room SR@1m ↑
OrbitRIR2.00129.43%40.56%19.61%
FLAC2.06727.60%28.89%26.47%
Yaw-Aug FLAC2.01231.51%39.44%24.51%
Few-ShotRIR2.76718.23%27.22%10.29%

Table Q2. Sensitivity of analysis-by-synthesis localization to candidate-grid resolution. Results are reported for OrbitRIR on HAA and MP3D. The oracle median measures the distance from each ground-truth source position to its nearest grid candidate. Latency is reported in seconds per query. The MP3D grid-resolution diagnostic is separate from the received-audio comparison in Table Q1.

DatasetGridOracle median (m) ↓Localization median (m) ↓SR@1m ↑Latency (s/query) ↓
HAA1.0 m0.6442.28423.96%1.093
HAA0.5 m0.2522.03130.99%8.253
HAA0.25 m0.1211.82830.99%72.796
MP3D1.0 m0.4682.62125.00%0.942
MP3D0.5 m0.2272.69620.31%2.909
MP3D0.25 m0.1022.90225.00%10.804

W1. Applicability to received audio and measured environments.

Direct RIRs are generally unavailable in passive localization (cf. DeepNeRAP [1]); we study active localization with known excitation. We convolve a sine sweep with MP3D RIRs [6,7], deconvolve the received audio, and compare it with predicted RIRs using frozen AGREE. OrbitRIR reaches 29.43% SR@1m versus 29.17% with direct RIR input using the same weights (Table Q1), without retraining. On measured HAA RIRs from four rooms [2] at a 0.5 m grid, OrbitRIR reduces mean error from FLAC's 2.814 to 2.605 m (Appendix G.3). HAA uses RIR observations, not recorded source waveforms.

W2. Candidate-grid dependence.

Candidates are generated from mapped geometry, never inserted from ground truth. On AcousticRooms, a 1.0-to-0.5 m grid improves OrbitRIR SR@1m from 38.3% to 53.1%; at 0.25 m it stays 53.1% despite 7.4× more candidates (724 to 5,354; Table U2). On HAA (Table Q2), refining 0.5 to 0.25 m improves oracle median (0.252 to 0.121 m) and localization median (2.031 to 1.828 m), but SR@1m stays 30.99% while latency rises from 8.253 to 72.796 s/query. MP3D varies non-monotonically. Thus, finer grids reduce discretization error but do not guarantee better localization.

Q1. Localization in complex multi-room environments.

On 128 held-out pairs across four MP3D buildings (60 same-room, 68 cross-room; Table Q1), OrbitRIR achieves the lowest overall median error (2.001 m) and highest same-room SR@1m (40.56%). Yaw-Aug leads overall SR@1m (31.51%), and FLAC leads cross-room SR@1m (26.47% vs. OrbitRIR's 19.61%). OrbitRIR mean error rises from 1.812 m (same-room) to 3.980 m (cross-room), reflecting the difficulty of intervening geometry not visible in the receiver panorama. Cross-room performance remains limited.

Q2. Generalization to unseen source–receiver placements.

Thank you for highlighting the sensitivity of RIRs to changes in source and receiver positions. We extended our evaluation to held-out source–receiver configurations, physically measured acoustics, and cross-dataset transfer to investigate generalization beyond the original AcousticRooms setting.

On 128 held-out MP3D source–receiver pairs (Table Q1), OrbitRIR achieves 2.001 m median localization error, compared with 2.067 m for FLAC and 2.012 m for Yaw-Aug FLAC. On HAA, which contains physically measured RIRs, OrbitRIR achieves 2.645 m mean localization error at 0.25 m grid spacing, compared with 2.844 m for FLAC and 2.924 m for Yaw-Aug FLAC (Appendix G.3). We also evaluate zero-shot transfer from AcousticRooms to MP3D without MP3D-specific adaptation. In this setting, OrbitRIR achieves 3.533 m mean localization error, compared with 3.681 m for FLAC (Appendix G.4).

These results provide evidence of generalization to held-out configurations and different acoustic environments. However, they do not directly quantify the sensitivity of RIR synthesis to small, controlled source or receiver displacements. We therefore distinguish generalization across unseen configurations from local spatial robustness.

References: [1] He et al., ICML 2024. [2] Wang et al., CVPR 2024. [3] Rosinol et al., IJRR 2021. [4] He et al., WACV 2025. [5] Dokmanić and Vetterli, ICASSP 2012. [6] Chen et al., NeurIPS 2022. [7] Chang et al., 3DV 2017.

Reviewer_3wyjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper comprises two parts. One is to improve the performance of FLAC (a multimodal few-shot RIR predictor) by leveraging the raw invariance property of the omnidirectional receiver. To achieve this, the presented method rolls and rotates the panoramic depth and relative source positions together at four angles (0, 90, 180, 270), pass them to ViT, and average the resulting features. The other is to solve the source localization task via analysis-by-synthesis of the trained RIR predictor.

Critical strengths

None.

Critical weaknesses
  1. Regarding the first part of the submission (few-shot RIR estimation), the improvements over "Yaw-Augmented FLAC" seem very small.
  2. Regarding the second part of the submission (localization), the localization problem studied here appears quite artificial and unrealistic. Personally, it is hard to imagine a scenario where an RIR's source position is unknown while a receiver-centered panoramic depth, along with reference RIRs and their relative positions, is available.
  3. And importantly, the two contributions, the frame averaging and the localization method, are quite orthogonal and do not add much insight when combined.
Questions for authors
  1. Does the proposed model have better yaw invariance than the yaw-augmented FLAC, especially at angles other than 0, 90, 180, and 270? Did the author evaluate the yaw-augmented FLAC on the localization task?
  2. It seems like the proposed model is more complex and requires more compute. Is this true?
1 comment
AuthorsOpenReview ↗

C4 Invariance, Data Efficiency, and Localization Trade-offs

Thank you for the feedback. We address yaw augmentation, practical localization, and the forward–inverse connection through new experiments and expanded evaluation.

W1. The gain over Yaw-Aug FLAC is small.

At full training data, results are mixed. At Kctx=8K_{\mathrm{ctx}}=8, OrbitRIR improves all six metrics at 25%, 50%, and 75% training data (Table W1); at 50%, EDT drops from 44.100 to 43.000 ms and R@1 rises from 4.980% to 5.690%. At Kctx=1K_{\mathrm{ctx}}=1, all six metrics improve at 50% and 75% (revised Table 3). RIR-to-geometry retrieval is reported in Appendix F.2.

Table W1. Synthesis and retrieval at Kctx=8K_{\mathrm{ctx}}=8. Best values are bold within each data fraction.

DataMethodEDT ↓C50 ↓T60 ↓R@1 ↑R@5 ↑R@10 ↑
25%Yaw-Aug42.985 ± 0.0841.071 ± 0.0029.863 ± 0.0114.548 ± 0.16314.751 ± 0.20622.272 ± 0.159
25%OrbitRIR42.968 ± 0.0821.041 ± 0.0019.742 ± 0.0164.901 ± 0.02415.042 ± 0.26022.547 ± 0.155
50%Yaw-Aug44.100 ± 0.0361.093 ± 0.0019.869 ± 0.0094.980 ± 0.10315.979 ± 0.11823.746 ± 0.099
50%OrbitRIR43.000 ± 0.0420.989 ± 0.0029.242 ± 0.0095.690 ± 0.10216.585 ± 0.12724.516 ± 0.100
75%Yaw-Aug42.468 ± 0.0191.110 ± 0.0029.291 ± 0.0115.223 ± 0.10016.159 ± 0.08423.724 ± 0.104
75%OrbitRIR40.787 ± 0.0531.049 ± 0.0018.895 ± 0.0135.482 ± 0.07616.566 ± 0.19024.340 ± 0.219
100%Yaw-Aug39.923 ± 0.0581.013 ± 0.0027.965 ± 0.0155.356 ± 0.03616.528 ± 0.22123.961 ± 0.133
100%OrbitRIR38.794 ± 0.0740.978 ± 0.0028.202 ± 0.0175.394 ± 0.04716.459 ± 0.04624.194 ± 0.151

Frame averaging guarantees C4C_4 invariance, whereas augmentation encourages it. The gains suggest a useful low-data inductive bias. Models use one training run and five evaluation seeds (42–46); 25–75% runs used 2×L40 GPUs and 100% runs used 2×A6000 GPUs.

W2. Practicality of acoustic localization. Our setting is map-aware active localization, not unrestricted passive localization. Prior mapping and calibration provide geometry and sparse reference RIRs; the query source position remains unknown. A cooperating source emits a known probe so the receiver can estimate its RIR by deconvolution. We test this with simulated received audio in MP3D (Appendix G.2). The frozen generator needs no room-specific localization training, but does require prior calibration and known excitation.

W3. Connecting the two contributions. Localization tests the same frozen generator with C4C_4-invariant geometry conditioning, not a separate learned method. For candidate p\mathbf p, sample h^p∼pθ∗(⋅∣qp,O)\hat h_{\mathbf p}\sim p_{\theta^*}(\cdot\mid q_{\mathbf p},\mathcal O) and select

p^=arg⁡max⁡p∈Pcos⁡(Ea(hobs),Ea(h^p)),\hat{\mathbf p}=\arg\max_{\mathbf p\in\mathcal P}\cos(E_a(h_{\mathrm{obs}}),E_a(\hat h_{\mathbf p})),

where EaE_a is frozen AGREE. No localization-specific training is needed. Under a 90° frame change, SR@1m shifts only from 51.6% to 51.0% (64 queries, three seeds).

Q1. Yaw invariance at other angles, and Yaw-Aug on localization.

FA guarantees C4C_4 invariance, but not invariance at intermediate yaw angles (Appendix D). At the C4C_4 headings, OrbitRIR produces nearly identical predictions and achieves lower C50 and EDT errors than Yaw-Aug, while Yaw-Aug retains a lower T60 error. At the intermediate angles (45°, 135°, 225°, and 315°), Yaw-Aug generally achieves better acoustic prediction and retrieval performance. These results distinguish the exact finite-group invariance enforced by frame averaging from the broader empirical rotation robustness encouraged by augmentation.

We also evaluated Yaw-Aug FLAC on localization. In the original 64-query evaluation (Table 2), Yaw-Aug achieves a lower median error (0.892 vs. 0.928 m) and higher SR@1m (54.7% vs. 53.1%), while OrbitRIR achieves higher [email protected] (28.1% vs. 21.9%). In the expanded 128-query evaluation at 0.5 m grid spacing (Appendix G.1), OrbitRIR achieves a lower median error (0.893 vs. 0.916 m), while both methods attain 53.1% SR@1m. These comparisons indicate comparable localization performance, with different advantages across metrics and evaluation settings.

Q2. Model complexity and computational cost.

Frame averaging uses four yaw frames instead of one. On dedicated A6000s, OrbitRIR costs 69.4 vs. 21.4 h to train (3.24×), 221.6 vs. 126.7 ms per RIR (1.75×), and 15.1 vs. 8.9 ms per localization candidate (1.70×). Yaw-Aug retains FLAC's single-frame inference architecture. On 128 AcousticRooms queries at 0.5 m spacing, OrbitRIR has lower median error than FLAC/Yaw-Aug (0.893 vs. 0.933/0.916 m), but all achieve 53.1% SR@1m. Thus, C4C_4 consistency and data-limited synthesis gains incur extra computation without universal localization gains.

Reviewer_UtttOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents OrbitRIR, a geometry-constrained generative model for few-shot Room Impulse Response (RIR) synthesis. The key idea is to incorporate receiver-yaw invariance through Frame Averaging, improving the physical consistency of generated RIRs. The authors further demonstrate that the pretrained acoustic model can be used for 3D sound source localization through analysis-by-synthesis, without localization-specific training. Experiments on AcousticRooms show improvements over existing baselines in both RIR synthesis and localization.

Critical strengths
  1. Interesting and well-motivated approach. Incorporating physical symmetry through Frame Averaging is a principled way to improve acoustic modeling and generalization.

  2. Promising experimental results. The method consistently improves RIR synthesis quality over FLAC, and the localization experiments demonstrate the usefulness of the learned acoustic representation for downstream spatial reasoning.

3.Good experimental analysis. The comparisons with yaw augmentation, the invariance evaluation, and the qualitative visualizations provide useful evidence supporting the proposed approach.

Critical weaknesses
  1. Computational overhead. Frame Averaging introduces additional computational cost, while the localization procedure requires synthesizing RIRs for individual candidate positions. A more detailed discussion of the accuracy-efficiency trade-off would strengthen the practical motivation.

  2. Incomplete discussion of related work. The discussion of recent acoustic localization methods is not sufficiently comprehensive. In particular, recent work such as AV-Twin (https://arxiv.org/abs/2512.10778), which also explores acoustic modeling and spatial localization, is missing. I encourage the authors to cite and discuss this work in Section 2, particularly clarifying the differences in problem formulation, assumptions, and localization methodology.

  3. Limited localization evaluation. The localization experiments are conducted on a relatively small evaluation set (64 queries) and rely on a predefined candidate grid. It would be helpful to evaluate the robustness of the method across more environments and different candidate-grid resolutions.

Questions for authors

See weakness

1 comment
AuthorsOpenReview ↗

Computational Trade-offs, C4 Rotation Consistency, and Expanded Localization Evaluation

Thank you for the constructive feedback and for highlighting the importance of evaluating the computational cost, related work, and robustness of the localization procedure. We have expanded the evaluation and revised the manuscript to address these concerns.

Table U1. RIR synthesis accuracy and receiver-yaw consistency on unseen AcousticRooms. Results are averaged over five evaluation seeds. The rotation-consistency measurements compare predictions across the four C4C_4 headings.

EvidenceMetric / settingFLACYaw-Aug FLACOrbitRIR
Prediction accuracyT60 error (%), K=8, 0° ↓8.997.968.20
C50 error (dB), K=8, 0° ↓1.0091.0130.978
EDT error (ms), K=8, 0° ↓40.6539.9238.79
R@1 (%), K=8, 0° ↑5.175.405.40
R@5 (%), K=8, 0° ↑15.4316.5316.45
R@10 (%), K=8, 0° ↑23.4123.9624.20
Rotation robustnessT60 spread across C4 headings (pp) ↓0.8970.0710.001
C50 range across C4 (dB) ↓1.009–1.1431.013–1.0270.978–0.978
EDT range across C4 (ms) ↓40.65–47.3939.92–40.2838.79–38.80
EDT error at 180° (ms) ↓47.3940.1838.80

W1. Computational overhead and the accuracy–efficiency trade-off. Frame averaging incurs overhead: on two dedicated GPUs, FLAC and OrbitRIR train for 21.4 and 69.4 h; OrbitRIR costs 1.75× per synthesized RIR and 1.70× per localization candidate. Higher training times in the submission reflected GPU sharing (revised Appendix B.1).

In return, OrbitRIR reduces C4 T60-error spread from 0.071 to 0.001 percentage points versus Yaw-Aug (Table U1). At Kctx=8K_{\mathrm{ctx}}=8, it improves C50 and EDT but not T60. Localization is comparable across FLAC-based models (Table U2), highlighting the accuracy–cost trade-off.

Against PanoFormer-style roll-and-flip augmentation (Appendix F.1, Table 11), OrbitRIR lowers EDT, C50, and T60 errors at both context sizes, with similar retrieval but higher inference cost.

W2. Related work: AV-Twin. Thank you for pointing out AV-Twin (Lan et al., MobiSys 2026). We have added this work to Section 2 and clarified the distinction between the two approaches. AV-Twin reconstructs room-specific acoustic fields and trains a room-specific CNN for localization. In contrast, OrbitRIR uses sparse acoustic context to condition a generative forward model in previously unseen environments, then applies analysis-by-synthesis localization without room-specific localization training or fine-tuning. The approaches therefore differ in their assumptions about room-specific adaptation and their use of the learned acoustic model for localization.

W3. Evaluation size, environments and grid resolution. Thank you for suggesting a broader localization evaluation. We expanded the AcousticRooms evaluation from 64 to 128 queries across the 16 unseen rooms with available meshes and evaluated candidate-grid spacings of 1.0, 0.5, and 0.25 m.

Table U2. AcousticRooms mesh-grid localization across candidate-grid resolutions. We evaluate 128 queries from 16 unseen rooms with available meshes. SR denotes localization success rate. Best results within each grid spacing are shown in bold.

GridCandidates/queryMethodMedian error (m) ↓SR@1m (%) ↑[email protected] (%) ↑
1.0 m103OrbitRIR1.21938.311.7
FLAC1.14343.010.2
Yaw-Aug FLAC1.20243.810.9
Few-ShotRIR1.57521.92.3
0.5 m724OrbitRIR0.89353.118.0
FLAC0.93353.118.0
Yaw-Aug FLAC0.91653.116.4
Few-ShotRIR1.55822.74.7
0.25 m5,354OrbitRIR0.92053.116.4
FLAC0.97351.617.2
Yaw-Aug FLAC1.03047.721.1
Few-ShotRIR1.58123.43.1

The results show that finer candidate grids do not uniformly improve localization accuracy. For OrbitRIR, reducing grid spacing from 1.0 to 0.5 m improves SR@1m from 38.3% to 53.1% and reduces median error from 1.219 to 0.893 m. Further refinement to 0.25 m increases the average candidate count from 724 to 5,354, while SR@1m remains at 53.1% and median error increases slightly to 0.920 m. At 0.5 m spacing, OrbitRIR achieves the lowest median error, while all three FLAC-based models attain the same SR@1m.

These findings indicate that localization accuracy depends on both grid resolution and the fidelity of the acoustic predictions. Increasing candidate density reduces discretization error but does not necessarily improve the selection of the correct source region. We have also added complementary evaluations on multi-room MP3D environments and physically measured HAA acoustics to assess localization beyond the original AcousticRooms setting.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Clarifying OrbitRIR's Contributions and Expanded Experimental Evidence

We thank the reviewers for their careful assessment of OrbitRIR. The discussion has helped us clarify the paper's main contribution: explicitly enforcing the appropriate rotation symmetry in an acoustic forward model improves the consistency of its predictions and can benefit downstream inverse inference, particularly when training data are limited. Source localization is therefore not a separate algorithmic contribution, but an analysis-by-synthesis test of whether the learned forward acoustic model can provide useful spatial information or not.

We provide new evaluations to address the reviewers' main concerns.

  1. Comparison with yaw augmentation. We added standard PanoFormer-style augmentation (full-circle panorama roll + mirror flip) at full training data, and a training-data ablation (25 / 50 / 75 / 100 %) against roll-only Yaw-Aug. At full data, OrbitRIR has lower EDT, C50 and T60 errors than Roll + Flip with comparable retrieval; with 25–75 % of the data it is better on all six metrics at Kctx=8K_{\mathrm{ctx}}=8.
  2. Downstream localization and evaluation scale. We doubled the AcousticRooms localization set to 128 queries (all 16 unseen rooms with a mesh), swept the candidate grid (1.0 / 0.5 / 0.25 m), and added 128 held-out source–receiver pairs in multi-room MP3D scenes, a received-audio protocol, and measured HAA rooms.
  3. Computational trade-offs. We report training and per-candidate inference costs alongside accuracy, rather than presenting architectural symmetry as free of computational overhead.

Taken together, the added evidence supports a precise contribution: symmetry-by-design provides rotation-consistent acoustic predictions and useful gains in data-limited modeling, with downstream localization that is competitive with the baselines.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.