- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces UniAmbi, a unified framework for generating first-order Ambisonics audio from text, video, or joint video-text inputs, supporting both environmental sounds and transcript-controlled speech. It combines a four-channel audio VAE, a parameter-sharing looped diffusion transformer, and loop-aware sliding-window mixture-of-experts routing.
Critical strengths
UniAmbi addresses a meaningful problem by supporting text, video, and joint video-text conditioning for both environmental sounds and transcript-controlled speech in native first-order Ambisonics. The four-channel VAE is well motivated: jointly encoding the sound field, with intensity and axis-wise reconstruction objectives, explicitly accounts for cross-channel spatial relationships. The integration of this representation with iterative generation and generation-level feedback is technically coherent.
Critical weaknesses
Parameter sharing reduces model size, but the main paper report neither inference latency nor FLOPs, peak memory, or training cost. Table 3 also omits parameter counts for the different loop configurations, preventing a direct assessment of the quality-versus-resource trade-off. At the same effective depth, the looped models do not consistently outperform the unshared DiT, and additional loops generally worsen several metrics. Consequently, the results establish parameter compactness more clearly than an advantageous computational trade-off.
Removing MoE changes both model capacity and routing, so the resulting degradation does not establish how much improvement comes specifically from expert specialization. Likewise, the “w/o SW” comparison does not separately establish the benefits of loop conditioning and sliding windows relative to the fixed-chunk routing discussed in Section 3.3. Parameter- and compute-matched controls, together with separate ablations of loop conditioning and window overlap, would make the architectural contribution more convincing.
The reported spatial metrics primarily measure agreement in directional energy or intensity, whereas semantic alignment is evaluated separately. These measurements may not reliably distinguish correct source placement from a scene with plausible overall spatial statistics but incorrect assignments of individual sounds to directions. The authors acknowledge this limitation but it directly affects the central claim of controllable spatial generation. Source-aware localization tests, multi-source identity-direction association tests, and a listening study would provide stronger validation.
Section 4.1 describes a substantial training corpus, but does not clearly report evaluation-set sizes, train/test splitting criteria, or source-recording/video-level overlap controls. It is also unclear whether the compared systems receive comparable task-specific training data or adaptation. Given the extensive synthetic spatialization pipeline and multimodal training mixture, these details are necessary to distinguish architectural improvements from data advantages and to assess generalization to genuinely unseen recordings and scenes.
Questions for authors
How exactly are the per-loop OPSD targets constructed from the combined feedback, including the evaluator implementations, reward weights, and reward-gating threshold ? How sensitive are the reported improvements to these choices?
Under matched training data and inference FLOPs, does loop-aware sliding-window routing still outperform an unshared DiT and a fixed-window MoE? Could the authors also report intermediate-loop quality and reward trajectories to establish whether successive loops actually refine the generated sound field?
Reminder: Discussion Phase Ends November 18 (11:59 PM AoE)
Hi reviewers, a quick reminder that the discussion phase ends on November 18 at 11:59 PM AoE. Please take some time to read the author responses, engage in further discussions if you have any remaining questions or concerns, and update your reviews and scores as appropriate. Thank you!