- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper generates spatial soundscapes in First Order Ambisonics (FOA) by decomposing scenes into individual controllable sound objects (category, interval, gain, trajectory) via a plan, generate, verify, compose pipeline. Using linear FOA encoding, verified objects are summed directly, allowing local editing and single-object resampling without full scene re-rendering.
Critical strengths
-
Clean Linear Design: Because FOA encoding and composition are linear in source signals, per-object generation and summation are mathematically exact. This allows single-object resampling without touching the rest of the scene and grounds local editing in the architecture rather than treating it as an ad-hoc patch.
-
Reusable FOA Corpus: The simulated FOA dataset fills a real gap by offering unified event taxonomies and clip-level split trajectories to prevent data leakage across splits.
Critical weaknesses
-
Circular Benchmark Design: "Event Recall" is measured against reference events generated by the authors' own planner. The metric tests how faithfully the pipeline reproduces its own plan rather than how well it follows the prompt, creating an artificial gap between the proposed method and baselines. An independent human-annotated reference set is required.
-
Confounded Spatial Claims & Ablations:
- Trajectory control is presented as a capability report without comparative spatial accuracy metrics against prior baselines.
- The verification ablation compares a single-sample setup against the full model's best-of-N selection, confounding the benefits of verification with multi-sample selection.
- Missing baseline comparisons and unverified spatial validation outside their custom renderer further weaken the empirical claims.
Questions for authors
- Evaluation: What is the Event Recall score when evaluated against a human-annotated reference set independent of the planner?
- Ablation Control: How does a best-of-N sampling baseline perform without the verification gate?
- Knowledge Graph Utility: What measurable gain does the curated knowledge graph provide compared to relying on implicit LLM knowledge?
- Temporal Metrics: Given equal performance with the top baseline on objective temporal metrics, what evidence supports the claim of superior temporal consistency?
6 comments
Response (2/2)
Q3: Gain of the ASKG over implicit LLM knowledge.
We added a detailed ablation of the ASKG components (Appendix G, Table 8). All variants share the same 100 instructions, planner, prompt, and plan validation, and add knowledge cumulatively: LLM-only (no retrieved knowledge), + Scene–event, + Event–property, and + Event–event (the full ASKG). We report two rule-based metrics, namely the share of planned events that belong to the matched scene (In-scene) and the share of applicable precedes relations satisfied by the plan (Precedes), along with the overall win rate of the full ASKG in pairwise GPT-4o comparisons (both presentation orders, ties counted as 0.5; >50 favors the full ASKG). All metrics include 95% bootstrap CIs.
| Variant | In-scene (%) | precedes (%) | Win rate of full (%) |
|---|---|---|---|
| LLM-only | 78.3 [74.3, 82.0] | 52 [30, 71] | 55.5 [50.0, 60.8] |
| + Scene–event | 96.3 [94.4, 98.0] | 50 [31, 68] | 54.2 [48.8, 59.8] |
| + Event property | 98.9 [97.9, 99.8] | 51 [33, 69] | 53.7 [47.0, 60.5] |
| + Event–event (full, ours) | 98.8 [97.3, 99.8] | 73 [60, 85] | – |
-
Event–event relations (+ Event–property vs. full) provide cross-event consistency. They raise
precedessatisfaction from 51% to 73%, and the LLM judge significantly prefers the full ASKG over + Event–property on same-source consistency (win rate 57.3%, 95% CI [52.0, 62.5]). The LLM planner does not reliably produce these constraints on its own. -
Scene–event relations (LLM-only vs. + Scene–event) ground event selection in the target scene, raising the share of in-scene events from 78.3% to 96.3%.
Q4: Temporal consistency.
We agree that the objective gain over the strongest baseline is small: SoundscapeGen matches SoundscapeAgent in mT-IoU (0.52 vs. 0.51), as stated in Section 4.3. Our claim of better temporal consistency rests on two points. First, SoundscapeGen clearly improves mT-IoU over the other baselines (0.52 vs. 0.16–0.34). Second, listeners rate its temporal consistency higher than all baselines, including SoundscapeAgent (3.90 vs. 3.56). We will make this distinction explicit in the final version.
Thank you for a careful rebuttal and for adding the spatial and ASKG experiments. The spatial comparison, the mono plus renderer control, the ASKG ablation and the concession on temporal consistency are all convincing, and I agree that mono and stereo baselines have no comparable spatial column.
My remaining concern now is the verification ablation. The new semantic only row reaches the same recall as the full model and a mT-IoU within 0.01 of it. The gain over one sample generation therefore comes from drawing more candidates and selecting on CLAP, not from the temporal and spatial checks. The realism score is missing from this table although it carried the largest gap in the original ablation. Without that cell I cannot judge how much verification contributes beyond rejection sampling.
I also ask one clarifying question. Appendix G states that the 100 plans are manually verified but does not describe the procedure. Could the authors elaborate on how the verification was carried out. This would let a reader judge how independent the references are from the planner.
Given the above, I'm inclined to raise my score. Adding the realism results for the semantic-only variant and describing the verification procedure could be helpful to decide my final score.
Response (1/2)
We thank the reviewer for the helpful suggestions. We have revised the manuscript accordingly, with all changes highlighted in blue. Our responses are provided below.
All methods in Table 2, including the baselines, are conditioned on the same plan: end-to-end models receive it as a text prompt and compositional methods as structured input (Section 4.3). Table 2 therefore measures how faithfully each system realizes a given plan, independent of planning. Furthermore, all 100 reference plans are manually verified before being used as evaluation references (Appendix G).
Whether the plan follows the prompt is evaluated separately in Table 3, where the judge sees only the user instruction and the plan. ASKG-grounded plans score 4.83 / 5 on event relevance, supporting the plan as a reasonable proxy for the prompt. Together, Tables 2 and 3 cover the path from prompt to plan to audio.
At the single-object level, Table 1 compares trajectory accuracy with ImmerseDiffusion, SonicMotion, and PhysWave (angular error 1.52° vs. 2.17°–30.25°). At the scene level, the baselines in Table 2 output mono or stereo audio (Table 5), which cannot resolve elevation or front–back direction, so we compare spatial quality through the listening test instead: 3.78 for SoundscapeGen vs. 3.45 for the strongest baseline. We also added two comparisons (Appendix I). Against SpatialSonic (BEWO) on lateral direction, SoundscapeGen achieves 95.4% vs. 76.8% accuracy for static sources and 85.2% vs. 48.1% for moving sources. Against mono generation followed by our analytic renderer, which attains near-perfect direction by construction, SoundscapeGen achieves higher recall (95.36% vs. 88.86%) and mT-IoU (0.52 vs. 0.47).
Regarding validation outside our renderer, training and evaluating on synthetically spatialized FOA follows established practice in spatial audio generation; for example, ImmerseDiffusion (Heydari et al., ICASSP 2025) also measures direction with intensity-vector DOA estimation. This estimator is derived directly from the FOA encoding and is therefore independent of the generator. The listening test provides an additional check that relies on neither the renderer nor the estimator.
Iterative resampling is an integral part of our verification module: because diffusion sampling is stochastic, the module draws new candidates until one passes all checks, so the w/o verification row uses a single candidate by construction (Appendix F). As the best-of-N baseline the reviewer asks for, we added a Semantic-only verification variant to Table 4. It uses the same per-round sampling (Nc=5) and resampling cap (R=5), but selects candidates by the CLAP check alone, without the temporal and spatial checks, which results in a nearly identical generation budget.
The two verification variants are close on scene-level averages because the temporal and spatial checks change the selected candidate for only 23 of 438 objects (5.3%). However, these 23 cases correspond to failures that CLAP cannot detect, such as a clip that matches its event label globally but is silent during its planned interval. On these 23 objects, the mean per-object Spatial Accuracy (objects silent in their planned interval counted as 0) rises from 51.9% with semantic-only selection to 86.9% with full verification.