acceptodds
Under review as a conference paper at ICLR 2027

SoundscapeGen: Smart Spatial Soundscape Construction from High-level Descriptions

Abstract

Generating spatial soundscapes from high-level natural-language descriptions requires not only realistic audio synthesis but also reliable control over what sound events occur, when they occur, and how they move in three-dimensional space. Existing end-to-end and compositional approaches often rely on the implicit knowledge of large language models (LLMs) for scene planning, and provide limited guarantees that the generated audio follows the intended semantic, temporal, and spatial structure. We introduce SoundscapeGen, a framework that represents a soundscape as a set of associated controllable sound objects and constructs it through a Plan → Generate → Verify → Compose pipeline. An acoustic scene knowledge graph (ASKG) provides explicit priors for event selection, temporal relations, source properties, and motion, while a fine-tuned physics-guided First-Order Ambisonics generator synthesizes each sound object with explicit trajectory control. Object-level semantic, temporal, and spatial verification further filters inconsistent generations before composition. Experiments show that SoundscapeGen improves event coverage and temporal consistency over both end-to-end text-to-audio and compositional baselines, while providing explicit control over individual source trajectories. Its object-based representation further supports localized interactive editing without regenerating the entire soundscape. Demos are available at https://soundscapegeneration.github.io/SoundscapeGen/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Text-to-Audio Generation, Soundscape Generation, First-Order Ambisonics

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_WqeEOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper generates spatial soundscapes in First Order Ambisonics (FOA) by decomposing scenes into individual controllable sound objects (category, interval, gain, trajectory) via a plan, generate, verify, compose pipeline. Using linear FOA encoding, verified objects are summed directly, allowing local editing and single-object resampling without full scene re-rendering.

Critical strengths
  1. Clean Linear Design: Because FOA encoding and composition are linear in source signals, per-object generation and summation are mathematically exact. This allows single-object resampling without touching the rest of the scene and grounds local editing in the architecture rather than treating it as an ad-hoc patch.

  2. Reusable FOA Corpus: The simulated FOA dataset fills a real gap by offering unified event taxonomies and clip-level split trajectories to prevent data leakage across splits.

Critical weaknesses
  1. Circular Benchmark Design: "Event Recall" is measured against reference events generated by the authors' own planner. The metric tests how faithfully the pipeline reproduces its own plan rather than how well it follows the prompt, creating an artificial gap between the proposed method and baselines. An independent human-annotated reference set is required.

  2. Confounded Spatial Claims & Ablations:

  • Trajectory control is presented as a capability report without comparative spatial accuracy metrics against prior baselines.
  • The verification ablation compares a single-sample setup against the full model's best-of-N selection, confounding the benefits of verification with multi-sample selection.
  • Missing baseline comparisons and unverified spatial validation outside their custom renderer further weaken the empirical claims.
Questions for authors
  1. Evaluation: What is the Event Recall score when evaluated against a human-annotated reference set independent of the planner?
  2. Ablation Control: How does a best-of-N sampling baseline perform without the verification gate?
  3. Knowledge Graph Utility: What measurable gain does the curated knowledge graph provide compared to relying on implicit LLM knowledge?
  4. Temporal Metrics: Given equal performance with the top baseline on objective temporal metrics, what evidence supports the claim of superior temporal consistency?
6 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for the helpful suggestions. We have revised the manuscript accordingly, with all changes highlighted in blue. Our responses are provided below.

W1 & Q1: Circularity of Event Recall.

All methods in Table 2, including the baselines, are conditioned on the same plan: end-to-end models receive it as a text prompt and compositional methods as structured input (Section 4.3). Table 2 therefore measures how faithfully each system realizes a given plan, independent of planning. Furthermore, all 100 reference plans are manually verified before being used as evaluation references (Appendix G).

Whether the plan follows the prompt is evaluated separately in Table 3, where the judge sees only the user instruction and the plan. ASKG-grounded plans score 4.83 / 5 on event relevance, supporting the plan as a reasonable proxy for the prompt. Together, Tables 2 and 3 cover the path from prompt to plan to audio.

W2(a) & (c): Spatial accuracy against baselines and spatial validation.

At the single-object level, Table 1 compares trajectory accuracy with ImmerseDiffusion, SonicMotion, and PhysWave (angular error 1.52° vs. 2.17°–30.25°). At the scene level, the baselines in Table 2 output mono or stereo audio (Table 5), which cannot resolve elevation or front–back direction, so we compare spatial quality through the listening test instead: 3.78 for SoundscapeGen vs. 3.45 for the strongest baseline. We also added two comparisons (Appendix I). Against SpatialSonic (BEWO) on lateral direction, SoundscapeGen achieves 95.4% vs. 76.8% accuracy for static sources and 85.2% vs. 48.1% for moving sources. Against mono generation followed by our analytic renderer, which attains near-perfect direction by construction, SoundscapeGen achieves higher recall (95.36% vs. 88.86%) and mT-IoU (0.52 vs. 0.47).

Regarding validation outside our renderer, training and evaluating on synthetically spatialized FOA follows established practice in spatial audio generation; for example, ImmerseDiffusion (Heydari et al., ICASSP 2025) also measures direction with intensity-vector DOA estimation. This estimator is derived directly from the FOA encoding and is therefore independent of the generator. The listening test provides an additional check that relies on neither the renderer nor the estimator.

W2(b) & Q2: Verification vs. best-of-N sampling.

Iterative resampling is an integral part of our verification module: because diffusion sampling is stochastic, the module draws new candidates until one passes all checks, so the w/o verification row uses a single candidate by construction (Appendix F). As the best-of-N baseline the reviewer asks for, we added a Semantic-only verification variant to Table 4. It uses the same per-round sampling (NcN_c=5) and resampling cap (R=5), but selects candidates by the CLAP check alone, without the temporal and spatial checks, which results in a nearly identical generation budget.

VariantCand. / objectRecallmT-IoUSpat. Acc.
w/o verification191.650.5183.55
Semantic-only verification8.8595.360.5184.10
Full verification (ours)9.3895.360.5284.31

The two verification variants are close on scene-level averages because the temporal and spatial checks change the selected candidate for only 23 of 438 objects (5.3%). However, these 23 cases correspond to failures that CLAP cannot detect, such as a clip that matches its event label globally but is silent during its planned interval. On these 23 objects, the mean per-object Spatial Accuracy (objects silent in their planned interval counted as 0) rises from 51.9% with semantic-only selection to 86.9% with full verification.

AuthorsOpenReview ↗

Response (2/2)

Q3: Gain of the ASKG over implicit LLM knowledge.

We added a detailed ablation of the ASKG components (Appendix G, Table 8). All variants share the same 100 instructions, planner, prompt, and plan validation, and add knowledge cumulatively: LLM-only (no retrieved knowledge), + Scene–event, + Event–property, and + Event–event (the full ASKG). We report two rule-based metrics, namely the share of planned events that belong to the matched scene (In-scene) and the share of applicable precedes relations satisfied by the plan (Precedes), along with the overall win rate of the full ASKG in pairwise GPT-4o comparisons (both presentation orders, ties counted as 0.5; >50 favors the full ASKG). All metrics include 95% bootstrap CIs.

VariantIn-scene (%)precedes (%)Win rate of full (%)
LLM-only78.3 [74.3, 82.0]52 [30, 71]55.5 [50.0, 60.8]
+ Scene–event96.3 [94.4, 98.0]50 [31, 68]54.2 [48.8, 59.8]
+ Event property98.9 [97.9, 99.8]51 [33, 69]53.7 [47.0, 60.5]
+ Event–event (full, ours)98.8 [97.3, 99.8]73 [60, 85]–
  • Event–event relations (+ Event–property vs. full) provide cross-event consistency. They raise precedes satisfaction from 51% to 73%, and the LLM judge significantly prefers the full ASKG over + Event–property on same-source consistency (win rate 57.3%, 95% CI [52.0, 62.5]). The LLM planner does not reliably produce these constraints on its own.

  • Scene–event relations (LLM-only vs. + Scene–event) ground event selection in the target scene, raising the share of in-scene events from 78.3% to 96.3%.

Q4: Temporal consistency.

We agree that the objective gain over the strongest baseline is small: SoundscapeGen matches SoundscapeAgent in mT-IoU (0.52 vs. 0.51), as stated in Section 4.3. Our claim of better temporal consistency rests on two points. First, SoundscapeGen clearly improves mT-IoU over the other baselines (0.52 vs. 0.16–0.34). Second, listeners rate its temporal consistency higher than all baselines, including SoundscapeAgent (3.90 vs. 3.56). We will make this distinction explicit in the final version.

Thank you for a careful rebuttal and for adding the spatial and ASKG experiments. The spatial comparison, the mono plus renderer control, the ASKG ablation and the concession on temporal consistency are all convincing, and I agree that mono and stereo baselines have no comparable spatial column.

My remaining concern now is the verification ablation. The new semantic only row reaches the same recall as the full model and a mT-IoU within 0.01 of it. The gain over one sample generation therefore comes from drawing more candidates and selecting on CLAP, not from the temporal and spatial checks. The realism score is missing from this table although it carried the largest gap in the original ablation. Without that cell I cannot judge how much verification contributes beyond rejection sampling.

I also ask one clarifying question. Appendix G states that the 100 plans are manually verified but does not describe the procedure. Could the authors elaborate on how the verification was carried out. This would let a reader judge how independent the references are from the planner.

Given the above, I'm inclined to raise my score. Adding the realism results for the semantic-only variant and describing the verification procedure could be helpful to decide my final score.

Reviewer_uya5OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SoundscapeGen creates spatial soundscapes from high-level descriptions. An LLM uses an acoustic scene knowledge graph (ASKG) to plan sounds, timing, gains, and trajectories. A fine-tuned PhysWave model generates each sound separately, followed by verification, resampling, and mixing. Keeping the sounds separate also supports editing. The paper reports better event recall and listener ratings, with little change in temporal overlap compared with SoundscapeAgent.

Critical strengths
  1. I like the object-based representation. Keeping sounds separate makes timing, gain, and trajectory control straightforward. It also allows edits without regenerating every source.
  2. The event recall results are encouraging. I also appreciate that the FOA dataset is split by original recording. Keeping all spatial renderings of a recording in the same split avoids an obvious source of leakage.
Critical weaknesses
  1. I am not convinced that ASKG adds much beyond retrieved context. The generator follows PhysWave, and LLM planning is already established. Several planning gains are small, with no uncertainty estimates. The paper needs stronger evidence for what its graph and verification add.
  2. PhysWave is adapted to the new dataset, while comparable adaptation is not reported for the baselines. I cannot tell how much of the gain comes from the method rather than in-domain fine-tuning.
  3. The verification ablation gives the method more samples. It compares repeated generation and selection with a single unchecked sample. A comparison with CLAP selection under the same sampling budget is needed to show that the verification rules help.
  4. I would like a comparison with SpatialSonic from Both Ears Wide Open (https://arxiv.org/abs/2410.10676) on shared capabilities. Mono generation followed by the paper's own spatial renderer would also test whether learned FOA generation is necessary.
  5. The editing planner is underspecified, and there is no test of edit accuracy or preservation of unrelated content. Regenerating a source may also change its acoustic identity when only its trajectory should change.
  6. What does ASKG add beyond retrieved context? Please also distinguish it from EgoAVU's MCG (https://arxiv.org/abs/2602.06139).
Questions for authors

See Weaknesses

2 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for the helpful suggestions. We have revised the manuscript accordingly, with all changes highlighted in blue. Our responses are provided below.

W1 & W6: What the ASKG adds beyond retrieved context.

We agree that the ASKG acts on the LLM planner as retrieved context. Its value lies in supplying structured acoustic knowledge, especially event–event relations, that the LLM's implicit knowledge does not reliably provide. We verify this with a detailed ablation of the ASKG components (Appendix G, Table 8). All variants share the same 100 instructions, planner, prompt, and plan validation, and add knowledge cumulatively: LLM-only (no retrieved knowledge), + Scene–event, + Event–property, and + Event–event (the full ASKG). We report two rule-based metrics, namely the share of planned events that belong to the matched scene (In-scene) and the share of applicable precedes relations satisfied by the plan (Precedes), along with the overall win rate of the full ASKG in pairwise GPT-4o comparisons (both presentation orders, ties counted as 0.5; >50 favors the full ASKG). To address the reviewer's concern about uncertainty, all metrics include 95% bootstrap CIs.

VariantIn-scene (%)Precedes (%)Win rate of full (%)
LLM-only78.3 [74.3, 82.0]52 [30, 71]55.5 [50.0, 60.8]
+ Scene–event96.3 [94.4, 98.0]50 [31, 68]54.2 [48.8, 59.8]
+ Event–property98.9 [97.9, 99.8]51 [33, 69]53.7 [47.0, 60.5]
+ Event–event (full, ours)98.8 [97.3, 99.8]73 [60, 85]–
  • Event–event relations (+ Event–property vs. full) provide cross-event consistency. They raise precedes satisfaction from 51% to 73%, and the LLM judge significantly prefers the full ASKG over + Event–property on same-source consistency (win rate 57.3%, 95% CI [52.0, 62.5]). The LLM planner does not reliably produce these constraints on its own.
  • Scene–event relations (LLM-only vs. + Scene–event) ground event selection in the target scene, raising the share of in-scene events from 78.3% to 96.3%.

W6: Comparison with EgoAVU's MCG.

The two graphs differ in both purpose and content. EgoAVU targets audio-visual understanding: an LLM builds an MCG for each video clip from its captions to generate narrations and QA data for training multimodal LLMs, where nodes are objects and sounds linked to their visual sources. In contrast, the ASKG is a global knowledge base (50 scenes, 140 event categories) that constrains spatial soundscape generation. It encodes what the MCG does not, namely event properties (duration, loudness, motion) and event–event relations (precedes, same-source), which the planner turns into onsets, gains, and trajectories. We added this discussion to Section 2.

W2: In-domain fine-tuning vs. method.

Table 4 isolates the effect of FOA fine-tuning. Without it (i.e., with the original PhysWave), SoundscapeGen still achieves higher Event Recall than the strongest baseline SoundscapeAgent (90.72% vs. 88.63%), with comparable mT-IoU (0.50 vs. 0.51). Thus, the gains are not attributable to generator fine-tuning alone; our planning and verification pipeline contributes independently of it.

W5: Editing.

We added the editing prompt and output schema to Appendix K (Figure 8), alongside the editing examples. Unrelated content is preserved by construction: objects stay separate until composition, so unedited objects are reused without regeneration. As a result, removal, onset-shift, and loudness edits are exact: they delete, shift, or rescale an existing object. Edits that add an object or change its category, duration, or trajectory generate a new object, which passes through the same verification module as in the full pipeline.

We agree that regenerating a source for a trajectory edit can change its acoustic identity, and we measured this on 100 trajectory edits. The mean CLAP audio-embedding cosine similarity between the edited and original object is 0.80. This is close to the similarity between two independent samples of the same unedited plan (0.84) and far above that between objects of different classes (0.33). The edited source therefore remains the same kind of sound, and its variation is comparable to that of resampling.

AuthorsOpenReview ↗

Response (2/2)

W3: Verification under a matched budget.

Iterative resampling is an integral part of our verification module: because diffusion sampling is stochastic, the module draws new candidates until one passes all checks, so the w/o verification row uses a single candidate by construction (Appendix F). To separate CLAP-based selection from the temporal and spatial checks, we added a Semantic-only verification variant to Table 4. It uses the same per-round sampling (NcN_c=5) and resampling cap (R=5), but accepts a candidate once it passes the CLAP check, which results in a nearly identical generation budget.

VariantCand. / objectRecall (%)mT-IoUSpat. Acc. (%)
w/o verification191.650.5183.55
Semantic-only verification8.8595.360.5184.10
Full verification (ours)9.3895.360.5284.31

The two verification variants are close on scene-level averages because the temporal and spatial checks change the selected candidate for only 23 of 438 objects (5.3%). However, these 23 cases correspond to failures that CLAP cannot detect, such as a clip that matches its event label globally but is silent during its planned interval. On these 23 objects, the mean per-object Spatial Accuracy (objects silent in their planned interval counted as 0) rises from 51.9% with semantic-only selection to 86.9% with full verification.

Retry cap. 373 of 438 objects (85.2%) pass all checks within the five-round cap. The remaining 65 objects (14.8%, across 54 of the 100 scenes) fall back to the highest-CLAP candidate. Of these, 59 fail the semantic check, 7 fail the temporal check, and 2 fail the spatial check (an object can fail multiple checks). These residual failures mainly reflect the synthesis limits of the underlying generator. Because our framework is generator-agnostic, a stronger single-source generator would further reduce them.

W4: Comparison with SpatialSonic and with mono generation + our renderer.

Following the reviewer's suggestion, we added both comparisons in Appendix I (Tables 9 and 10).

SpatialSonic (BEWO; Table 9). SpatialSonic generates 10 s stereo clips, and its position control covers only frontal azimuths, so it cannot produce our FOA scenes. We therefore compare on the capability the two systems share: single-source generation with a target frontal azimuth, either static or moving left/right. For each planned object in our test scenes whose trajectory stays in the frontal half-plane, we generate one clip with the official BEWO_mix checkpoint and its position interface. Our FOA objects are decoded to binaural with the same KEMAR HRTF used for playback. SpatialSonic places sources mainly through the interaural time difference (ITD; median 0.36 ms) and produces almost no level difference (median below 0.2 dB). We therefore score direction by the sign of the ITD for all systems.

MethodAcc., static (%)Acc., moving (%)CLAP
SpatialSonic76.8 [71.1, 83.0]48.1 [38.3, 59.3]0.526
SoundscapeGen (ours)95.4 [92.3, 97.9]85.2 [76.5, 92.6]0.545

Static: sources with |azimuth| ≥ 20°. Moving: sources with |Δazimuth| ≥ 20°. Brackets denote 95% bootstrap CIs. SoundscapeGen places sources more accurately, especially for moving sources, where SpatialSonic is close to chance, while achieving slightly higher text–audio alignment.

Mono generation + our renderer (Table 10). We replaced the FOA generator with a mono TTA model, TangoFlux (Hung et al., 2026), and rendered each mono clip to FOA along its planned trajectory using our analytic renderer R in Eq. (11). Plans, verification (semantic and temporal checks), retry cap, and composition are identical to the full system.

VariantRecall (%)mT-IoUSpat. Acc. (%)
Mono TTA + renderer88.860.4799.82
SoundscapeGen (ours)95.360.5284.31

The renderer computes the exact FOA encoding of the planned direction, so its spatial accuracy is close to 100% by construction. However, its recall and temporal alignment are lower; even the original PhysWave without fine-tuning achieves higher recall (90.72%, Table 4). The two options are therefore complementary: learned FOA generation offers higher semantic and temporal fidelity, while analytic rendering offers exact direction. Since our framework is generator-agnostic, either can be used depending on the application.

Reviewer_FPrKOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SoundscapeGen constructs multi-source spatial soundscapes from high-level natural-language descriptions through a Plan–Generate–Verify–Compose pipeline. An acoustic scene knowledge graph guides event selection, temporal relationships, source properties, and motion planning. A fine-tuned PhysWave model generates individual First-Order Ambisonics sound objects, which are checked for semantic, temporal, and spatial consistency before composition. Keeping the objects separate also enables localized editing. The evaluation includes single-object generation, a 100-scene multi-source benchmark, planning evaluation, ablations, and a listening study. The reported results show improved event recall and perceived spatial consistency over the evaluated baselines.

Critical strengths
  1. The object-based representation provides useful control over individual events, including their timing, gain, and trajectories. Reusing unaffected objects during editing is a practical advantage for interactive soundscape creation (Sections 3.2 and 3.6).

  2. The experiments examine several components of the pipeline rather than only the final output. On the multi-source benchmark, the reported event recall improves from 88.63% for the strongest evaluated baseline to 95.36%, and spatial-consistency MOS improves from 3.45 to 3.78 (Table 2). These results support the practical value of the combined system.

Critical weaknesses
  1. Limited methodological novelty: LLM-based planning, audio generation, and composition are already established in frameworks such as WavJourney. The spatial generator follows the architecture and training objective of PhysWave. The main additions appear to be the acoustic scene knowledge graph, the FOA dataset and fine-tuning, and object-level verification using existing semantic, temporal, and spatial estimators. The resulting system may be practically useful, but the methodological contribution beyond integrating these elements appears limited. In particular, the ASKG ablation removes the entire knowledge context, so it does not establish how much benefit comes specifically from the event–event relations, beyond supplying scene-specific event candidates and property priors. This makes the significance of the proposed relational formulation difficult to assess.

  2. Verification gains are not clear. Appendix F states that the full system samples three candidates per round and can resample for five rounds, whereas the 'without verification' ablation uses a single candidate per object. Table 4 therefore changes both the verification procedure and the number of generated candidates. It does not isolate the benefit of verification from extra sampling and CLAP-based selection. A comparison using the same candidate-generation budget, with CLAP-only selection versus the full semantic/temporal/spatial checks, would make the contribution clearer.

Minor related-work context: Please broaden the discussion of language-guided audio generation and editing. Relevant additional context includes Audio-Agent: Leveraging LLMs For Audio Generation, Editing and Composition, which uses LLM-based task decomposition and generator calls; Both Ears Wide Open: Towards Language-Driven Spatial Audio Generation (ICLR 2025), which studies language-controlled spatial audio with moving and multiple sources; and SmartDJ: declarative audio editing with audio language model (ICLR 2026), which addresses instruction-guided stereo editing and spatial event manipulation. A brief discussion of the relevant task formulations and control capabilities would improve the paper's positioning.

Questions for authors
  1. Which benefits specifically require the event–event relations in the ASKG, beyond scene-conditioned retrieval of candidate events and typical properties? Separating these effects would help clarify the methodological contribution of the relational formulation.

  2. How much of the verification gain remains when candidate-generation budgets are matched, separating CLAP-only selection from the additional temporal/spatial checks? How often does the retry cap leave an object that still fails one or more verification criteria?

3 comments
AuthorsOpenReview ↗

Response (1/2)

We thank the reviewer for the helpful suggestions. We have revised the manuscript accordingly, with all changes highlighted in blue. Our responses are provided below.

W1 (part 1): Methodological novelty.

We agree that LLM-based planning, audio generation, and composition are established, and we do not claim the generator architecture as a contribution: our single-source generator follows PhysWave and is fine-tuned on our constructed FOA dataset to cover the unified 140-category taxonomy (Sec. 3.4, Appendix D). Our contribution lies in formulating spatial audio creation as relational soundscape generation and in addressing two gaps left open by prior frameworks within a Plan → Generate → Verify → Compose pipeline (see Table 5 in Appendix A for a functional comparison):

  • Knowledge-grounded scene planning. Prior compositional frameworks rely on the implicit knowledge of the LLM planner to infer scene structure. The ASKG supplies explicit scene–event associations, event–property priors, and event–event relations, and the ablation below shows that the relational priors substantially improve cross-event consistency.
  • Object-level verification. Prior frameworks do not verify whether each generated sound object follows the scene plan. We check every generated FOA object against its planned semantics, temporal activity, and spatial trajectory before composition.

W1 (part 2) & Q1: Contribution of the event–event relations in the ASKG.

We added a detailed ablation of the ASKG components (Appendix G, Table 8). All variants share the same 100 instructions, planner, prompt, and plan validation, and add knowledge cumulatively: LLM-only (no retrieved knowledge), + Scene–event, + Event–property, and + Event–event (the full ASKG). We report two rule-based metrics, namely the share of planned events that belong to the matched scene (In-scene) and the share of applicable precedes relations satisfied by the plan (Precedes), along with the overall win rate of the full ASKG in pairwise GPT-4o comparisons (both presentation orders, ties counted as 0.5; >50 favors the full ASKG). All metrics include 95% bootstrap CIs.

VariantIn-scene (%)Precedes (%)Win rate of full (%)
LLM-only78.3 [74.3, 82.0]52 [30, 71]55.5 [50.0, 60.8]
+ Scene–event96.3 [94.4, 98.0]50 [31, 68]54.2 [48.8, 59.8]
+ Event–property98.9 [97.9, 99.8]51 [33, 69]53.7 [47.0, 60.5]
+ Event–event (full, ours)98.8 [97.3, 99.8]73 [60, 85]–
  • Event–event relations (+ Event–property vs. full) provide cross-event consistency. They raise precedes satisfaction from 51% to 73%, and the LLM judge significantly prefers the full ASKG over + Event–property on same-source consistency (win rate 57.3%, 95% CI [52.0, 62.5]). The LLM planner does not reliably produce these constraints on its own.
  • Scene–event relations (LLM-only vs. + Scene–event) ground event selection in the target scene, raising the share of in-scene events from 78.3% to 96.3%.
AuthorsOpenReview ↗

Response (2/2)

W2 & Q2: Verification under a matched budget.

Iterative resampling is an integral part of our verification module: because diffusion sampling is stochastic, the module draws new candidates until one passes all checks, so the w/o verification row uses a single candidate by construction (Appendix F). To separate CLAP-based selection from the temporal and spatial checks, we added a Semantic-only verification variant to Table 4. It uses the same per-round sampling (NcN_c=5) and resampling cap (R=5), but accepts a candidate once it passes the CLAP check, which results in a nearly identical generation budget. We also corrected a typo in the original submission: the system samples NcN_c=5 candidates per round, not three.

VariantCand. / objectRecall (%)mT-IoUSpat. Acc. (%)
w/o verification191.650.5183.55
Semantic-only verification8.8595.360.5184.10
Full verification (ours)9.3895.360.5284.31

The two verification variants are close on scene-level averages because the temporal and spatial checks change the selected candidate for only 23 of 438 objects (5.3%). However, these 23 cases correspond to failures that CLAP cannot detect, such as a clip that matches its event label globally but is silent during its planned interval. On these 23 objects, the mean per-object Spatial Accuracy (objects silent in their planned interval counted as 0) rises from 51.9% with semantic-only selection to 86.9% with full verification.

Retry cap. 373 of 438 objects (85.2%) pass all checks within the five-round cap. The remaining 65 objects (14.8%, across 54 of the 100 scenes) fall back to the highest-CLAP candidate. Of these, 59 fail the semantic check, 7 fail the temporal check, and 2 fail the spatial check (an object can fail multiple checks). These residual failures mainly reflect the synthesis limits of the underlying generator. Because our framework is generator-agnostic, a stronger single-source generator would further reduce them.

W3: Related work.

We added Audio-Agent, BEWO, and SmartDJ to Section 2 and discussed how their task formulations and control capabilities relate to ours. In short, BEWO generates the entire spatial scene as a single stereo mixture, Audio-Agent decomposes the task with an LLM but produces monaural audio without verification, and SmartDJ edits the stereo mixture directly. In contrast, SoundscapeGen plans and verifies individual FOA objects, and an edit modifies only the affected objects.

Dear Reviewer,

Thank you for your valuable suggestions. Could you please let us know whether our revisions have addressed your concerns? We would appreciate your feedback.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.