acceptodds
Under review as a conference paper at ICLR 2027

Where to Perturb: Reference-Structured Corruption for One-Shot SFX Variation

Abstract

One-shot SFX variation requires generating diverse yet recognizable alternatives from a single reference. Limited variability can lead to perceptual repetition in interactive media (e.g. video games), while manually designing variations is costly. Yet reference-guided flow models typically condition on the reference while applying a largely uniform corruption level across time, despite its heterogeneous acoustic structure. While recent asynchronous flow methods demonstrate that corruption can vary across a sequence, they do not specifically exploit the internal structure of an audio reference to allocate stochasticity for identity-preserving variation. We ask whether the internal structure of an audio latent can directly determine where and how strongly stochasticity should be introduced. We propose a Reference-Structured Corruption (RSC) that transforms a single reference into multiple stochastic trajectories without auxiliary conditioning or per-reference optimization. The method partitions the latent into content-aligned regions from latent geometry and activity, then assigns activity-conditioned regional flow times, yielding continuous control between preservation and regeneration. On Woosh, our method increases within-reference diversity from to WCIS while maintaining identity drift close to the level observed in real production SFX variations, and improves FAD from to . On Stable Audio 3, it similarly increases WCIS from to while preserving generation quality ( vs. FAD). These results show that the learned latent representation can serve not only as the generation space, but also as a signal for structuring the stochastic process itself. An anonymized, frozen demo page is available online. Code is included in the supplementary material and will be released publicly.

Primary area:
generative models
Keywords:
audio generation, sound effects, reference-based generation, latent diffusion, flow matching, stochastic variation, adaptive corruption, asynchronous denoising, latent representations, identity-preserving generation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_ikrZOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Reference-Structured Corruption (RSC), a training-free method for generating variations of a sound effect from a single reference. It divides the reference audio latent into temporal regions and assigns different corruption levels according to each region’s activity, allowing some parts to change more than others. The method is evaluated on Woosh and Stable Audio 3 using objective metrics, comparisons with existing methods, ablation studies, and a small human evaluation. Results show increased variation while maintaining similar generation quality, although the trade-off between diversity and preservation remains, and the method requires adapting the pretrained flow models to support different corruption levels across time.

Critical strengths
  1. The main strength is that the method uses the structure of the reference audio to decide where to introduce more variation, rather than applying the same corruption level across the whole clip. This is a simple and useful idea for generating sound effects that remain recognizable but are not identical.

  2. The authors test the method on two different flow models and include ablations and a human evaluation. The results show increased diversity with relatively small changes in generation quality, suggesting that the approach is not limited to a single backbone.

Critical weaknesses
  1. The improvement in diversity is relatively modest, and the trade-off with reference preservation is still present. On Woosh, WCIS increases from 1.29 to 1.41, while identity drift also increases. The results do not clearly show that the method produces substantially better variations than existing approaches.

  2. The human evaluation is limited to 7 reference sounds and 15 participants. The results show a trade-off between diversity and identity preservation, but the study is too small to establish that the method consistently produces useful variations across different types of sound effects. In addition, the method requires adapting the pretrained flow model to handle asynchronous corruption, so it is not entirely training-free in practice.

Questions for authors
  1. Can the authors clarify how much of the improvement comes from the activity-based allocation itself, compared with simply using different corruption levels across temporal regions?

  2. In the human evaluation, the method does not consistently improve both diversity and identity preservation. Do the authors have results on more reference sounds that show whether this trade-off holds across different types of SFX?

4 comments
AuthorsOpenReview ↗

COMMENT 1/2: Concerns

Thank you for the supportive review and the two precise questions. You mentioned the simplicity and usefulness of the idea, while suggesting our method is not limited to a single backbone, with extended ablations and results, and that the method allows increased diversity with relatively small impact on generation quality. We answer the rest of your concerns below.

D1) Modest diversity gain and the preservation–variation trade-off. After harmonising the protocol, the WCIS gain no longer holds at U(0.7,0.85)\mathcal{U}(0.7, 0.85): random chunks and flow times reach 1.371.37 and RSC 1.331.33 (Tab. 2). The new Fig. 5 compares every corruption method against RSC consistently: against the same Woosh+DF baseline (Random Gk,τkG_k, \tau_k), RSC outperforms it at 4 of 7 intervals on Woosh and 5 of 7 on SAO3. Further, a counter-intuitive result of our calibration study is that production variations are only moderately diverse: on CLAP-LAION they reach a diversity of 0.210.21 against an identity of 0.790.79, so more diversity is not better in itself. The trade-off you point out, where more diversity comes with more identity drift, is the main limitation when generating variations of a sample: a stronger corruption regenerates more of the reference and raises diversity, but often at the cost of the elements that carry its identity, as the starting level does in SDEdit (Meng et al., 2022). The new Fig. 5 shows that RSC improves this balance. At matched flow-time intervals, it places more variations inside the production region than the regional and global corruption variants at four to seven of seven intervals on Woosh and SAO3, mainly at the higher-centre intervals, where the activity map keeps more of the identity; ODE inpainting on Woosh is the exception, doing better at five of seven. Against text-to-audio and audio-text-to-audio generation, the new Fig. 4 shows the gap that reference corruption fills without any text guidance (CLAP-LAION identity 0.780.78 against at most 0.240.24).

Comparison to existing approach. We add 3 baselines to Tab.1 and Fig. 4, and we show a clear advantage of RSC in the preservation-variation frontier; RSC outperforms all three in quality (P%P\%, R%R\%, FAD) and identity (CLAP-LAION 0.480.48 vs. at most 0.240.24). To evaluate them on a shared protocol, we generate the same 1,500 references, 10 variations each. Qwen3-Omni-Captioner (Xu et al., 2025) describes each reference, and Qwen3-Omni-Instruct (Xu et al., 2025) condenses the description into one 35-45 word sentence. From this caption, we generate with (i) Woosh text-to-audio (Hadjeres et al., 2026), (ii) Stable Audio 3 small-sfx text-to-audio (Evans et al., 2026), and (iii) Stable Audio 3 small-sfx from the reference audio (default method init noise level of 0.9) with the prompt "create a variation of this sample that is ⟨caption⟩". While it is true that the baselines come closer to Production SFX diversity, their variations no longer match the reference.

D2) Training-free statement. We agree that the training-free claim was ambiguous. RSC is an inference-time method with no per-reference training or optimization needed; it runs on a backbone adapted once to accept frame-wise flow times. On Woosh, even that adaptation is optional, since its timestep embedding can be evaluated per frame: we evaluate RSC on the released weights and show that the results remain close after adaptation; CLAP identity 0.80→0.780.80\rightarrow 0.78, P$%$ 0.93→0.900.93\rightarrow0.90, R$%$ 0.85→0.870.85\rightarrow0.87 and FAD 4.68→4.324.68\rightarrow4.32 (Tab.1). For the sake of clarity we replace training-free to inference-time method. We clarify the scope in the Introduction section.

AuthorsOpenReview ↗

COMMENT 2/2: Q&A

Q1) How much of the improvement comes from the activity-based allocation itself. We add a corruption variant of RSC derived from the relative activity corruption, which means it relies on the content-aligned chunking but with random flow times applied to each chunk. We compare in Fig.5, on matching flow-time intervals, how many generated variations of RSC and of each corruption method fall closer to the Production SFX ground truth, in the CLAPLAION\text{CLAP}_{\text{LAION}} identity-diversity plane (α\alpha-precision (Alaa et al., 2022)). Against content-aligned GkG_k with random τk\tau_k, and at restricted high corruption flow field intervals, RSC outperforms 4 out of 7 intervals on woosh and 5 of them on SAO3.

Q2) Human evaluation limitation. We agree that the limited number of references and participants is insufficient to establish clear production usefulness. We now explicitly acknowledge this limitation in the main paper and provide a thorough description of the evaluation protocol and its limitations in the appendix, including the wide confidence intervals, differences in stimulus formats, and inconsistent interpretations of the scoring instructions. We report these results transparently rather than claiming conclusive perceptual improvements, and highlight the need for larger and more reliable listening studies.

Evaluation and comparison of method on public dataset FSD50K. To answer the lack of reproducibility and open-source, we introduce a new ablation as a stratified evaluation on a subset of the FSD50K (CC-BY-NC). We evaluate each method's ability to generate variations across different categories (following the UCS-SFX ontology). We report results in Tab.10-11 and Fig.6. For fairness and comparison, we add 3 baselines to Tab.1 and Fig. 4, where we set up a shared protocol: the same 1,500 references, 10 variations each. Qwen3-Omni-Captioner describes each reference, and Qwen3-Omni-Instruct condenses the description into one 35-45 word sentence. From this caption, we generate with (i) Woosh text-to-audio, (ii) Stable Audio 3 small-sfx text-to-audio, and (iii) Stable Audio 3 small-sfx from the reference audio (default method init noise level of 0.9) with the prompt "create a variation of this sample that is ⟨caption⟩". RSC outperforms all three in quality (P%P\%, R%R\%, FAD) and identity (CLAPLAION\text{CLAP}_{\mathrm{LAION}} 0.780.78 vs. at most 0.240.24). The baselines come closer to Production SFX diversity, but their variations no longer match the reference. We add new citations and clarify the open-source status of the SFX high-quality generation methods in the Related Works section.

AuthorsOpenReview ↗

Gentle Reminder: revised version and replies

Dear Reviewer ikrZ, thank you again for the supportive review. As the discussion period closes on 18 November, we kindly remind you that the revised version of the paper is uploaded. We suggest starting with the new Fig. 5 (Sec. 4.2).

D1) Modest diversity gain. After harmonising the protocol, the WCIS gain at U(0.7,0.85)\mathcal{U}(0.7, 0.85) no longer holds (Tab. 2); since production variations are only moderately diverse (CLAP diversity 0.21 at identity 0.79), the paper now targets the production region rather than more diversity. Against 3 caption-based baselines, RSC keeps CLAP identity 0.78 against at most 0.24 (Tab. 1, Fig. 4).

D2) Training-free statement. Replaced by inference-time method; on Woosh, RSC also runs on the released weights (Tab. 1).

Q1) Activity-based allocation itself. Against content-aligned GkG_k with random τk\tau_k, RSC is better at 4 of 7 intervals on Woosh and 5 of 7 on SAO3, mainly at higher corruption levels (Fig. 5).

Q2) Human evaluation and more SFX types. The study is reported as a mixed result (A.3), and the FSD50K evaluation by UCS category gives an objective, though not perceptual, view across SFX types (Tables 10–11, Fig. 6).

We remain available for any further question before the deadline.

Thank you for the detailed response. The rebuttal and additional experiments have addressed my concerns, so I will keep my positive recommendation.

Reviewer_2jv1OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the one-shot SFX variation problem, given a single reference sound, generate diverse variations that still keep its identity. The authors propose Reference-Structured Corruption (RSC). Instead of noising the whole reference latent to one global flow time, as in SDEdit, they split the latent into temporal chunks using change points in the latent itself. Each chunk gets a randomly sampled flow time, which is then pushed up or down depending on how "active" the chunk is relative to the whole reference.

Critical strengths
  1. There is a practical need for this, the game sound effects need variation to avoid repetition.
Critical weaknesses
  1. The method is basically SDEdit applied per segment, with a heuristic segmentation step. Using different noise levels for different parts of a sequence has already been done in Diffusion Forcing and recent asynchronous flow methods.

  2. The method is not really training-free, as claimed. It needs Diffusion Forcing fine-tuning of the backbone so it can take per-frame flow times. It also adds inference cost, e.g., SAO3 goes from 10 to 48 NFE.

  3. The experiments don't support the claims:

  • (1) all the datasets used are internal data, so the results can't be checked or compared with other methods.

  • (2) the method shows no clear advantage over the simple global flow-time baseline in Table 1.

  • (3) the human study in Fig. 5 points the other way against the claims, Woosh+Ours is rated lower than plain Woosh on both diversity (42.6 vs. 43.7) and identity (57.1 vs. 61.0).

  • (4) in Table 6, simply changing the sampling range from U(0.70, 0.85) to U(0.75, 0.85) gives almost the same numbers as the full method. This suggests the method's effect is mostly the same as raising the overall noise level.

Questions for authors

See weakness.

4 comments
AuthorsOpenReview ↗

COMMENT 1/2: Concerns

Thank you for the direct review. You highlighted the direct need for our method in order to avoid repetition in SFX variations. Although several of your points are correct, we address your concerns below and revise the paper accordingly.

C1) SDEdit per segment; Diffusion forcing and asynchronous flows. Our contribution lies in how a non-uniform corruption is automatically structured and allocated from a single untouched reference, without human interaction or additional guidance through textual, visual, or audio cues. Unlike SDEdit (Meng et al., 2022), which relies on a global corruption level applied to the user-provided guide, RSC automatically determines where and how strongly to introduce corruption to balance variation and identity preservation. We agree that the underlying mechanism follows SDEdit-style partial noising and that heterogeneous flow levels build on Diffusion Forcing (Chen et al., 2024) and asynchronous flows. We claim neither, and the revision now acknowledges these connections in Sec.1 and Sec.2. A closer prior work is Differential Diffusion (Levin & Fried, 2025), which applies spatially varying corruption strengths using a user-supplied change map without retraining, whereas RSC derives both the regions and their corruption strengths directly from the reference. We have accordingly updated the Related Work (Sec. 2) to discuss SDEdit and Differential Diffusion, clarifying their connections to RSC and distinguishing our contribution through the reference-driven allocation of corruption strengths. We also report a (0.5s) clear-context inpainting baseline with the SDE solvers on Woosh (Tab. 2 and Fig.5) and show that RSC outperforms the SDE-inpainting on 44 out of 77 matching flow-time intervals. Further, in Tab. 2, SDE-inpainting holds a high precision score of 0.990.99 at the cost of recall 0.560.56 against 0.890.89 for RSC. On WCIS, CLAP diversity and eJSDe_JSD metrics RSC offers a closer identity-diversity trade-off while being closer to the SFX production ground truth, while SDE inpainting is slightly closer on CLAP identity (−0.05−0.05 against −0.07−0.07). This overall highlights a continuous control between preservation and regeneration derived from the reference without per-reference optimization.

C2) Concerns on segmentation being heuristic. The segmentation is a fixed rule on a frozen latent; it does depend on the encoder window size and thus holds 4 tunable hyper-parameters (contrastive window size WW, pp peak prominence threshold, dd minimum separation between selected peaks and minimum region length mminm_{min}). We do not claim the rule is optimal, and RSC does not rely on it being so. (i) It is an inference-time interface. Both were adapted on random contiguous partitions with independent flow levels (diffusion forcing), which is what RSC produces. We claim inference-time further by running RSC on the base Woosh checkpoint with no fine-tuning (Tab.1); made possible thanks to the per-frame sampling schedule of the Woosh-Flow model. (ii) The results are not driven by GkG_k itself, in Fig.5 we show a heatmap of better identity-diversity trade-offs of RSC against other methods and replacing content-aligned cuts GkG_k with random barely changes the CLAPLAION\text{CLAP}_{\mathrm{LAION}} metrics, while on the same chunks GkG_k the activity-driven τk\tau_k improves results on four out of 7 flow time scales. (iii) The hyper-parameters were not tuned on the evaluation set. For both backbones, we highlight in Appendix Tab.18 and Tab.19 the hyper-parameter sweep results on 100 reference samples disjoint from the evaluation set.

C3) Training-free statement. We agree that the training-free claim was ambiguous. RSC is an inference-time method with no per-reference training or optimization needed; it runs on a backbone adapted once to accept frame-wise flow times. On Woosh, even that adaptation is optional, since its timestep embedding can be evaluated per frame: we evaluate RSC on the released weights and show that the results remain close after adaptation; CLAP identity 0.80→0.780.80\rightarrow 0.78, P$%$ 0.93→0.900.93\rightarrow0.90, R$%$ 0.85→0.870.85\rightarrow0.87 and FAD 4.68→4.324.68\rightarrow4.32 (Tab.1). For the sake of clarity we replace training-free to inference-time method. We clarify the scope in the Introduction section. Inference cost concerns. The increase from 10 to 48 NFE on SAO3 comes from the sampler, not from RSC: for a fair comparison, every corruption scheme uses the same 24-step Heun solver, and the regional field itself adds no network evaluation. Reducing the per-frame Heun sampler from 48 to 10 NFE makes RSC sampling on SAO3 4.7×4.7\times faster (3.433.43sec →0.73\rightarrow 0.73sec per variation) with no significant change in the preservation-variation balance ($\text{CLAP}_{LAION} $identity 0.603→0.6000.603\rightarrow0.600, diversity 0.241→0.2400.241\rightarrow 0.240), and only the small quality changes expected when reducing NFE (Tab.17).

AuthorsOpenReview ↗

COMMENT 2/2: Q&A

Q1) Evaluation on public sound collection. We added a public evaluation on 500 FSD50K references (CC-BY-NC subset), grouped into UCS categories with the relabeling of Beck & Lerch (2026), where RSC and each corruption variant are evaluated at their best selected [τmin,τmax][\tau_{min},\tau_{max}] interval (Tables 10–11, Fig. 6). Fig. 6, uses RSC (inference-time), shows that its advantage depends on the sound category: structured regional allocation is closer to production than random τk\tau_k on human, weather, birds, footsteps, crowds and doors, and closer than a global τ\tau on human, fire, objects, doors and wood, while other relatively larger and monotone sound categories favour simpler global corruption (e.g. animals). Overall, RSC remains competitive with the independently selected baselines, although the differences in the calibrated CLAP identity–diversity distance are small (within 0.01). This may partly come from the selected intervals: at matching intervals on the internal set, the new Fig. 5 shows that activity-relative corruption matters most at restricted and high corruption levels.

Comparison with open-source baselines. We add 3 baselines to Tab.1 and Fig. 4, and we show a clear advantage of RSC in the preservation-variation frontier; RSC outperforms all three in quality (P%P\%, R%R\%, FAD) and identity (CLAPLAION\text{CLAP}_{LAION} 0.480.48 vs. at most 0.240.24). To evaluate them on a shared protocol, we generate the same 1,500 references, 10 variations each. Qwen3-Omni-Captioner (Xu et al., 2025) describes each reference, and Qwen3-Omni-Instruct (Xu et al., 2025) condenses the description into one 35-45 word sentence. From this caption, we generate with (i) Woosh text-to-audio (Hadjeres et al., 2026), (ii) Stable Audio 3 small-sfx text-to-audio (Evans et al., 2026), and (iii) Stable Audio 3 small-sfx from the reference audio (default method init noise level of 0.9) with the prompt "create a variation of this sample that is ⟨caption⟩". The baselines come closer to Production SFX diversity, but their variations no longer match the reference.

Q2) Comparison against a global flow time τ\boldsymbol{\tau}. At the flow-time interval [0.7,0.85][0.7, 0.85] of the former Tab.1, now Tab.2, the global τ\tau does score closest to the ground truth on within-reference diversity and identity drift (WCIS at −0.05-0.05 against −0.19-0.19 for RSC, and eJSDe_{JSD} at −0.05-0.05 against −0.09-0.09). The CLAPLAION\text{CLAP}_{\text{LAION}} identity-diversity pair points the other way: the global τ\tau is further from the diversity ground truth (+0.13+0.13 against +0.04+0.04) and twice as far from the identity ground truth (−0.14-0.14 against −0.07-0.07). (i) These scores alone are not enough to tell the two methods apart. (ii) We select the metric pair that best separates within-identity from between-identity variations in our calibration study (Fig.11), LAION-CLAP, and use it as the main basis of comparison. We then add another visualization of results: Fig.5 compares each corruption method against RSC at matched flow-time intervals, by how well it fits the ground-truth distribution. Against the global τ\tau, RSC wins at 5 of 7 intervals on Woosh and 6 of 7 on SAO3.

Q3) Human Evaluation. We agree that human evaluation does not show a clear improvement over other corruption methods. We have revised the main paper to explicitly acknowledge these negative results and the limitations of the study, including the small number of participants and references, wide confidence intervals, and differing interpretations of the scoring instructions. We also added a thorough description of the evaluation protocol and its limitations in the appendix to provide transparency and help inform future listening tests for SFX variation application.

Q4) Precision on sampling range impact. We clarify that Gk+τk∼U(0.75,0.85)G_k + \tau_k \sim \mathcal{U}(0.75, 0.85) and Gk+τk∼U(0.80,0.85)G_k + \tau_k \sim \mathcal{U}(0.80, 0.85) (in the submitted version, now in A.1.1 / Tab.5) are both configurations of the full RSC method, evaluated at narrower corruption intervals. The reviewer highlights that changing the sampling range from U(0.70,0.85)\mathcal{U}(0.70, 0.85) to U(0.75,0.85)\mathcal{U}(0.75, 0.85) produces almost identical results to the full method. However, these configurations already include both content-aligned chunking and activity-conditioned allocation. Their similar results indicate that RSC maintains a stable preservation-variation balance when narrowing the flow-time interval toward higher corruption levels.

AuthorsOpenReview ↗

Gentle Reminder: revised version and replies

Dear Reviewer 2jv1, as the discussion period closes on 18 November, we kindly remind you that our two comments above address each of your concerns and that the revised version of the paper is uploaded. We suggest starting with Sec. 2, then the new Fig. 5.

C1) SDEdit per segment. We agree that the mechanism follows SDEdit-style partial noising and builds on Diffusion Forcing, and claim neither; Secs. 1–2 now discuss SDEdit and Differential Diffusion, whose change map is user-supplied, whereas RSC derives it from the reference.

C2) Heuristic segmentation. Replacing GkG_k with random cuts barely changes the CLAP-LAION metrics, while the activity-driven τk\tau_k improves results at 4 of 7 intervals (Fig. 5); the hyper-parameters were tuned on 100 references disjoint from the evaluation set (Tabs. 18–19).

C3) Training-free and cost. Replaced by inference-time method; at $\sim$10 NFE on SAO3, RSC runs 4.7×4.7\times faster with identity and diversity unchanged (Tab. 17).

Q1) Public data and baselines. 500 FSD50K references (Tables 10–11, Fig. 6) and 3 open baselines, against which RSC keeps CLAP identity 0.78 against at most 0.24 (Tab. 1, Fig. 4).

Q2) Global flow time. At matched intervals, RSC is closer to the Production SFX ground truth at 5 of 7 intervals on Woosh and 6 of 7 on SAO3 (Fig. 5).

Q3) Human evaluation. Now reported as a negative result, with its limitations (A.3).

Q4) Sampling range. The U(0.75,0.85)\mathcal{U}(0.75, 0.85) rows were already the full RSC method at a narrower interval (A.1.1, Tab. 5).

If some of these answers do not resolve your concerns, we would appreciate knowing which ones while we can still reply.

Thank you for the detailed response. The clarifications on the training-free wording, the NFE cost, and running RSC on the released Woosh weights address my concerns on those points. However, my main concerns remain.

  • First, the response confirms that the mechanism is SDEdit-style partial noising with Diffusion Forcing–style per-frame flow times, and the newly cited Differential Diffusion already applies region-wise corruption strengths without retraining. The remaining contribution is a heuristic allocation map, and the response says that replacing the content-aligned cuts with random cuts barely changes the metrics.

  • The reported effect has reversed without explanation. Eq. 10 is the same as Eq. 9 in the submission, but the effect of λ now points the other way. In the submission, raising λ moved RSC toward more variation (WCIS 1.36 → 1.41, identity 0.70 → 0.67). In the revision, it moves toward more preservation (1.35 → 1.33, 0.71 → 0.72). The original "full method" result at U(0.7, 0.85) (WCIS 1.41, CLAP div 0.31, identity 0.67) now essentially matches the "inverted map" in Table 7 (1.40, 0.31, 0.67). The abstract's 1.29 → 1.41 claim has also been removed. The evaluation reference values changed between versions, but the baselines barely moved (global τ: 1.46/0.23 → 1.47/0.24; content-aligned chunks with random τ_k: 1.35/0.20 → 1.36/0.21). Only RSC changed.

  • The ablation I asked about was removed, not moved. The response says the U(0.75, 0.85) rows are now in A.1.1 / Table 5, but no U(0.75, 0.85) configuration appears anywhere in the revision.

  • The FSD50K differences are small, the new caption-based baselines lose identity by construction, and the human study is negative. I don't think the revision supports the main claim, and I keep my score.

Reviewer_17uPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Reference-Structured Corruption (RSC) for generating diverse, identity-preserving sound-effect variations from a single audio reference. The method partitions the reference latent into temporal regions using latent contrast and silence-relative activity, defined as the L2 distance between each latent frame and a fixed latent representation of silence. The method then applies different randomly sampled flow times for each audio segment based on its relative activity, assigning stronger corruption to more active regions to allow greater variation during generation. The authors experimented with Woosh and StableAudio 3 backbones, with both being fine-tuned for asynchronous flow. Experiments on 500 internal SFX references, together with ablations and a human evaluation, examine the trade-off between variation diversity, reference identity preservation, and generation quality.

Critical strengths
  • The method is simple, and the evidence supports the effectiveness of content-aligned chunking. In Table 6, the proposed chunking mechanism reduces FAD and increases recall from 4.82 to 4.37 and from 0.77 to 0.88 compared to random chunking, respectively. This supports the usefulness of reference structure for defining corruption regions.

  • The method offers a controllable identity-diversity trade-off without requiring per-reference training once the backbones have been fine-tuned for asynchronous flow. In Table 2, increasing the activity-conditioning strength increases both the diversity and the identity drift.

  • The author provides thorough ablation studies, which help understand their critical design choices.

Critical weaknesses
  • The rationale behind the activity-condition is insufficiently established. In Section 3.2, equation 4, the method uses the L2 distance to a silence latent as an activity proxy to assign stronger corruption to more active regions. However, in the same section, Line 209 notes that "digital silence does not encode to the origin". This indicates that even silent frames could receive nonzero activity scores, which are unrelated to acoustic intensity. Additionally, even if the proxy measure activity, it does not establish which regions can tolerate stronger corruption without losing identity. Without evidence that activity predicts tolerance to corruption, it remains unclear whether the proposed allocation improves the identity-diversity trade-off beyond a suitably tuned random allocation.

  • Comparisons with existing SFX variation methods are limited. Although Section 2 discusses relevant approaches such as SpecSinGAN and Simi-SFX, Table 1 primarily compares backbone variants and corruption/inpainting configurations. These comparisons help isolate the effects of RSC but do not establish its competitiveness against existing solutions to the task.

  • The asynchronous inference procedure is underspecified. Algorithm 2 summarizes denoising as an ODE solve without defining how regional flow times progress toward zero, how velocity updates account for different regional time increments, or how regions that reach zero are handled. Although Table 9 reports NFE, this does not sufficiently specify the sampling procedure for reproduction.

  • The flow-time analysis in Appendix A.1.1 does not separate the effects of interval location and width. Table 5 changes both simultaneously, making it unclear which factor drives the observed diversity–identity trade-off. Varying the interval location at a fixed width and the width at a fixed midpoint would provide a more controlled analysis.

Questions for authors
  1. Can you verify whether silence representations remain stable across temporal positions and input lengths? If they vary, then a non-active (silence) audio segment can produce a non-uniform latent representation, which weakens the relation between latent magnitude and activity.

  2. Could you compare RSC with compatible SFX variation methods discussed in Section 2, or clarify which differences in their settings prevent a meaningful comparison? How would the methods be evaluated under a shared reference and adaptation protocol?

  3. Could you specify the asynchronous sampling procedure underlying Algorithm 2, including the regional flow-time schedules, velocity update equations, solver, and treatment of regions that reach zero before others?

  4. Could you provide more details about the 500 internal SFX references, including their source, category distribution, and selection criteria?

  5. Could you improve the presentation by using narrative citations where author names form part of the sentence and placing tables and figures closer to their first references? The current citation formatting and placement of some tables and figures make the paper harder to follow. For example, Figure 7 is referred to in Section 3.2, line 215, but is placed at the end of the appendix.

4 comments
AuthorsOpenReview ↗

COMMENT 1/2: Concerns

Thank you for the precise questions, several of which led to new experiments. Overall, you mentioned a simple method with evidence supporting the effectiveness of the content-aligned chunking; you also highlight the controllability of RSC method over the identity-diversity trade-off without requiring further training once the backbones handle asynchronous flows. Further, you underlined the thoroughly motivated design choices. We answer your concerns below.

B1) Rationale of activity condition. Silent latent as an activity proxy. Your concern over silent frames, which could receive nonzero activity scores (unrelated to acoustic intensity) is the reason why we do not estimate the norm of the latent for the activity but subtract zsilz_{sil} (Eq. 5). Activity predicts tolerance to corruption at high noise range. To address whether activity predicts tolerance to corruption, the new Fig. 5 compares, on matching flow-time intervals, how many generated variations of RSC and of each corruption method fall closer to the Production SFX ground truth, in the CLAP-LAION identity-diversity plane (α\alpha-precision (Alaa et al., 2022)). Against content-aligned GkG_k with random τk\tau_k, and at restricted high corruption flow field intervals, RSC outperforms 4 out of 7 intervals on Woosh and 5 of them on SAO3. Inverted activity mapping. To answer which regions can tolerate stronger corruption without losing identity, we ran the inverted allocation: τk∗=(1−λ∣rk∣)τk+λ∣rk∣a~k,a~k=τmin⁡+τmax⁡−ak,\tau_k^{\ast} = \left(1-\lambda |r_k|\right)\tau_k + \lambda |r_k| \tilde a_k, \qquad \tilde a_k = \tau_{\min} + \tau_{\max} - a_k, i.e. a~k=τmin⁡\tilde a_k = \tau_{\min} if rk≥0r_k \ge 0 and τmax⁡\tau_{\max} otherwise (Eq. 11), at all seven flow-time intervals on 1,5001,500 reference samples. In Fig. 7 (+Tab. 7), we log a heatmap comparing trade-offs in the CLAP-LAION identity–diversity plane, highlighting which setup is closer to our Production SFX calibration (ground truth). (i) The signs matter to preserve the identity of the sound, as rkr_k constantly preserves identity over −rk-r_k at every interval. (ii) On the identity-diversity trade-off, rkr_k is consistently closer to ground truth. (iii) Inverting the map moves each interval towards diversity at the cost of losing the identity. The activity map rkr_k avoids this by construction.

B2) Location and width of the flow-time interval. For better assertion of difference, we added a column distinguishing interval location and width of flow fields in Tab. 5 and 6, and further added two flow field scales of U(0.625,0.925)\mathcal{U}(0.625, 0.925) and U(0.55,1)\mathcal{U}(0.55, 1). Further to assert which of these factors drives the identity-diversity trade-off, the new visualization in Fig.5, comparing corruption methods at matched flow-time intervals, allows us to say that the location(center) of the interval has a higher impact on the identity-diversity trade-off; at a higher corruption level (4-5/7), RSC outperforms all corruption schemes on Woosh and SAO3 backbones, ODE inpainting remains closer at five of seven intervals.

AuthorsOpenReview ↗

COMMENT 2/2: Q&A

Q1) Stability of the silence representation. We add an ablation in Appendix, Fig.10, to show how encoded silence of different lengths and positions behaves in the latent activity signal. The stability of the silence representation depends on the architecture of the encoder itself. Woosh is a fully convolutional encoder: it outputs one latent every 10,ms, each computed within a local receptive field of about 90 ms90\,ms, which means a moment of silence, whatever its length or position, is always encoded the same. Whereas your point is fair for a transformer encoder such as SAO3's, whose attention spans about ±1,s\pm 1,s of the sequence; this gives no such guarantee, yet its silence latents deviate from zsilz_{\mathrm{sil}} by only 2−10%2-10\% of a median sounding frame across lengths, and by at most 2%2\% across positions. Thus, in both encoders the activity of silence remains below that of a −40-40dB frame, so a(t)a(t) never reads silence as activity.

Q2) Compatible and available comparative baselines. We add 3 baselines to Tab.1 and Fig.4. The SFX variation methods of Sec.2 (SimiSFX (Liu & Jin, 2024), SpecSinGAN (Barahona Rios & Collins, 2022)) either do not generate audio directly or have no public implementation, and few open models produce production-quality 48 kHz audio. We set up a shared protocol: the same 1,500 references, 10 variations each. Qwen3-Omni-Captioner (Xu et al., 2025) describes each reference, and Qwen3-Omni-Instruct (Xu et al., 2025) condenses the description into one 35-45 word sentence. From this caption, we generate with (i) Woosh text-to-audio (Hadjeres et al., 2026), (ii) Stable Audio 3 small-sfx text-to-audio (Evans et al., 2026), and (iii) Stable Audio 3 small-sfx from the reference audio (default method init noise level of 0.9) with the prompt "create a variation of this sample that is ⟨caption⟩". RSC outperforms all three on each backbone in quality (P%P\%, R%R\%, FAD) and identity (CLAPLAION\text{CLAP}_{LAION} 0.780.78 on Woosh and 0.680.68 on SAO3 vs. at most 0.240.24, Tab.1). The baselines come closer to Production SFX diversity, but their variations no longer match the reference.

Q3) Sampling schedule (frame-wise). Given the field τ\boldsymbol{\tau} of new Eq.4, all frames are integrated on one shared clock of SS number of steps (Heun sampler at 24steps) as \begin{equation} \tau^{(n)}(t)=(1-\frac{n}{S})\tau(t), \quad n=0,\ldots,S \end{equation} so each frame follows its own straight path from τ(t)\tau(t) to 0 with step size h(t)=τ(t)Sh(t)=\frac{\tau(t)}{S}, and all frames reach τ=0\tau=0 together at n=Sn=S (Eq. 4, Alg. 2). No region finishes early or waits for the others, and a frame with initial τ(t)=0\tau(t)=0 has h(t)=0h(t)=0 and stays clean. The network receives the full field τ(n)\tau^{(n)} at each step, so the regional field adds no function evaluation: 24 Heun steps with two calls (2$^{\text{nd}}$order) each give 48 NFE.

Q4) UCS Ontology on SFX references. We report on both used datasets (Internal and FSD50K (CC-BY-NC subset)) the ontology of each referenced sample used for evaluation (grouped into UCS categories with the relabeling of Beck & Lerch (2026)). We will further disclose (on the GitHub repo) the selected set of uni-labelled reference samples from FSD50K CC-BY-NC, unseen in Woosh pretraining.

Q5) Presentation and readability. The former Fig. 7 is now Fig. 3, in Sec. 3.2 next to its first mention, and the main-text tables and figures now sit next to their first reference. We switched to narrative citations where the authors are part of the sentence. Results are regrouped into Fig. 4 (RSC against generation baselines) and Fig. 5 (RSC against corruption schemes), and every calibration column is reported as a signed difference from Production SFX with its standard deviation. We strongly encourage the reviewer to take a look at the new version of the paper.

AuthorsOpenReview ↗

Gentle Reminder: revised version and replies

Dear Reviewer 17uP, as the discussion period closes on 18 November, we kindly remind you that our two comments above answer each of your questions, several of which led to new experiments, and that the revised version of the paper is uploaded. We suggest starting with Fig. 7 and the new Fig. 5.

B1) Rationale of the activity condition. Inverting the activity map loses identity at every interval (Fig. 7, Tab. 7), and against content-aligned GkG_k with random τk\tau_k, RSC is better at 4 of 7 intervals on Woosh and 5 of 7 on SAO3 (Fig. 5).

B2) Location and width of the interval. Tables 5 and 6 now separate both, and the location (centre) has the larger impact on the trade-off (Fig. 5).

Q1) Stability of silence. On both encoders, encoded silence stays below the activity of a −40-40 dB frame across lengths and positions (Fig. 10).

Q2) Comparative baselines. 3 caption-based generation baselines under one protocol; RSC keeps CLAP identity 0.78 on Woosh against at most 0.24 (Tab. 1, Fig. 4).

Q3) Sampling schedule. All frames share one clock and reach τ=0\tau = 0 together, with no extra NFE (Eq. 4, Alg. 2).

Q4) UCS ontology. Both reference sets are reported by UCS category, and the selected FSD50K subset will be released.

Q5) Presentation. Narrative citations, and figures and tables placed next to their first reference (the former Fig. 7 is now Fig. 3).

We remain available for any remaining question before the deadline. We strongly encourage you to look at the new version of the paper.

Reviewer_17uPOpenReview ↗

Response to Author Rebuttal

Thank you for the detailed response and substantial additional experiments. The revised manuscript addresses my concerns about the asynchronous sampling procedure and the separation of sampling-interval location and width. The silence-latent stability tests, expanded dataset description, and public FSD50K evaluation also improve the empirical support.

While I understand that subtracting the silent latent addresses the choice of reference point, my concerns about the proposed rationale remain. It involves two distinct links as follows:

  • First, the variation in SAO3 silence latents in Fig. 10 suggests that the score may reflect encoder-dependent contextual effects as well as acoustic activity. Although this variation is small in the tested settings, a fixed silence reference does not guarantee zero activity for all silent regions. The results also do not establish that larger latent distances on non-silent regions indicate greater acoustic activity or intensity.

  • Second, the inverted-mapping ablation generally supports the proposed allocation, but does not establish that higher activity predicts greater tolerance to corruption without identity loss. In the matched-interval comparisons in Fig. 7, the inverted mapping is closer to the identity calibration at one interval and the diversity calibration at two intervals. Moreover, being closer to the calibration value does not necessarily mean better identity preservation. The claims should therefore be qualified.

The revisions have improved my assessment, and I would raise my score to 2.5 (borderline reject) if that option were available. On the current four-point scale, I maintain my score of 2.

Reviewer_S751OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

The paper work on the task of One-shot SFX variation which generating diverse but recognizable alternatives from one single reference audio. They propose RSC: Reference-Structured Corruption. It partition the audio into different activity chunks based on detecting the changing in latent activity between regions. Based on that, it assigns different level of noise to them: high-activity chunk have higher level of initial noise (more diverse), and low-activity chunk have lower level of initial noise (low diverse).

Critical strengths

Avoiding perceptual repetition of game SFX is a real production need. Framing the "where" and "how much" of corruption as something derived from the reference itself is a clear and simple idea

The paper method is intuitive and novel (using latent activity level to divide chunks), and the experiments prove that it is better than random boundary.

The training cost is cheap, and one time only. RSC needs no training.

Critical weaknesses

Some results not indicating better performance

  • In figure 4, woosh even before DF has more diversity (WCIS) compared to “woosh + DF + ours” at equal identity drift
  • figure 5, both diversity and identity preserving of “woosh + ours” is not better than woosh alone. The diversity of SAO3 also better than “SAO3+ours”

Training-free claim

Although RSC is described as training-free, both pretrained backbones require asynchronous adaptation before the proposed corruption strategy can be applied. Please re-clarify the scope of training-free in the introduction.

Presentation problem:

Figure 7 shown up on the method section but is positioned in the last page of appendix.

Table 6 contains the component-level ablation needed to assess the contributions of content-aligned segmentation and activity-conditioned corruption but placed in the appendix.

On table 1, the closest and second closest are bold, but the discussion “impact of RSC” part highlighting the highest WCIS and CLAP,... So which should be focused on, closeness to GT or higher/lower metric?

On figure 5 the diversity of “woosh+ours” is bold even though it is not the best. Probably because it is closest to GT, but it may make the reader distracted.

Questions for authors
  • Please refer to weakness, and also additional questions:
    • What evidence supports assigning stronger corruption to high-activity regions rather than low-activity regions (Eq. 9)? Could the authors compare against, for example, inverse-activity allocation to validate this design choice.
    • How sensitive is the proposed L2 activity measure to the choice of the fixed silence embedding (Eq. 4)? Could the authors compare it with direct latent norms or other distance measures?
    • Each region starts at a different noise level, but Algorithm 2 just says the result is passed to the ODE solver, and Section 3.1 only describes sampling with a single shared noise level. How are different regions jointly go toward zero? Could the authors describe the sampling schedule?
    • The evaluation set, the FAD reference set use internal data; ESC-50 is used only for calibration. Could you report the main comparison (Table 1) on ESC-50 or another public sound collection?
3 comments
AuthorsOpenReview ↗

COMMENT 1/2: Concerns

Thank you for your careful reading and the concrete questions. The reviewer highlighted a clear and simple idea with an intuitive and novel approach, with results showing clear improvement over random boundaries. As well as the low and one-time training cost of the inference-time method. Regarding your concerns, we try to answer all of them below.

A1) RSC performance. Figure 4, diversity at equal identity drift. The submitted Fig. 4 showed the identity–diversity trade-off over several intervals, where Woosh with a global τ\tau is more diverse than our method, even before fine-tuning. This is true: when the intervals of Tables 5 and 6 are matched on identity, a global τ\tau is at least as diverse for the same drift (WCIS 1.321.32 against 1.211.21 at eJSD=0.14e_{JSD}=0.14 on Woosh, likewise on SAO3, while CLAP is mixed). RSC does not push this frontier outward; it changes where an interval lands on it and how stable that point is. On Woosh, at the same interval, it keeps more identity (0.570.57 against 0.460.46 at U(0.80,0.95)\mathcal{U}(0.80,0.95)) and barely moves when the interval widens around a fixed centre (+0.03+0.03 WCIS against +0.34+0.34). Our point was that a variation should not drift towards too much diversity but stay as close as possible to the ground truth, which the submitted figure did not show clearly. The new Fig. 5 compares each corruption variant with RSC directly, at matching flow-time intervals, by α\alpha-precision against the ground-truth distribution: against a global τ\tau, RSC is closer at five of seven intervals on Woosh and six of seven on SAO3.

Figure 5, human study, and choice of metrics. In the human evaluation (submitted Fig. 5, now Fig. 12), other corruption methods score higher on diversity. On Woosh, they also score higher on identity, but given the seven references, 15 listeners, and wide confidence intervals, the study does not provide clear evidence that the methods differ; we reframe this study as a basis for improvement in future work. A good SFX variation has to balance diversity with preservation of identity, and both scores must be read jointly. This is why the paper calibrates where that balance lies: the new Fig. 4 shows the covariance of the production-made variations, which marks the target region. The submitted version drew the trade-off with WCIS, for within-reference diversity, and eJSDe_{JSD}, for identity drift. These metrics are still reported (Tables 5 and 6), but we now draw the trade-off with CLAPLAIONCLAP_{LAION} cosine similarity and cosine distance, whose calibration separates within-identity from between-identity pairs more clearly (Fig. 11).

A2) Training-free statement. We agree that the training-free claim was ambiguous. RSC is an inference-time method with no per-reference training or optimization needed; it runs on a backbone adapted once to accept frame-wise flow times. On Woosh, even that adaptation is optional, since its timestep embedding can be evaluated per frame: we evaluate RSC on the released weights and show that at their respective selected flow-time intervals the results remain close after adaptation; CLAP identity 0.80→0.780.80\rightarrow 0.78, P$%$ 0.93→0.900.93\rightarrow0.90, R$%$ 0.85→0.870.85\rightarrow0.87 and FAD 4.68→4.324.68\rightarrow4.32 (Tab.1). For the sake of clarity we replace training-free to inference-time method. We clarify the scope in the Introduction section.

A3) Presentation and readability. Both items you asked to move are now in the main text: the activity figure is Fig. 3 in Sec. 3.2, next to Eq. 5, and the component ablation is Table 2 and Fig. 5 in Sec. 4.2, with every scheme compared at the same band. We strongly encourage you to take a look at the new version of the paper.

AuthorsOpenReview ↗

COMMENT 2/2: Q&A

Q1) Why stronger corruption to high-activity regions. We ran the inverted allocation: τk∗=(1−λ∣rk∣)τk+λ∣rk∣a~k,a~k=τmin⁡+τmax⁡−ak,\tau_k^{\ast} = \left(1-\lambda |r_k|\right)\tau_k + \lambda |r_k| \tilde a_k, \qquad \tilde a_k = \tau_{\min} + \tau_{\max} - a_k, i.e. a~k=τmin⁡\tilde a_k = \tau_{\min} if rk≥0r_k \ge 0 and τmax⁡\tau_{\max} otherwise (Eq. 11), where most noise goes to the least active regions, at all seven flow-time intervals on 1,5001,500 reference samples. In Fig. 7 (+Tab. 7), we log a heatmap comparing trade-offs in the CLAP-LAION identity–diversity plane, highlighting which setup is closer to our Production SFX calibration (ground truth). (i) The signs matter to preserve the identity of the sound, as rkr_k constantly preserves identity over −rk-r_k at every interval. (ii) On the identity–diversity trade-off, rkr_k is consistently closer to ground truth. (iii) Inverting the map moves each interval towards diversity at the cost of losing the identity. The activity map rkr_k avoids this by construction.

Q2) Stability of the zsilz_{sil} embedding and other distances. Generation is robust to the choice of distance: replacing ∥zt−zsil∥2\lVert z_t - z_{\mathrm{sil}}\rVert_2 by the raw latent norm, ℓ1\ell_1, squared ℓ2\ell_2 or cosine distance changes CLAP identity and diversity by at most 0.012, on 50 internal and 50 FSD50K references (Fig. 8). The latent norm is one variant that misreads silence. As digital silence does not encode to the origin (∥zsil∥2=3.8\lVert z_{\mathrm{sil}}\rVert_2 = 3.8 on Woosh), ∥zt∥2\lVert z_t\rVert_2 gives silent frames 0.41 of the activity of a typical sounding frame instead of 0, and it changes the diversity impact on the internal reference. Sensitivity to the silence reference. RSC is slightly sensitive (at most ±0.015\pm0.015) to the exact silence reference as long as it stays below the silence threshold. When we replace digital silence with the latent of white noise at −50-50, −60-60 or −70-70 dB, CLAP identity and diversity move by at most about 0.0150.015, and by at most 0.010.01 at −70-70 dB, on both backbones and both datasets (Fig. 9). The largest changes in this experiment are generally observed with the −40-40 dB reference. This is the direction Eq. 10 implies: a louder reference gives silent frames a non-zero activity, so quiet regions are pulled less towards τmin\tau_{\text{min}} and receive more noise consequently.

Q3) Sampling schedule (frame-wise). Given the field τ\boldsymbol{\tau} of new Eq. 4, all frames are integrated on one shared clock of SS steps (Heun sampler at 24 steps) as τ(n)(t)=(1−nS)τ(t),n=0,…,S,\tau^{(n)}(t)=\left(1-\frac{n}{S}\right)\tau(t), \qquad n=0,\ldots,S, so each frame follows its own straight path from τ(t)\tau(t) to 0 with step size h(t)=τ(t)Sh(t)=\frac{\tau(t)}{S}, and all frames reach τ=0\tau=0 together at n=Sn=S (Eq. 4, Alg. 2). No region finishes early or waits for the others, and a frame with initial τ(t)=0\tau(t)=0 has h(t)=0h(t)=0 and stays clean. The network receives the full field τ(n)\tau^{(n)} at each step, so the regional field adds no function evaluation: 24 Heun steps with two calls (2$^{\text{nd}}$ order) each give 48 NFE.

Q4) Evaluation on public sound collection. We added a public evaluation on 500500 FSD50K references (CC-BY-NC subset), grouped into UCS categories with the relabeling of Beck & Lerch (2026), where RSC and each corruption variant are evaluated at their best selected [τmin,τmax][\tau_{min},\tau_{max}] interval (Tables 10–11, Fig. 6). Fig. 6, uses RSC (inference-time), shows that its advantage depends on the sound category: structured regional allocation is closer to production than random τk\tau_k on human, weather, birds, footsteps, crowds and doors, and closer than a global τ\tau on human, fire, objects, doors and wood, while other relatively larger and monotone sound categories favour simpler global corruption (e.g. animals). Overall, RSC remains competitive with the independently selected baselines, although the differences in the calibrated CLAP identity–diversity distance are small (within 0.01). This may partly come from the selected intervals: at matching intervals on the internal set, the new Fig. 5 shows that activity-relative corruption matters most at restricted and high corruption levels.

AuthorsOpenReview ↗

Gentle reminder: revised version and replies

Dear Reviewer S751, as the discussion period closes on 18 November, we kindly remind you that our two comments above answer each of your concerns and questions and that the revised version of the paper is uploaded. We suggest starting with the new Fig. 5 (Sec. 4.2), which addresses your main concern on RSC performance.

RSC performance. We agree that a global τ\tau is at least as diverse at equal drift (submitted Fig. 4). The new Fig. 5 compares each corruption variant with RSC at matching flow-time intervals: against a global τ\tau, RSC is closer to the Production SFX ground truth at 5 of 7 intervals on Woosh and 6 of 7 on SAO3. The human study is now reported as a mixed result (A.3).

Training-free statement. We agree that the training-free claim was ambiguous. RSC is an inference-time method with no per-reference training or optimization needed; it runs on a backbone adapted once to accept frame-wise flow times.

Questions and presentation. The inverted allocation is in Fig. 7 (+Tab. 7), the silence reference and distance ablations in Figs. 8–9, the frame-wise sampler in Eq. 4 and Alg. 2, and the FSD50K evaluation in Tables 10–11 and Fig. 6. Training-free is replaced by inference-time method, and the activity figure and component ablation are now in the main text (Fig. 3, Tab. 2).

We remain available for any remaining question before the deadline, and we would be grateful if the revision could be taken into account in your final assessment.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Comment

We thank the reviewers for careful and constructive reviews. Reviewers agree that game and media sound effects need variations to avoid perceptual repetition (S751, 2jv1). They found the core idea; deriving where and how strongly to corrupt from the reference itself, "clear and simple" and "intuitive and novel" (S751), and "a simple and useful idea" (ikrZ). They also note that it needs no per-reference training (S751, 17uP), offers a controllable identity–diversity trade-off (17uP), is "not limited to a single backbone" (ikrZ), and comes with thorough ablations (17uP).

The revision keeps the method, sharpens its claims and adds new or improved figures (Figs. 4–6). Like SDEdit (Meng et al., 2022), RSC starts from a partially noised reference, so more noise still trades identity for variation. What it controls is where the noise goes: each region's flow time follows its activity in the reference, with no per-reference optimisation. This matters most at higher corruption levels, where a single global level loses the most identity, activity-relative corruption keeps more of it while keeping diversity close to production, and holds the operating point stable.

Where activity-conditioned allocation helps (all reviewers). The new Fig. 5 compares every corruption scheme with RSC at the same flow-time interval, by the share of variations that fall inside the production region of the CLAP identity–diversity plane (α\alpha-precision; Alaa et al., 2022). The gain is concentrated at higher corruption levels. At every interval centred at 0.775 or above, and on SAO3 already from 0.7, RSC places more variations inside this region than every regional and global variant (4 of 7 intervals on Woosh and 5 of 7 on SAO3 against random flow times). At the highest interval, U(0.80,0.95)\mathcal{U}(0.80, 0.95), it keeps CLAP identity 0.57 on Woosh, against 0.54 for random flow times and 0.46 for a global flow time (Tab. 5). The operating point is also stable: widening the interval from 0.15 to 0.45 at centre 0.775 moves RSC by +0.03 WCIS and 0.01 CLAP identity, against +0.34 and 0.06 for a global flow time, because Eq. 10 scales the spread of each region's flow time by (1−λ∣rk∣)(1-\lambda|r_k|).

Training-free claim and cost (all reviewers). We now call RSC an inference-time method. It requires no per-reference training or optimization, on a backbone adapted once to accept frame-wise flow times. On Woosh, it also runs on the released weights (Tab. 1), and Stable Audio 3 (SAO3) needs a one-time adapter of 19.3M parameters (1.3%1.3\% of the DiT). The regional field adds no network evaluation. The 48 NFE on SAO3 come from our shared 24-step Heun sampler, and at SAO3's default $\sim$10 NFE, RSC samples 4.7×4.7\times faster with identity and diversity unchanged (Tab. 17).

Public data and other methods (all reviewers). We add 500 FSD50K references (CC-BY-NC) grouped into UCS categories (Beck & Lerch, 2026), evaluated with 5 corruption schemes (Tab. 10-11, Fig. 6), and three caption-based generation baselines under one protocol (Tab. 1, Fig. 4). Reference corruption keeps CLAP identity at 0.78, against at most 0.24 for text-to-audio and audio-text-to-audio generation.

Sampler, silence and activity measure (S751, 17uP). The frame-wise sampler is detailed (Eq. 4, Alg. 2): all frames share one clock and reach τ=0\tau = 0 together. Changing the activity distance moves CLAP identity and diversity by at most 0.012 (Fig. 8), a silence reference below −40-40,dB by at most about 0.015 (Fig. 9), and encoded silence is stable across lengths and positions (Fig. 10).

Human study (S751, 2jv1, ikrZ). With 7 references and 15 listeners, the study cannot reliably show the benefits of RSC over SoTA methods. We report it as a mixed result, along with its protocol and limitations in A.3, and leave a larger study to future work. Presentation (S751, 17uP). The activity figure moves to Sec. 3.2, the component ablation to the main text (Tab. 2, Fig. 5), and each caption states one reading rule.

Protocol updates. Re-running the evaluation led us to harmonize the protocol, which changes some numbers quoted in the reviews. All regional schemes now run on one adapted checkpoint, and every scheme uses the same 24-step Heun sampler (except SDE at 47-step stochastic sampler), so each comparison isolates the corruption scheme. The mean activity Aˉ\bar{A} now excludes the zero padding of short references. The evaluation uses three disjoint seeds of 500 references, with per-reference standard deviations. The production calibration, set against the broader ESC-50 classes (Piczak, 2015), now uses 196 identities and N=10N = 10 variations per reference.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.