acceptodds
Under review as a conference paper at ICLR 2027

PRECOMPUTED DECODER JACOBIANS FOR FEW-STEP AUDIO GUIDANCE

Abstract

Training-free guidance steers a pretrained generator with the gradient of a score on its output. In latent audio generators this score sits behind a decoder, so every guidance step pays for differentiating it, and the common remedy trains a latent head that imitates the decoder's gradient. We observe that for attributes such as loudness the decoder's Jacobian changes little from one latent state to another, and propose Mean-Jacobian guidance (MaJa), which replaces it with a single matrix: the Jacobian averaged over a handful of training states, computed once and reused for every state, prompt and step. Paired with a head that predicts only the attribute value, this takes the decoder out of the sampling loop and makes guidance an order of magnitude cheaper. On few-step samplers the shortcut comes without a loss of accuracy in almost every setting. Although its direction has a cosine of only 0.60 to 0.91 to the decoder gradient, MaJa is level with or ahead of differentiating the decoder in fourteen of sixteen settings across two generators and five attributes, and reduces target error by 31% on average relative to learned latent guidance heads. Closer imitations of the gradient do not help: a head that reproduces it at cosine up to 0.99 controls no better, and the exact gradient through the remaining sampler steps does worse. On these samplers, agreement with the decoder gradient is not what makes a guidance direction effective.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
training-free guidance, distillation, few-step generation, latent audio diffusion, controllable generation, Sobolev training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_D8S3OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Mean Jacobian, MaJa, a technique that speeds up and matches/improves on decoder-gradient guidance in audio models by removing the need to use the decoder for the update. MaJa relies on the average jacobian computed on a calibration set of the decoder’s gradient for a specific attribute, and a small approximation head of the attribute itself. MaJa’s updates are not perfect approximations of the decoder’s updates, achieving lower cosine similarities than another technique shown in the paper but, despite this, managing to match/increase the guidance’s results, while being 14x faster than the use of a decoder.

Critical strengths
  1. MaJa is a very clean and understandable technique, which not only has good performances, but is also presented with an auxiliary test to assess whether it is likely to apply.
  2. The authors try their best to provide adversarial baselines to MaJa, going so far as to train a head that matches the decoder at a cosine of 0.99, nonetheless getting worse results than with MaJa.
  3. The authors are honest about the work, as they do report where MaJa “fails” (it doesn’t surpass the decoder gradient), namely two attributes (new prompts) on ACE-step Turbo.
Critical weaknesses
  1. The paper’s writing is terse, the tables could use a bit of a glow up and overall understanding the results is difficult because so many things are scattered and send a reader to the appendix.
  2. Although extremely explored, the scope of the experiments is a bit narrow, only 5 attributes are explored, across two generators. Furthermore on one of the two generators (ACE) the two losses appear on both the only two attributes tested (on only one of the two types of prompts reported).
Questions for authors
  1. Could the authors provide results on a third few-step audio model? Even with few attributes tested?
  2. In Table 7 AudioX gets lots of advantage in running in the later steps, could you show whether the same happens with ACE? Furthermore, do you have an explanation for why ACE doesn’t see the same results as AudioX in Table 1?
2 comments

Response to Reviewer D8S3

We sincerely thank the reviewer for the careful and detailed reading, and for recognising the screen, the adversarial baselines and the reporting of the ACE results. The questions about ACE were particularly valuable to us: following them up showed where the late-step advantage comes from and how the precomputed matrix should be built. We report these findings below.

W1: presentation. We thank the reviewer for this comment and take it on board. Many results were placed in the appendix so that each anticipated objection, including the adversarial baselines the reviewer kindly highlighted, could be answered in full; we accept that this left too much of the reading in the appendix. Please see point 1 of the general response. In the final version the main table will be redesigned so that the results are no longer scattered across the appendix.

W2 / Q1: a third few-step model, and the two ACE losses. This is a very fair request: a third model is the most direct test of whether the result is general, and we thank the reviewer for suggesting it.

  • Third generator. We added Stable Audio Open Small, whose 8-step sampler comes from adversarial post-training rather than distillation, so it also tests whether the result depends on distillation. All five attributes of Table 1 follow the AudioX pattern (table at the end of this response): on loudness MaJa is ahead of the decoder gradient by +0.38 dB [+0.28, +0.50] (15/16 prompts), and a random matrix of the same shape does nothing on every attribute. The decoder-free form, with the equal-cost value head of Table 46, is level with the decoder gradient as well.
  • The two ACE losses at 48 prompts. Using the pre-registered ±0.2 dB equivalence test of Appendix E.2 on a 48-prompt ACE evaluation set, loudness is equivalent to the decoder gradient (−0.12 dB, 90% CI [−0.20, −0.04]). On brightness, with the 24-state matrix of Table 1, the difference is −0.22 dB [−0.37, −0.02] (Table 1: −0.34 dB); a larger matrix narrows it further (Q2b).

Q2a: does the late-step advantage of Table 7 hold on ACE? We thank the reviewer for this question, which turned out to be very informative. The advantage holds on the two generators that re-noise their clean prediction at every step, and on ACE it depends on the sampler and on the step size. We ran three checks.

  • Stable Audio Open Small (re-noising, ping-pong). The pattern holds: guiding only the last three steps keeps +0.29 of the +0.38 dB advantage, while the first three steps do almost nothing for either direction.
  • ACE with its own re-noising sampler. ACE-Step Turbo samples with a deterministic Euler update, whereas AudioX-Turbo (DMD) and Stable Audio Open Small re-noise at every step. We switched the same ACE model to its built-in re-noising sampler (same eight-step schedule; unguided outputs within the seed-to-seed spread of the Euler outputs, CLAP FAD 0.26 against 0.25). Restricting guidance to the late steps then improves MaJa's standing relative to the decoder gradient on all four evaluation sets (by +0.02 and +0.06 dB on loudness and by +0.36 and +0.66 dB on brightness), in the same direction as on AudioX and Stable Audio Open Small; with Euler it worsened it on three of four.
  • ACE with its native Euler sampler. With a single step size calibrated on the decoder gradient for all seven steps, late-only guidance left MaJa behind. That step size is not suited to a three-step schedule: when each direction chooses its own step size on the development prompts (the protocol of Appendix E.3), MaJa is level with the decoder gradient on all four late-only evaluation sets (−0.08, +0.08, +0.10 and +0.04 dB; every interval includes zero).

The late-step pattern is therefore tied to re-noising samplers, and on ACE's deterministic sampler late-only MaJa matches the decoder gradient once the step size is chosen per method.

Q2b: why does ACE differ from AudioX? For loudness the difference is small: on 48 prompts MaJa is within the ±0.2 dB equivalence bound (above). For brightness, a large part of the gap comes from the size of the averaged matrix, which we tested directly.

The ACE brightness matrix in Table 1 averages 24 states at three guidance times. Built from 48 or 80 states covering more guidance times (the recipe we use for ACE loudness), MaJa's brightness error drops by 0.14 to 0.23 dB on every evaluation set, on 8 of 8 or 10 to 11 of 12 prompts. MaJa then moves ahead of the decoder gradient on unseen classes (+0.22 dB [+0.07, +0.43]; +0.30 dB with late-only guidance), and on new prompts the gap narrows from −0.40 dB with the 24-state matrix to −0.17 dB [−0.33, −0.02] on the same prompts.

(continued)

It is not that the Jacobian varies more along ACE's trajectory: one matrix per guidance time, or one per cluster of states, changes the result by at most 0.03 dB. The final version will report these analyses and recommend building the matrix from at least 48 states.

Stable Audio Open Small, per attribute. MaJa error minus decoder-gradient error is shown with the sign flipped, so positive means MaJa is better (dB, 95% CI):

AttributeNew promptsUnseen instruments
Loudness gain+0.38 [+0.28, +0.50]+0.23 [−0.03, +0.47]
Brightness shelf+0.36 [−0.12, +1.15]+0.04 [−0.57, +0.57]
High-band gain+0.12 [−0.12, +0.36]+0.19 [+0.07, +0.31]
Spectral centroid shift+0.08 [−0.07, +0.22]−0.19 [−0.37, −0.02]
Onset shape−0.03 [−0.08, +0.02]−0.06 [−0.20, +0.09]

A random matrix of the same shape changes the error by at most 0.05 dB on every row.

Reviewer_8WzdOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

In this work, the authors propose a method called MaJa to apply loudness/EQ control to few-step audio generators using guidance during sampling from a precomputed decoder Jacobian. In traditional guidance during sampling, backpropagation through a VAE decoder is typically necessary and costly. The proposed method precomputes a fixed average Jacobian per model and control task, then applies that control without backpropagating through the VAE decoder. Evaluation is performed on a variety of loudness and EQ settings across two open-weight models, showing comparable results.

Critical strengths
  1. Novel and relatively straightforward method. The result and method are novel to my knowledge and relatively straightforward to understand. Avoiding backpropagation through the VAE decoder addresses a costly part of guidance during sampling. The related work and references are great.

  2. Useful experiments and transparency. The experiments are useful and show that the method works for volume and EQ-type controls across two open-weight models. The extensive experiments and transparency are appreciated and support the demonstrated low-level control applications.

Critical weaknesses
  1. Scope of the demonstrated controls. The main question is whether the proposed work generalizes beyond low-level loudness and EQ controls. The experiments are useful, but their control-task scope is relatively narrow compared with alternatives such as DITTO and LatCH, which cover more tasks. The main question in my mind is whether this method can generalize to other controls beyond loudness and EQ that are approximately localizable and linear through audio VAE decoders. The CLAP results seem to support this concern. This limits what I can conclude about the broader applicability of the method.

  2. Helping readers understand the main insight. The appendices and experiments are extensive and not sufficiently trimmed down, so much of the effort to distill the results and scientific conclusions is left to the reader. Reducing the work to its most important aspects is a critical part of paper writing and should be applied here to reduce the 75 pages.

The extensive experiments and transparency are appreciated, but there is so much detail that it risks overwhelming readers. More time should be spent reducing the length and helping readers understand the main findings and reproducible insight. Sentence structure is good overall, although terms such as operator, panel, direction source, and survival make it harder to read than necessary.

Questions for authors
  1. Clarification: scope and generalization. What do the current results establish about generalization beyond low-level loudness and EQ controls, and how should the CLAP results affect the claimed scope of the method?

  2. Clarification: practical speed versus quality (minor). What is the main end-to-end speedup? Can you provide a more succinct presentation of the quality-versus-control-versus-speed trade-off? There is also no complete pipeline timing.

4 comments

Response to Reviewer 8Wzd

We sincerely thank the reviewer for the thoughtful review and for the positive assessment of the method, the related work and the transparency of the experiments. The questions were precise and very useful to us: the question on scope led us to test the screen prospectively, and the question on end-to-end speed led us to time the full pipeline. Both are reported below.

W1 / Q1: generalisation beyond loudness and EQ, and the role of the CLAP result. This is the central question about the method, and we are grateful that the reviewer raised it so concretely. We answer it with new experiments and with an explicit scope statement.

  1. Beyond loudness and EQ. The original five attributes already include onset strength (a transient attribute, controlled by reshaping transients) and spectral centroid. In addition, we tested rhythm transfer, the beats task of LatCH: the target is another clip's onset envelope. MaJa is ahead of the decoder gradient on 3 of 4 evaluation sets (+0.19, +0.19, +0.11 dB) and level on the fourth.
  2. A prospective test of the screen. On AudioX and on a third generator (Stable Audio Open Small), we chose new attributes not used anywhere in the paper (low-band loudness, spectral flatness, stereo width and a mel-cepstral ripple coefficient, and on the other side harmonicity, spectral crest factor and spectral rolloff), computed their screen values, and wrote down our predictions, with a hash and a timestamp, before running any guidance on them. On every attribute that screened above about 0.4, MaJa was level with or ahead of the decoder gradient (18 of 18 predictions fixed in advance). Below that value the screen is not decisive: MaJa fell clearly behind on crest factor, as predicted, but matched or beat the decoder gradient on spectral rolloff. The screen is therefore a sufficient condition for applying MaJa: it never certified an attribute on which MaJa then failed.
  3. How the CLAP result affects the scope. The CLAP attribute is the low end of the screen axis (held-out cosine 0.02, against 0.43 to 0.92 for the five attributes of Table 1), and the screen identifies it before guidance is run. We therefore state the scope as: MaJa applies to attributes whose screen value is high on the given generator; for an attribute that screens low, the decoder gradient should be compared on a few development prompts first. The abstract and Section 4.3 will be revised to say exactly this.

W2: length and terminology. We appreciate this comment and accept it. The short names were introduced because each object recurs across many tables (the precomputed matrix, a fixed set of prompts and seeds, the fraction of a step that reaches the output), and they kept those tables compact; we see now that for a reader new to the paper they add a layer of translation. In the final version, the main text will carry the method, the main results and the controls that decide the claim; the appendix will open with an index organised by question; and operator, panel, direction source and survival will be replaced by plain descriptions (general response, point 1).

Q2: end-to-end speed and the quality, control and speed trade-off. We thank the reviewer for asking for this: the full-pipeline time is the number a practitioner needs, and it is more informative than the per-step cost we reported. We measured the full pipeline (conditioning, few-step sampling with guidance, final decode), batch 1, median of repeated runs:

GeneratorUnguidedDecoder gradientDecoder-free MaJaEnd-to-end speed-upGuidance speed-upPeak memory (decoder / MaJa)
AudioX, 10 s1.00 s2.48 s1.05 s2.4×47×21.2 / 13.5 GB
ACE-Step, 20 s0.60 s3.92 s0.69 s5.7×52×26.9 / 8.7 GB
Stable Audio Open Small, 10 s0.50 s4.05 s0.60 s6.8×48×13.4 / 5.0 GB

These timings are for the decoder-free form. Its accuracy is reported in Table 46 (AudioX loudness: 1.71 dB with the equal-cost value head, against 2.03 dB for the decoder gradient); on Stable Audio Open Small it is level with the decoder gradient on loudness (+0.12 and +0.13 dB on new prompts and unseen instruments). It is least accurate where the value head generalises least, on unseen instruments.

Long clips and larger cards. The decoder gradient's memory grows with clip length, so we also timed ACE-Step on an 80 GB A800. The full decoder backward needs 26.9, 47.4 and 67.9 GiB at 20, 40 and 60 s, about 1 GiB per second of audio: it fits up to 70 s and runs out of memory from 80 s, so a larger card moves the limit but does not remove it. Decoder-free MaJa uses exactly the unguided generator's memory and adds 0.06 to 0.08 s of guidance at every length.

(continued)

ACE-StepUnguidedDecoder gradient, full backwardDecoder gradient, tiled exactDecoder-free MaJaEnd-to-end speed-up (vs full / vs tiled)
20 s0.51 s1.95 s (26.9 GiB)3.55 s0.54 s3.6× / 6.6×
40 s0.56 s3.61 s (47.4 GiB)6.83 s0.64 s5.7× / 10.7×
60 s0.64 s6.13 s (67.9 GiB)10.11 s0.72 s8.5× / 14.0×
120 s1.07 sout of memory19.84 s1.15 snot applicable / 17.3×

On the guidance itself the speed-up is 24×, 49× and 82× against the full backward at 20, 40 and 60 s, and 239× against the tiled gradient at 120 s. The decoder paths run about twice as fast on the A800 as on the A6000, so these speed-ups are, if anything, smaller than on the paper's hardware.

On the quality side, guidance changes the audio little with either direction: on Stable Audio Open Small, MaJa's FAD to the unguided clips stays within 0.02 of the decoder gradient's on each attribute, and waveform correlation is 0.99 for both on loudness. The final version will add one compact table per generator that puts target error, these quality guards and cost side by side.

Thanks for the thorough response. All the tested attributes are still low-level signal descriptors, though, so please say in the limitations that melodic and semantic controls aren't covered.

The end-to-end timing and the long-clip result (the decoder gradient runs out of memory from about 80 s, while MaJa adds under 0.1 s at any length) are the practical numbers I was after. I'd put both in the main text.

I'm keeping my positive score.

Thank you for the follow-up and both suggestions are well taken. We will make the scope explicit in the limitations: the attributes we test are frame-level signal attributes, with rhythm transfer, which changes when notes occur, as the furthest step beyond static descriptors. Melodic and semantic controls sit at the low end of the screen, as the CLAP attribute of Appendix E.5 shows, and for them the paper recommends the decoder gradient. We will also move the end-to-end timing and the long-clip result into the main text (Section 6.7).

We are sincerely grateful for the time and care you have given our paper throughout the review. Your comments have improved it a great deal.

Reviewer_1C8ROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes MaJa, a cheap way to do training-free guidance in latent audio generators. Paired with a small head that predicts the attribute value, the decoder is no longer needed during sampling, which saves a lot of time and memory.

Critical strengths
  1. Sufficient detail makes the experimental data very comprehensive.
Critical weaknesses
  1. I strongly suggest the author restructure the paper, ensuring the main body clearly explains the methods and key results, rather than placing them in the appendix.

  2. All five attributes in the paper are simple DSP quantities, and every target is itself a DSP operation on the unguided clip. The paper notes that signal processing reaches every target exactly (zero error), while MaJa gets about 1.6 dB. So the guidance mostly reproduces a post-processing step, less accurately than the post-processing itself.

Questions for authors

see Weaknesses.

5 comments

Response to Reviewer 1C8R

We thank the reviewer for the review and for paying attention that the experimental data are comprehensive.

W2: post-processing reaches every target.

All five attributes in the paper are simple DSP quantities, and every target is itself a DSP operation on the unguided clip. The paper notes that signal processing reaches every target exactly (zero error), while MaJa gets about 1.6 dB. So the guidance mostly reproduces a post-processing step, less accurately than the post-processing itself.

It is true that each of the five targets has an exact signal-processing solution. This is by design, and it does not make guidance redundant: the targets are how we measure guidance, and what guidance adds beyond post-processing shows on targets that post-processing cannot reach cleanly. We take the two parts in turn.

Why the targets are signal-processing operations. Each target is defined as a signal-processing operation on the unguided clip so that the exact answer is known and the error can be measured precisely (Sec. 5.2); the zero-error row is this measurement floor, not a competing method. The question the paper asks is which direction a guidance step should follow, and on the same targets the standard alternative, the decoder gradient used by end-to-end methods, reaches 2.03 dB, which MaJa matches or beats at a fraction of the cost.

This is the established task design in audio control. LatCH [1] guides Stable Audio Open on intensity (smoothed RMS in dB, scored by MSE), beats and pitch, and is motivated, as MaJa is, by the cost of backpropagating through the decoder; DITTO [2] controls intensity the same way, together with melody and musical structure; Music ControlNet [3] and MuseControlLite [4] condition on melody, rhythm and dynamics extracted from audio; Sketch2Sound [5] controls loudness, brightness (spectral centroid) and pitch, scored by the L1 error to the input control signals; and SoundCTM [6] demonstrates training-free intensity control inside a few-step model. Two of our attributes, loudness and spectral centroid, are exactly the loudness and brightness controls of Sketch2Sound [5], and loudness is the intensity task of DITTO [2] and LatCH [1]. In all of these works post-processing could in principle move the attribute to the target; control during generation is studied because the attribute is then realised by the generator together with the rest of the audio. Guided inverse problems synthesise their measurements with a known operator for the same reason, so that the error against the ground truth is exact [7]. Within this setting, MaJa addresses the cost of the guidance step, the same bottleneck that LatCH [1] targets.

What guidance adds beyond post-processing. On rhythm transfer, the beats task of LatCH [1], the target is another clip's onset envelope, which changes when notes occur rather than how loud or bright they are. Here post-processing pays a large price: onset-aligned time stretching moves the onsets further toward the target, but only by rewriting the timing of the audio, with a FAD of 0.44 to 0.85 to the unguided clips (guidance: 0.03 to 0.10) and a predicted production quality (Audiobox Aesthetics) 0.7 to 1.6 points lower. On the same target MaJa is ahead of the decoder gradient on 3 of 4 evaluation sets (+0.19, +0.19, +0.11 dB) and level on the fourth. MaJa needs only a differentiable attribute, so nothing in the method changes when the target is not a post-processing edit; on such targets guidance makes the generator produce the target rather than distorting what it produced, and MaJa makes this guidance nearly free.

W1: restructure the paper. We thank the reviewer for this suggestion and accept it. The current main text does state the method (Section 4) and the main result (Table 1), with the full set of controls in the appendix, but we accept that too much of the supporting evidence sits there. In the final version, more of the main results will be in the main text, and the appendix will be organised around the questions each experiment answers. Please see point 1 of the general response for the full list of changes.

(continued)

Soundness rating. The review rates soundness 1 without naming a specific issue, so we briefly list how the claims are protected, and we would be glad to address any specific concern the reviewer has. Every comparison is paired on the same prompts and seeds, with 95% cluster-bootstrap intervals over instrument clusters. Results are reported on held-out prompts and on instrument classes unseen when the matrix was built. A random matrix of the same shape serves as a negative control and changes nothing. Step sizes are chosen on development prompts that are not used for evaluation, by the decoder gradient alone, which if anything favours the baseline. The 48-prompt comparisons use a pre-registered ±0.2 dB equivalence test (Appendix E.2). For this rebuttal, we recorded our predictions for the new attributes before running any guidance on them, and we repeated the main experiment on a third generator.

We hope this clarifies the role of the signal-processing targets and the value of guidance beyond them.

References

[1] Z. Novack, Z. Zukowski, C. Carr, J. Parker, Z. Evans, J. Taylor, T. Berg-Kirkpatrick, J. McAuley and J. Pons. Low-Resource Guidance for Controllable Latent Audio Diffusion. ICASSP 2026.

[2] Z. Novack, J. McAuley, T. Berg-Kirkpatrick and N. J. Bryan. DITTO: Diffusion Inference-Time T-Optimization for Music Generation. ICML 2024.

[3] S.-L. Wu, C. Donahue, S. Watanabe and N. J. Bryan. Music ControlNet: Multiple Time-Varying Controls for Music Generation. IEEE/ACM TASLP, 2024.

[4] F.-D. Tsai, S.-L. Wu, W. Lee, S.-P. Yang, B.-R. Chen, H.-C. Cheng and Y.-H. Yang. MuseControlLite: Multifunctional Music Generation with Lightweight Conditioners. ICML 2025.

[5] H. Flores García, O. Nieto, J. Salamon, B. Pardo and P. Seetharaman. Sketch2Sound: Controllable Audio Generation via Time-Varying Signals and Sonic Imitations. arXiv:2412.08550, 2024.

[6] K. Saito, D. Kim, T. Shibuya, C.-H. Lai, Z. Zhong, Y. Takida and Y. Mitsufuji. SoundCTM: Unifying Score-Based and Consistency Models for Full-Band Text-to-Sound Generation. ICLR 2025.

[7] H. Chung, J. Kim, M. T. McCann, M. L. Klasky and J. C. Ye. Diffusion Posterior Sampling for General Noisy Inverse Problems. ICLR 2023.

Dear Reviewer, thank you again for your time and comments. We would appreciate your feedback on our rebuttal, particularly whether the additional experiments and clarifications address your concerns. If any points remain unresolved, we would be happy to discuss them further.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank the three reviewers for their time and for their careful and constructive reviews. Two points recur across the reviews: the paper is hard to read, and the scope of the method should be stated and tested more clearly. Both comments were very helpful to us and we have acted on them. We summarise the changes here; each reviewer's specific questions are answered in the individual responses.

1. Presentation (R1, R2, R3). We are grateful to all three reviewers for pointing this out, and we are restructuring the paper accordingly for the final version. We would also like to explain briefly how the current version was organised, because this decides what we keep. The main result runs against first-order reasoning: a direction with cosine 0.6 to the decoder gradient guides as well as the gradient itself. We therefore built the main text around this one claim and its decisive evidence (Table 1 and the survival analysis of Figure 6), and gave every objection we could anticipate its own appendix: heads that imitate the gradient at cosine up to 0.99, the exact gradient through the remaining sampler steps, pre-registered equivalence tests, and the cases where the method fails. The transparency noted by R2 and the adversarial baselines and failure reporting noted by R3 come from this design. The reviews made clear to us that, in doing so, we let the evidence crowd out the main story. We therefore keep all of the evidence and change its organisation; concretely, the final version will make the following changes:

  • Main text. The main text carries the method, the main results and only the controls that decide the claim (the random matrix and the equivalence test). One compact table per generator replaces Table 1 and puts target error, the quality guards (FAD, waveform correlation) and cost side by side, so that no main result has to be looked up in the appendix.
  • Scope. The abstract and Section 4.3 state the scope given in point 3 below.
  • Appendix. The appendix opens with an index organised by the question each experiment answers, and adds the new experiments of this rebuttal (third generator, rhythm transfer, prospective screen test, full-pipeline timing, ACE sampler and matrix-size analyses).
  • Terminology. Plain descriptions replace our internal terms: operator → precomputed matrix, panel → evaluation set, direction source → guidance direction, survival → fraction of a guidance step that reaches the output.

2. New experiments.

  • A third few-step generator (R3, R2). Stable Audio Open Small, whose sampler comes from adversarial post-training rather than distillation. All five attributes of Table 1 follow the AudioX pattern: on loudness MaJa is ahead of the decoder gradient by +0.38 dB [+0.28, +0.50] on 15 of 16 prompts, a random matrix does nothing, and the late-step advantage also replicates. The per-attribute table is in the response to R3.
  • Rhythm transfer (R1, R2). The target is another clip's onset envelope, as in the beats task of LatCH. MaJa is ahead of the decoder gradient on 3 of 4 evaluation sets (+0.19, +0.19, +0.11 dB) and level on the fourth. Time-stretching post-processing transfers more onsets, but at a FAD of 0.44 to 0.85, against 0.03 to 0.10 for guidance.
  • A prospective test of the screen (R2). New attributes (low-band loudness, spectral flatness, stereo width, MFCC), with the predictions recorded and hashed before any guidance was run. On every attribute the screen rated high, MaJa was level with or ahead of the decoder gradient (18 of 18).
  • Full-pipeline timing (R2). On AudioX, 1.05 s against 2.48 s for decoder guidance (2.4× end to end, 47× on guidance), with 13.5 against 21.2 GB peak memory. On ACE, 5.7× at 20 s; the decoder gradient needs about 1 GiB of memory per second of audio, so even an 80 GB card runs out from 80 s, while MaJa adds no memory and under 0.1 s of guidance at any length (on 80 GB: 3.6× to 8.5× faster at 20 to 60 s; 17× at 120 s against the tiled exact gradient). On Stable Audio Open Small, 6.8×.
  • ACE analyses (R3). The sampler (Euler against re-noising), per-method step sizes, the size of the averaged matrix, and a 48-prompt evaluation set; see the response to R3.

3. Scope. We state the scope of MaJa explicitly: it applies to attributes whose screen value is high on the given generator, and the screen is computed before any guidance is run. The CLAP attribute in Appendix E.5 marks the low end of this axis, where the paper already recommends the decoder gradient. We revise the abstract and Section 4.3 accordingly.

We thank the reviewers again for their time and for comments that have made the paper clearer and stronger, and we would be glad to answer any further questions during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.