acceptodds
Withdrawn from ICLR 2027

Measure Before You Operate: Layer-Wise Gradient Surgery in Audio-Video Diffusion RL Is Necessary on Some Backbones and Inert on Others

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

OmniNFT, the current recipe for RL post-training of joint audio–video diffusion models, protects the audio tower from video-loss gradients with a per-block scale (“layer-wise gradient surgery”) whose constants were tuned on one backbone, LTX-2, and published as block indices. We ask three questions: where the relevant blocks are on other backbones, whether the surgery is needed at all, and whether its exact placement matters. Two probes that take hours rather than training runs locate the synchronization-critical blocks and measure per-block gradient leakage. On five architectures the zones sit at different relative depths, and no index-copying or rescaling heuristic recovers them; on LTX-2 itself our probes do not recover the published window. Within a model line the zones do not move under SFT, weight souping or RL. Sixteen controlled RL arms on three backbones then give different answers to the other two questions. Whether surgery is needed depends on the backbone: on a 29B model, fifty epochs of RL without it leave word error rate 39% above the SFT start while the training reward keeps rising, and the calibrated schedule from the same start lowers WER by 25% (); on a 3B model and on the public Ovi, none of the schedules, including no surgery, can be told apart. Where the surgery is placed cannot be resolved at this budget on any backbone: differences between schedules are within noise and change sign between prompt sets. A first-order argument shows that the surgery acts through the product of leakage and gradient conflict, which explains the inert cases but not the 3B one. The recipe also contains the first speech-intelligibility reward for joint AV RL, which lowers WER by 28% on public Harvard sentences and 48% on the VABench speech subset, with smaller gains under independent ASR judges. Probe code and the public evaluation protocol will be released.

Primary area:
generative models
Keywords:
Modality-Wise Diffusion RL, Audio-Video Generation, Cross-Modal Attention, Audio-Video Diffusion Transformer, Reward Routing

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_nYD2OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether layer-wise gradient surgery in OmniNFT transfers across joint audio-video diffusion models. The authors introduce two probes to identify synchronization-sensitive blocks and measure gradient leakage and conflict. They evaluate five architectures and sixteen RL configurations across three backbones. The results suggest that sensitive regions differ across architectures but remain stable within a model line. Surgery substantially improves speech intelligibility on the 29B model, while no significant differences between schedules are detected on the 3B model or Ovi. The paper also presents a first-order analysis of gradient interference and an ASR-based speech reward.

Critical strengths
  1. The 29B clean-start experiment provides a substantial result within the tested runs. From the same SFT baseline (WER 0.235), the no-surgery configuration reaches 0.326, while the measured schedule improves to 0.175. Independent ASR judges support the direction of the result. The reward-selected no-surgery checkpoint also performs approximately 35% worse than the baseline, highlighting a practical limitation of reward-based checkpoint selection.

  2. The probes identify architecture-dependent synchronization regions that remain relatively stable within a model line. The authors acknowledge that measured surgery placement does not consistently outperform alternative schedules and that non-significant differences do not establish equivalence. These findings provide useful empirical insights and help clarify the limitations of transferring training interventions across architectures.

Critical weaknesses
  1. OmniNFT's Equations (8) and (9) define its coefficient as a detach fraction, where 0.1 retains 90% of the backward gradient. This paper instead uses 0.1 as the retained-gradient multiplier and attributes an additional coefficient of 0.3 and blocks 40–47 to the published schedule, although these settings are not specified in the original formulation. This raises concerns about whether the LEGACY and INDEX-COPY baselines faithfully represent OmniNFT. The claimed LTX-2 synchronization-region mismatch also requires direct comparison with OmniNFT's Figure 2(d), which reports an A2V ablation. Without these clarifications, the conclusions about transferring published surgery settings remain uncertain.

  2. Most surgery comparisons use one training seed per configuration, so prompt-level statistical significance does not establish reproducibility across independent training runs. The 3B model also has the highest reported leakage ratio (0.25) without a detectable surgery benefit, indicating that leakage alone is insufficient to predict necessity. Gradient conflict, which is central to Proposition 1, was measured only on Ovi, leaving the different outcomes on the 29B and 3B models insufficiently explained. The main positive result also relies on a non-public 29B backbone, limiting independent validation.

Questions for authors
  1. How were OmniNFT's detach-fraction convention and published schedule translated into the retained-gradient coefficients of 0.1 and 0.3 and the attributed blocks 40–47?

  2. What directly comparable evidence supports the claimed LTX-2 synchronization-region mismatch, particularly against OmniNFT's Figure 2(d), which also reports an A2V ablation?

  3. Given the single training seed per configuration and the absence of a detected surgery benefit on the high-leakage 3B model, what evidence supports the reproducibility and predictive validity of the proposed diagnostic rule?

  4. How do the authors reconcile the introduction's claim of a 39–46% WER increase at every evaluated checkpoint with the 17% increase at epoch 30 (0.275 versus 0.235) and the 35% increase at the reward-selected epoch 41 (0.317 versus 0.235)?

Reviewer_hS72OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper revisits a particular RL recipe, OmniNFT, used for post-training audio-video diffusion models. Authors focus on the layer-wise gradient surgery used in this recipe and investigate its generality across backbones, and whether it is needed at all. They conclude that the surgery is only needed for some backbones and they have inconclusive results on whether the "placement" of the surgery in terms of layers matters at all.

Critical strengths

None

Critical weaknesses

Disregarding the fact that the whole paper has been not only written, but likely also executed by AI (Claude Opus 4.x):

  • The scope of the paper is too narrow for a general ML conference, such as ICLR. It focuses on analyzing a specific component of an RL recipe used for audio-video diffusion models.
  • The writing is poor and difficult to follow. It does not convey any high level ideas and reads more like a series of ablations. The paragraphs are difficult to parse as they contain overwhelming amount of numerical detail.
  • The contribution is unclear: what did we learn from this work, or what are the practical outcomes? Did we obtain a better overall recipe?
Questions for authors

Please help the ML community better allocate reviewer time by reconsidering the submission of manuscripts like this.

Reviewer_ZwztOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper examines OmniNFT, a proposed recipe for improving RL post-training of joint audio-video diffusion models that contains three key features: (1) modality-wise advantage routing, (2) layer-wise gradient surgery, and (3) region-wise loss reweighting. The paper attempts to experimentally answer and verify three questions. Because the paper is very difficult to read, with little background setup, unexplained references, and jargon, I have rephrased the three key questions and how they are supported by the paper as follows: (1) Which layers need protection through gradient surgery? OmniNFT uses "gradient surgery" to address the problem where video loss can leak into the audio generator, causing the audio output to degrade. The first question, then, is which layers need protection through gradient surgery. The paper tests five pretrained models. During rollouts, the authors attach a forward hook that zeroes out one cross-modal attention direction and then generate outputs for a set of prompts to measure changes in audio-video desynchronization. The block ranges where muting causes substantial desynchronization are considered the layers of interest. As an answer to the first question, the paper lists the layer depths that cause the most desynchronization in each model. (2) Is gradient surgery required? This is assessed using a leak ratio, which measures how strongly video supervision influences audio parameters at each block. A higher leak ratio indicates heavier video-induced interference, suggesting that surgery is needed. Another metric, called "conflict cosine," measures the cosine distance between audio and video gradients at each block. The paper finds that whether surgery is needed depends on the backbone (e.g., the 29B model requires it, while the 3B model does not). The WER observations align with this interpretation. (3) Does the placement of surgery matter? The paper finds the answer to be inconclusive, as the WER differences are small. No strong evidence indicates that placement is critical. The authors' limited budget prevents them from conducting larger-scale experiments that might yield statistically significant conclusions.

Critical strengths

This is a technically focused paper that investigates analytical questions about a previously proposed RL method. The methodologies used are interesting. The authors are aware of the limitations of this paper and clearly state them.

Critical weaknesses

The most critical issue is the writing. The paper is very difficult to read. It reads as if it were written to communicate findings to the authors of OmniNFT, with no background setup, little context to motivate the topic, and no explanation of the jargon. Given that OmniNFT has only six citations at the time of this review, it can hardly be considered a widely known paper. Thus, this paper needs to provide the necessary background for readers, such as what OmniNFT does, what problem it solves, and why the questions raised in this paper are important. The technical content also reads like a generated report rather than an academic paper.

The paper also feels incomplete in several aspects:

  1. The authors frequently mention compute budget limitations throughout the paper, which feels out of place in an academic writing context. The third question remains inconclusive due to budget constraints, further making the paper feel incomplete.
  2. Conflict cosine is not measured for the 29B model, only for Ovi.
  3. The speech evaluation is limited to a single speaker and short-form speech. The authors acknowledge points (2) and (3) in the limitations section, but these issues nonetheless weaken the paper's conclusions.
Questions for authors

I have written down my detailed understanding of the paper in the Summary section. If there are any gaps between my understanding and the authors' intentions, please clarify. Are there any technical reasons, beyond compute budget constraints, why some results are incomplete (see points 1–3 in the Weaknesses section)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.