- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether layer-wise gradient surgery in OmniNFT transfers across joint audio-video diffusion models. The authors introduce two probes to identify synchronization-sensitive blocks and measure gradient leakage and conflict. They evaluate five architectures and sixteen RL configurations across three backbones. The results suggest that sensitive regions differ across architectures but remain stable within a model line. Surgery substantially improves speech intelligibility on the 29B model, while no significant differences between schedules are detected on the 3B model or Ovi. The paper also presents a first-order analysis of gradient interference and an ASR-based speech reward.
Critical strengths
-
The 29B clean-start experiment provides a substantial result within the tested runs. From the same SFT baseline (WER 0.235), the no-surgery configuration reaches 0.326, while the measured schedule improves to 0.175. Independent ASR judges support the direction of the result. The reward-selected no-surgery checkpoint also performs approximately 35% worse than the baseline, highlighting a practical limitation of reward-based checkpoint selection.
-
The probes identify architecture-dependent synchronization regions that remain relatively stable within a model line. The authors acknowledge that measured surgery placement does not consistently outperform alternative schedules and that non-significant differences do not establish equivalence. These findings provide useful empirical insights and help clarify the limitations of transferring training interventions across architectures.
Critical weaknesses
-
OmniNFT's Equations (8) and (9) define its coefficient as a detach fraction, where 0.1 retains 90% of the backward gradient. This paper instead uses 0.1 as the retained-gradient multiplier and attributes an additional coefficient of 0.3 and blocks 40–47 to the published schedule, although these settings are not specified in the original formulation. This raises concerns about whether the LEGACY and INDEX-COPY baselines faithfully represent OmniNFT. The claimed LTX-2 synchronization-region mismatch also requires direct comparison with OmniNFT's Figure 2(d), which reports an A2V ablation. Without these clarifications, the conclusions about transferring published surgery settings remain uncertain.
-
Most surgery comparisons use one training seed per configuration, so prompt-level statistical significance does not establish reproducibility across independent training runs. The 3B model also has the highest reported leakage ratio (0.25) without a detectable surgery benefit, indicating that leakage alone is insufficient to predict necessity. Gradient conflict, which is central to Proposition 1, was measured only on Ovi, leaving the different outcomes on the 29B and 3B models insufficiently explained. The main positive result also relies on a non-public 29B backbone, limiting independent validation.
Questions for authors
-
How were OmniNFT's detach-fraction convention and published schedule translated into the retained-gradient coefficients of 0.1 and 0.3 and the attributed blocks 40–47?
-
What directly comparable evidence supports the claimed LTX-2 synchronization-region mismatch, particularly against OmniNFT's Figure 2(d), which also reports an A2V ablation?
-
Given the single training seed per configuration and the absence of a detected surgery benefit on the high-leakage 3B model, what evidence supports the reproducibility and predictive validity of the proposed diagnostic rule?
-
How do the authors reconcile the introduction's claim of a 39–46% WER increase at every evaluated checkpoint with the 17% increase at epoch 30 (0.275 versus 0.235) and the 35% increase at the reward-selected epoch 41 (0.317 versus 0.235)?