- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies RL fine-tuning for joint audio-video generation. The authors identify three obstacles. Video and audio rewards often disagree for the same sample. Video gradients leak into shallow audio layers through cross-attention. A uniform loss also ignores sound-emitting regions. Building on DiffusionNFT, the proposed OmniNFT addresses these issues with three components. Modality-wise advantage routing computes one advantage per reward and sends each to the branch it evaluates, while the synchronization advantage goes to both branches. Layer-wise gradient surgery scales down the video-to-audio gradients in the shallow A2V cross-attention blocks. Region-wise loss reweighting uses V2A attention maps to up-weight sound-related video tokens. Experiments on LTX-2 and MOVA show gains over the base model and GDPO on JavisBench and VBench. Lip-sync, ASR, and human preference evaluations further support these results.
Critical strengths
-
Modality-wise advantage routing is the main contribution: Sending each advantage to the branch that the corresponding reward evaluates is well suited to RL for joint audio-video generation. Table 3 isolates this component, where DeSync improves from 0.412 to 0.322. This is the main reason for my positive rating.
-
The human study also prefers OmniNFT over GDPO: This agrees with the automatic metrics and provides a second, independent form of evidence for the reported improvement.
Critical weaknesses
Major weaknesses
W1. The synchronization gain from region-wise reweighting is not explained: This component reweights the video loss only, yet in Table 3 it improves DeSync from 0.334 to 0.269. The paper does not explain how a change confined to the video loss produces a gain in temporal alignment. The mechanism behind a reported synchronization gain is therefore left unaccounted for.
W2. The video evaluation is not comprehensive: Only three VBench dimensions are reported, namely aesthetic quality, imaging quality, and subject consistency. The remaining dimensions are not shown, so the video evaluation is incomplete.
Minor weaknesses
W3. The ablation is not monotonic and the trade-offs are not discussed: In Table 3, gradient surgery slightly worsens DeSync from 0.322 to 0.334, and region-wise reweighting lowers AQ from 5.917 to 5.715. The paper presents the three components as complementary without addressing either regression.
W4. The main baseline is not clearly described: The LTX-2 + GDPO row in Table 1 and the LTX-2 + vanilla RL row in Table 3 report the same numbers. The paper should state which advantage formulation and which reward set this baseline uses. Without that, the size of the reported improvement over the baseline cannot be interpreted.
W5. Related work on multi-reward RL for visual generation is missing: Work on multi-reward RL for image generation and video generation is not discussed. Relevant examples include:
- He et al., Learning profitable NFT image diffusions via multiple visual-policy guided reinforcement learning, ACMMM 2023.
- Lee et al., Parrot: Pareto-optimal Multi-Reward Reinforcement Learning Framework for Text-to-Image Generation, ECCV 2024.
- Li et al., T2V-Turbo: Breaking the Quality Bottleneck of Video Consistency Model with Mixed Reward Feedback, NeurIPS 2024.
- Lee et al., Calibrated Multi-Preference Optimization for Aligning Diffusion Models, CVPR 2025.
- Liu et al., VideoDPO: Omni-Preference Alignment for Video Diffusion Generation, CVPR 2025.
Questions for authors
See Weaknesses above
7 comments
Response to reviewer w3e9 - part 2
W3: The ablation is not monotonic and the trade-offs are not discussed: In Table 3, gradient surgery slightly worsens DeSync from 0.322 to 0.334, and region-wise reweighting lowers AQ from 5.917 to 5.715. The paper presents the three components as complementary without addressing either regression.
To examine the independent contribution of each component in the full framework, we conduct controlled ablations by removing modality-wise advantage routing (−routing), layer-wise gradient surgery (−surgery), and region-wise loss reweighting (−region) one at a time. The table below summarizes the results of OmniNFT and its ablated configurations across multiple random seeds (reported as ).
| Configuration | Routing | Surgery | Region | VQ ↑ | AQ ↑ | JavisScore ↑ | DeSync ↓ |
|---|---|---|---|---|---|---|---|
| OmniNFT | ✓ | ✓ | ✓ | 3.332 ± 0.096 | 5.715 ± 0.070 | 0.220 ± 0.008 | 0.270 ± 0.031 |
| OmniNFT−routing | ✗ | ✓ | ✓ | 2.631 ± 0.081 | 5.470 ± 0.073 | 0.209 ± 0.008 | 0.357 ± 0.037 |
| OmniNFT−surgery | ✓ | ✗ | ✓ | 2.948 ± 0.089 | 5.426 ± 0.071 | 0.200 ± 0.008 | 0.398 ± 0.036 |
| OmniNFT−region | ✓ | ✓ | ✗ | 3.246 ± 0.083 | 5.917 ± 0.073 | 0.209 ± 0.007 | 0.334 ± 0.034 |
These ablations and trade-offs are discussed in Section 6.3 (Table 2) and Appendix D.1 (Table 13) of the revised manuscript.
These results support the contributions of the three components within the full framework, while also revealing trade-offs between audio quality and audiovisual synchronization::
- Modality-wise advantage routing effectively coordinates joint audio-video optimization by directing each branch toward feedback aligned with its own objective;
- Layer-wise gradient surgery improves the reported VQ, AQ, JavisScore, and DeSync when routing and region reweighting are retained, supporting its contribution within the complete framework.;
- Region-wise loss reweighting reflects a reasonable trade-off, accepting a localized drop in AQ to secure substantial gains in overall synchronization and multi-dimensional alignment.
W4: The main baseline is not clearly described: The LTX-2 + GDPO row in Table 1 and the LTX-2 + vanilla RL row in Table 3 report the same numbers. The paper should state which advantage formulation and which reward set this baseline uses. Without that, the size of the reported improvement over the baseline cannot be interpreted.
Thank you for pointing out this naming ambiguity. GDPO in Table 1 and vanilla RL in Table 3 refer to the exact same baseline experiment, whose reward weights and normalization formulas are detailed below. We will unify the terminology as GDPO in the revision.
To address the substantial scale differences among multiple training rewards, we adopt a "normalize-first, route-by-modality" strategy: each of the 5 rewards is first standardized independently and then aggregated into three advantage channels—video, audio, and shared synchronization—eliminating scale bias in reward weighting. The exact definitions and coefficients are detailed below:
| Scorer | Score Definition | Weight | Channel |
|---|---|---|---|
| VideoAlign | calibrated with fixed mean/std | 1 | Video |
| HPSv3 | Frame-wise scoring; mean of top 30% frames as video quality score | 1.5 | Video |
| Audiobox | 0.5 | Audio | |
| CLAP | Cosine similarity between generated audio and prompt_a embeddings | 1 | Audio |
| DeSync | , where is the segment-wise mean of predicted absolute temporal offset | 1 | Shared Sync |
These reward definitions are specified in Section 5.1 and Appendix C.3 (Table 12) of the revised manuscript.
Following GRPO, the per-reward advantage for each prompt is computed as:
where is the rollout group size and . We then construct the three advantage channels:
For advantage assignment, and are routed to the video and audio branches respectively (whereas GDPO assigns the shared sum to both branches). Both are subsequently clipped to and mapped to optimality probabilities via .
W5: Related work on multi-reward RL for visual generation is missing
Thank you for recommending these relevant works. Multi-reward aggregation and preference optimization provide an important foundation for visual generation. Unlike prior methods that focus on unimodal image or video generation, OmniNFT specifically targets the cross-modal coupling structure of joint audio-video generative models. We will clearly distinguish and discuss these works in the revised manuscript.
Gentle Reminder for the Discussion
Dear Reviewer w3e9,
We sincerely thank you for taking the time to review our paper. In our rebuttal, we respond to your comments and summarize the additional evidence and revisions.
If you have time, we would greatly appreciate hearing which concerns remain unresolved or where our explanations are still insufficient. Your perspective would help us understand what we may have missed, and we would welcome the opportunity to clarify these points.
Best regards,
Paper 17080 Authors
Response to reviewer w3e9 - part 1
We sincerely thank you for your thorough review. We appreciate your recognition of modality-wise advantage routing as a core contribution of our paper and your endorsement that routing advantages to their corresponding modality branches is well suited for joint audio-video reinforcement learning.
Below, we respond point-by-point to your comments on mechanism explanation, evaluation comprehensiveness, and related work, detailing the computational pathway of region-wise loss reweighting, additional benchmark evaluations, and how our work relates to prior literature.
Region-wise loss reweighting improves temporal synchronization because it redistributes shared synchronization feedback across video tokens. Specifically, the shared synchronization advantage Aav enters the video policy loss through Eq. (7), while the spatial weights wi preserve frame indices and emphasize visual tokens in sound-emitting regions. Consequently, even though the weights are applied solely to the video loss, they directly reshape how the synchronization objective is optimized within the video branch. We further verify spatial correspondence using localization statistics and test whether accurate spatial placement provides additional gains via spatial shuffling. We construct an automated reference benchmark using the union of bounding boxes generated by frozen Gemma3 and GroundingDINO (where Gemma3 extracts sound-source phrases and GroundingDINO detects the corresponding objects), and compare against equal-area random pixel selection:
These localization results and visualizations appear in Appendix D.5 (Table 15 and Figure 8) of the revised manuscript.
Our critical-region selection based on V2A cross-attention maps substantially outperforms equal-area random selection, confirming its strong spatial correspondence with visible sound-emitting sources.
We further validate OmniNFT's region-wise reweighting design via an attention-map spatial shuffle control: randomly permuting spatial weight positions within each latent frame while preserving the exact weight distribution, sum, training budget, and all other components, as shown below:
These spatial-shuffle results are reported in Section 6.3 (Table 3), with the protocol in Appendix D.3 of the revised manuscript.
These experimental results demonstrate that: after spatial shuffling, VQ(3.008), AQ (5.487) and DeSync (0.417) degrade substantially compared to full OmniNFT, supporting the importance of accurate spatial localization in this reweighting design.
We provide results across all 16 VBench dimensions.
The complete 16-dimension VBench results are reported in Appendix E.1, Table 16 of the revised manuscript.
Abbreviations: SC = Subject Consistency; BC = Background Consistency; TF = Temporal Flickering; MS = Motion Smoothness; DD = Dynamic Degree; AesQ = Aesthetic Quality; ImgQ = Imaging Quality; OC = Object Class; MO = Multiple Objects; HA = Human Action; C = Color; SR = Spatial Relationship; S = Scene; AS = Appearance Style; TS = Temporal Style; OC′ = Overall Consistency.