- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Adaptive Reward Routing, a multi-reward optimization method for joint audio-video diffusion models based on forward-process reinforcement learning. The method consists of two components: (1) Cross-Modal Influence-Guided Routing, which dynamically adjusts token-level loss weights and layer-wise gradient routing based on cross-modal attention responses, and (2) Preference-Preserving Modality-Aware Reweighting, which adjusts reward weights according to modality-specific gradient interactions while preserving predefined reward preferences. The proposed approach is evaluated on LTX-2 and LTX-2.3 using JavisBench, demonstrating quantitative improvements in generation quality, cross-modal consistency, and audio-video synchronization over existing baselines.
Critical strengths
-
Practical formulation of multi-reward optimization for joint audio-video diffusion. The paper addresses the challenge of optimizing multiple potentially conflicting reward signals across audio and video modalities. This is an important problem for joint audio-video generation, where modality-specific quality and cross-modal consistency must be optimized simultaneously.
-
Simple and practical techniques. The proposed routing and reward reweighting mechanisms are relatively straightforward and appear easy to implement within existing diffusion RL frameworks. Their practical simplicity, together with the reported improvements on LTX-2 and LTX-2.3, is a strength of the paper.
Critical weaknesses
-
Limited methodological novelty and insufficient justification of the interaction between the proposed components. The proposed method retains the underlying DiffusionNFT framework and primarily introduces heuristic mechanisms for token weighting, layer-wise gradient scaling, and reward reweighting. Although these techniques are empirically effective, the methodological contribution remains relatively incremental. Furthermore, Cross-Modal Influence-Guided Routing and Preference-Preserving Modality-Aware Reweighting are developed largely independently. In particular, gradient-conflict estimation explicitly disables token routing, even though token routing changes the gradient geometry of the actual training objective. The paper does not sufficiently justify this design choice or establish whether jointly applying these mechanisms provides benefits or causes harm.
-
Insufficient justification of the influence criterion for reward-driven optimization. The proposed routing mechanism uses cross-attention response magnitudes as a proxy for cross-modal influence. Although Section 5.4 provides intervention-based validation showing that the proxy correlates with changes in predicted velocity, this does not necessarily establish that layers with greater influence on model predictions are the most effective locations for reward-driven optimization. Similarly, the layer-wise gradient norms illustrated in Figure 1(b) are noisy and not necessarily reliable indicators of functional importance. The paper would benefit from further analysis connecting the proposed influence criterion to actual improvements in reward objectives. Additionally, comparisons with recent alternative layer-wise influence or adaptability estimation approaches, such as the constructive influence function proposed in MusicInfuser (Hong et al., CVPR 2026), would help clarify the advantages and limitations of the proposed criterion.
Questions for authors
-
Interaction between routing and reward reweighting: Section 4.3 disables token routing during gradient-conflict estimation to avoid bias from the current token weights. However, token routing changes the gradient geometry of the actual optimization objective. Is there empirical evidence that estimating reward conflicts with uniform token weights is preferable to estimating them under the actual routed objective? How sensitive is performance to this design choice?
-
Influence estimation and optimization effectiveness: Section 5.4 demonstrates that the proposed cross-attention response proxy correlates strongly with changes in predicted velocity under direct interventions. However, it remains unclear whether layers with greater prediction influence are necessarily more beneficial for reward-driven optimization. Can the authors provide evidence connecting the influence scores to actual reward improvements achieved by updating the corresponding layers? How does the proposed criterion compare conceptually or empirically with other layer-wise influence or adaptability criteria?
7 comments
Response to Reviewer Fc2z (Part 2)
(W2,Q2) 4. Prediction Influence vs. Reward-Update Utility
Thank you for highlighting the distinction between prediction influence and update utility. We clarify why prioritizing influential locations does not simply reinforce their current behavior (4-a), then test update utility directly: response-based routing outperforms alternative criteria (4-b, Table R1), and stronger routing improves optimization within the tested range (4-c, Table R2).
4-a. Rationale clarification. As validated in Sec. 5.4, our proxy measures how strongly a location influences the model’s output. This identifies where to focus updates, while DiffusionNFT’s reward-conditioned positive/negative-policy regression losses determine how to update. Allocating more updates to influential locations does not unconditionally reinforce their current behavior. Instead, it enables DiffusionNFT’s reward supervision to reinforce beneficial behavior and correct unfavorable behavior at those locations.
4-b. Experiment 1: Comparison with alternative routing criteria. (Table R1) To test whether gains depend on the response-based criterion rather than nonuniform weighting alone, we replace only the routing criterion while keeping other training settings fixed. Gradient-magnitude routing uses gradients of the unweighted DiffusionNFT loss with respect to cross-modal responses, while random routing shuffles token weights and layer scales while preserving their distributions. Our method outperforms both alternatives across all six metrics. These results support the effectiveness of cross-modal responses for allocating reward-driven updates.
Table R1. Comparison of routing criteria on LTX-2.
| Method | VQ↑ | AQ↑ | CLAP↑ | AV-IB↑ | JavisScore↑ | DeSync↓ |
|---|---|---|---|---|---|---|
| OmniNFT | 3.136 | 5.614 | 0.416 | 0.222 | 0.195 | 0.390 |
| Random routing | 3.102 | 5.534 | 0.395 | 0.210 | 0.181 | 0.405 |
| Gradient-magnitude routing | 3.201 | 5.712 | 0.411 | 0.221 | 0.200 | 0.367 |
| Response-based routing (Ours) | 3.336 | 5.868 | 0.425 | 0.235 | 0.206 | 0.341 |
4-c. Experiment 2: Effect of routing strength. (Table R2) To verify whether prioritizing influential locations benefits reward optimization, we evaluate both OmniNFT (using its own routing strategy) and our method at routing strengths , keeping other settings and post-training steps fixed on LTX-2. Here, and scale token reweighting and layer detachment, respectively, from disabled () to each method’s default (). Both methods improve as routing strengthens, while ours performs better overall at each matched strength
Table R2. Effect of routing strength on LTX-2 under the same number of post-training steps.
| Method | VQ↑ | AQ↑ | TV-IB↑ | TA-IB↑ | CLIP↑ | CLAP↑ | AV-IB↑ | AVHScore↑ | JavisScore↑ | DeSync↓ | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| GDPO (reference) | — | 2.722 | 5.450 | 0.261 | 0.138 | 0.312 | 0.347 | 0.174 | 0.175 | 0.155 | 0.671 |
| OmniNFT | 0.2 | 2.891 | 5.503 | 0.262 | 0.140 | 0.310 | 0.373 | 0.193 | 0.192 | 0.171 | 0.566 |
| OmniNFT | 0.6 | 3.048 | 5.571 | 0.264 | 0.143 | 0.311 | 0.401 | 0.212 | 0.209 | 0.187 | 0.448 |
| OmniNFT | 1.0 | 3.136 | 5.614 | 0.265 | 0.145 | 0.312 | 0.416 | 0.222 | 0.219 | 0.195 | 0.390 |
| Ours | 0.2 | 3.098 | 5.592 | 0.264 | 0.149 | 0.312 | 0.386 | 0.202 | 0.201 | 0.178 | 0.419 |
| Ours | 0.6 | 3.214 | 5.754 | 0.267 | 0.160 | 0.313 | 0.412 | 0.224 | 0.223 | 0.197 | 0.392 |
| Ours | 1.0 | 3.336 | 5.868 | 0.268 | 0.167 | 0.314 | 0.425 | 0.235 | 0.234 | 0.206 | 0.341 |
4-d. Clarifying Fig. 1(b). The gradient norms illustrate how gradient propagation changes during training; they are not used to compute our routing scores. Prediction influence is validated through interventions in §5.4, while the training experiments above establish the practical utility of response-based update allocation.
Response to Reviewer Fc2z (Part 3)
(W2,Q2) 5. Comparison with Other Layer-Wise Method (MusicInfuser[3])
Thank you for highlighting MusicInfuser[3]. Both methods use internal model signals to identify useful adaptation locations, but differ in three respects:
5-a. Estimation criterion. MusicInfuser constructs guidance from full-model and layer-skipped predictions, then ranks layers using the resulting videos’ VBench imaging quality. Our criterion uses bidirectional cross-modal response magnitudes to identify interactions influencing audio and video outputs.
5-b. Fixed selection vs. dynamic allocation. MusicInfuser selects 16 of 48 layers for music-conditioning modules before training and keeps this selection fixed. We refresh layer-wise gradient scales throughout training, while deriving token weights from the same responses, thereby tracking evolving cross-modal functionality rather than selecting adaptation locations once.
5-c. Suitability for online adaptation. MusicInfuser’s selection requires guided generation and quality evaluation. Our routing signals are collected during normal rollout, without additional intervention passes or gradient probes for routing estimation, making frequent refreshes practical.
Our criterion therefore targets lightweight, continuous token/layer update allocation, whereas MusicInfuser targets quality-guided selection of fixed adaptation layers. We will incorporate this comparison into the revision.
[3] Hong S, Kemelmacher-Shlizerman I, Curless B, et al. Musicinfuser: Making video diffusion listen and dance[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 43751-43761.
Response to Reviewer Fc2z (Part 1)
Thank you for your thoughtful and constructive feedback. We sincerely appreciate the time and care you devoted to reviewing our work. Your comments have helped us sharpen our claims and strengthen the supporting evidence.
(W1) 1. Methodological Novelty
Thank you for recognizing the problem’s importance and our method’s practicality. Our novelty lies in unified optimization, model-adaptive mechanisms, and cross-architecture applicability. To our knowledge, within joint audio-video diffusion RL, ours is the first framework to:
1-a. Unify update allocation and reward coordination. We jointly adapt where reward-driven updates act and how competing rewards are combined, addressing both dimensions within one optimization framework.
1-b. Use model evolution to guide optimization. Evolving cross-modal functionality provides online routing signals for token weighting and layer-wise gradient scaling, while changing reward conflicts demand adaptive coordination anchored to user preferences. Both mechanisms respond to the evolving model rather than follow fixed optimization rules.
1-c. Support both dual-stream and single-stream architectures. We instantiate and validate the framework on dual-stream LTX and single-stream JavisDiT++, extending adaptive reward routing beyond architecture-specific layer structures.
(W1,Q1) 2. Interaction Between Reward Coordination and Influence-Guided Routing
The two components jointly define the training objective and update, rather than operate independently:
2-a. Reward coordination determines the supervision. It combines rewards into branch-specific advantages, which determine the weights of DiffusionNFT’s positive/negative-policy regression losses.
2-b. Influence-guided routing allocates that supervision. It applies token weights to these losses and controls gradient propagation through cross-modal pathways.
Empirically, the components are complementary. On LTX-2, routing alone achieves a JavisScore of 0.190, coordination alone achieves 0.183, and their combination reaches 0.206 (Table 2), supporting the benefit of applying both.
(W1,Q1) 3. Why Disable Token Routing During Conflict Estimation?
3-a. Rationale Clarification. We choose uniform probing to separate branch-level reward coordination from local update allocation for two reasons:
3-b. Controlled comparison. To test this choice, we change only the token weights used during conflict estimation, keeping priors, warm-up, refresh frequency, training budget, and actual training routing unchanged. Uniform probing performs better across all six metrics, supporting our default design.
Table R8. Uniform versus routed token weights for conflict estimation on LTX-2.