acceptodds
Under review as a conference paper at ICLR 2027

AV-GRPO: Modality-Anchored Decoupling Diffusion Reinforcement Learning for Joint Audio-Video Generation

Abstract

Recent years have witnessed remarkable progress in joint audio-video generation. Nevertheless, existing models still suffer from limited per-modality fidelity, inadequate text-modality alignment, and weak cross-modal synchronization. Reinforcement-learning-based post-training offers a promising avenue for addressing these shortcomings. However, naively extending such approaches to joint audio-video generation is challenging. Heterogeneous multimodal rewards entangle the learning signals of the two modalities and obscure credit assignment. Jointly optimizing both modality towers also incurs substantial computational cost despite their distinct optimization dynamics. Furthermore, the difficulty of synchronization evaluation varies with the sampled counter-part modality, making fair reward comparison difficult. To address these challenges, we propose AV-GRPO, a modality-anchored online diffusion reinforcement learning framework, together with 5DAV, a fully decoupled and difficulty-controllable training dataset. AV-GRPO integrates three key components: (1) modality-anchored rollouts that disentangle learning signals while reducing anchor-induced difficulty variation; (2) trajectory-locked frozen-tower optimization that reduces training costs and redirects credit assignment, thereby simplifying optimization. and (3) adaptive objectives and perturbation strengths tailored to modality-specific dynamics and sample difficulty. Collectively, these designs transform coupled multi-modal preference learning into a set of conditional unimodal subproblems, enabling more accurate reward attribution and more effective synchronization optimization. We construct 5DAV, a dataset decoupled along five dimensions, to facilitate systematic training. Experiments on JavisBench and VABench demonstrate that AV-GRPO consistently outperforms LTX-2.3 in generation quality, semantic alignment, and cross-modal synchronization under both LoRA and full fine-tuning settings. Extensive ablation studies further validate the effectiveness of the proposed designs.

Primary area:
generative models
Keywords:
Joint Audio-Video Generation, Multi-modal

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2xB4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose AV-GRPO, a modality-anchored online diffusion reinforcement learning framework, together with 5DAV, a fully decoupled and difficulty-controllable training dataset

Critical strengths

The method seems feasible. The figures are enough for a paper.

Critical weaknesses
  1. The discussion of related work is quite limited. For example, joint audio–video generation is briefly discussed around Line 452, but the coverage is far from comprehensive. There are substantially more relevant works in this area, covering both closed-source and open-source models, than the 6 papers currently cited. More importantly, the paper provides very little discussion or comparison of these existing approaches, which makes it difficult to understand how the proposed method is positioned relative to prior work.

  2. In Line 93, the paper introduces the 5DAV dataset with 5,760 prompts. However, there is no table comparing 5DAV with existing datasets in terms of scale, supported functionalities, video duration, or other important properties. And, Section 3.1 does not have any appendix reference for more details on this dataset. In addition, the dataset construction process is barely described in Line 263 or elsewhere. Without more details on how the prompts and samples are collected, generated, filtered, or quality-controlled, it is difficult to assess the quality and actual value of this dataset.

  3. There are also several places where the notation and presentation are quite confusing. For example, in Lines 268–300, the paper uses expressions such as “Semantic Hierarchy (W)”. I could not understand what the letter in parentheses represents. It does not appear to be an abbreviation, an initial, or a mathematical symbol defined elsewhere in the paper. This notation feels arbitrary and should either be clearly defined or removed.

  4. I am similarly confused by Figure 2. The text refers to “rows 2–3” and “rows 4–5”, but I can only identify either four rows, or six rows under a different interpretation of the layout. Neither interpretation seems consistent with the description. Even after trying to infer the intended correspondence, I still cannot tell which model produces the upper row and which produces the lower row. More importantly, it is difficult to visually distinguish whether the upper or lower result is supposed to be better. I suggest clearly labeling the model associated with each row and using red boxes or other visual annotations to highlight the relevant differences.

  5. Figure 1 has a similar presentation issue. Its caption contains only one sentence and does not sufficiently explain the figure. In particular, the audio examples on the right are labeled “1, 2, ..., G”. When I tried to understand what “G” means here, I found that the same symbol is used with a completely different meaning elsewhere in the paper, e.g., Line 148 states that “Flow-GRPO draws GG trajectories”. This makes the figure unnecessarily ambiguous.

  6. I am also confused by Table 1. The AV-Align column contains almost no meaningful comparison: only the LTX-based model has a reported value, while the remaining entries are shown as “–”. More generally, the table contains a large number of missing entries. For open-source models evaluated on open-source data, I do not see a clear reason why so many results should be left unavailable, especially when these missing values substantially reduce the completeness and usefulness of the comparison.

  7. Table 1 also appears to contain missing references and possibly multiple naming errors. For example, I searched the paper for “See&Hear” and “FolleyCrafter” but could not find corresponding discussions or citations elsewhere in the manuscript. I strongly suspect that these may be typos, such as “FolleyCrafter” instead of “FoleyCrafter” and “See&Hear” instead of “Seeing&Hearing”. In the latter case, however, the corresponding paper of “See&Hear” does not appear to be cited at all.

These are not isolated issues. I encountered many similar inconsistencies and confusing descriptions throughout the manuscript. In its current form, the paper appears to have serious proofreading, notation, citation, and presentation problems. These issues significantly affect readability and make it unnecessarily difficult to understand and evaluate the technical contribution. I do not think the current presentation meets the standard required for acceptance.

Questions for authors

See above

3 comments

Dear Reviewer 2xB4,

We thank you for your careful reading and detailed comments on the presentation of our paper. We apologize for the unclear notation, figure descriptions, and incomplete discussion of related work. Your feedback has helped us identify where clearer explanations and more consistent presentation are needed. We respond to each concern below.

W1: Coverage and Positioning of Related Work

We have expanded Section 4.1 to include additional representative open-source and closed-source audio–video generation models. Beyond adding references, we now discuss their generation paradigms and summarize their relevant capabilities and design choices based on publicly available descriptions. We also clarify AV-GRPO’s position within this literature. AV-GRPO builds on an existing joint audio–video generator and introduces reward-guided post-training to improve generation quality, text alignment, and cross-modal synchronization while retaining standard joint denoising at inference. The expanded discussion makes the relationship between our contribution and existing audio–video generation approaches more explicit.

W2: Dataset Comparison, Construction, and Quality Control

5DAV is a training dataset designed specifically for audio–video post-training optimization. It contains 5,760 text prompts organized into 288 combinations of five explicitly specified attributes, with 20 prompts per combination. Its design enables balanced coverage of these combinations, targeted sampling by synchronization difficulty and instruction granularity, and controlled subset ablations. In our experiments, Given a text prompt from 5DAV,each generated video contains 97 frames at 24 FPS.

Comparison of dataset annotations and controllability. 5DAV combines generation prompts with explicit synchronization-difficulty and instruction-granularity levels, balanced across 288 five-attribute combinations.

DatasetScaleContent OrganizationAudio–Video Generation TextExplicit Synchronization-Difficulty LevelsExplicit Instruction-Granularity LevelsBalanced Coverage of Attribute Combinations
VGGSound200K+ audio–video clipsSound-category labels onlyCategory labels; detailed generation prompts require additional constructionNot specifiedNot specifiedNot specified
JavisBench10,140 captioned audio–video samples5 aspects and 19 categoriesCaptions derived from audio–video samplesNot specifiedNot specifiedNot specified
VABench778 T2AV + 521 I2AV cases7 content categories with structured prompts and QA pairsStructured generation promptsNot specifiedNot specifiedNot specified
5DAV (Ours)5,760 prompts5 explicitly specified attributes and 288 combinationsPrompts constructed for specified attribute combinations4 levels3 levels288 combinations × 20 prompts

“Not specified” means that the published dataset description does not specify the corresponding level-based annotation or balanced combination design.

We have added Appendix F detailing the construction and quality control of 5DAV. For each combination of five explicitly specified attributes, human annotators manually author candidate prompts. To ensure semantic richness and simulate diverse real-world user inputs, annotators employ rich sentence structures and varied expressions rather than fixed templates, yielding 80 candidates per category. Candidates are then manually filtered for label validity, category compliance, audio–visual depictability, linguistic quality (checking for grammar, contradiction, and ambiguity), and exact/near-duplicate removal. Within each category, annotators group candidates by scene, entity, action, and sound description to perform diversity-aware selection, retaining 20 semantically and lexically diverse prompts (nominal 25% acceptance) and authoring additional candidates if fewer than 20 pass. All selected prompts then undergo a final manual quality control, where annotators rate category compliance, audio–visual depictability, temporal-window feasibility, and linguistic quality; low-scoring prompts are rejected or revised, and disagreements are adjudicated by a third senior annotator. Each final prompt is released with its five-dimensional labels. 5DAV stores no audio–video clips, since the RL rollouts are generated online by the policy during training.

The practical value of this design is supported by the dataset experiments in Section 3.4.1. We compare 5DAV with VGGSound under the stated training protocol.

W3: Notation for the Five Dataset Dimensions

We have replaced the single-letter labels with abbreviations derived from the corresponding dimension names, such as “Semantic Hierarchy (SH).”, to improve clarity.

W4: Model Labels and Visual Comparisons in Figure 2

We have revised the description to clarify that, for each prompt, the upper video is generated by the original LTX-2.3 model, while the lower video is generated by AV-GRPO (full). We have also added Appendix G, which details, for each case, how AV-GRPO (full) outperforms LTX-2.3 in both the generated video and audio, with visual annotations highlighting the key differences between the two outputs.

W5: Figure 1 Caption and the Meaning of G

We have expanded the caption of Figure 1 to make it clearer.

The G in the figure and the G in Line 148 both refer to the number of videos or audio samples drawn per rollout group. Since each video/audio sample corresponds to one trajectory, the two uses of G are consistent. G can be chosen as needed and does not have to be fixed. In AV-GRPO, we set G=8; that is, for each prompt, we generate G=8 videos or audio samples along with their generation trajectories.

W6: Missing Entries in Table 1

We would like to clarify that the missing entries for the cascaded baselines in Table 1 are not results omitted in our paper. They are inherited from the reporting protocol of the original JavisBench paper and the existing comparison tables. Specifically, in JavisDiT[1] (Table 1) and JAVISDIT++[2] (Table 1), the papers that introduce JavisBench, these entries are already missing for the same cascaded baselines. Likewise, OmniNFT [3] (Table 1) reports the same missing entries for these baselines. JavisDiT leaves modality-dependent metrics unreported for A2V/V2A methods because these methods receive one modality from an upstream model, so metrics for that modality do not directly measure the generation capability of the method being compared. Other scores, such as VQ/AQ for these cascaded baselines, are also unavailable in the source tables. Our Table 1 preserves exactly the same reporting coverage as JavisDiT and OmniNFT; we do not additionally omit any results that were available in the original benchmark paper or in these prior comparison tables.

[1] JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization

[2] JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generation

[3] OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation

AV-Align is an additional metric that we include beyond the main JavisBench. It provides additional evidence of AV-GRPO’s synchronization improvements and complements the broader comparisons based on JavisScore and DeSync. We now additionally report AV-Align values for two powerful models as reference.

W7: Method Names and References

“See&Hear” follows the abbreviated name used in OmniNFT and JavisDiT++, and refers to “Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners.” We have also corrected “FolleyCrafter” to “FoleyCrafter” and added references for all methods in Table 1.

Reviewer_2xB4OpenReview ↗

Comment

Thank you for your response.

However, it seems that other reviewers still share similar concerns about the manuscript. Therefore, I will maintain my current score for now and continue to follow the discussion among the reviewers.

Reviewer_h4JMOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents AV-GRPO, an RL post-training framework for joint audio-video generation. It fixes one modality’s full denoising trajectory while sampling and optimizing the other, alternating between the two towers with modality-specific noise levels and loss weightings. The authors also introduce 5DAV, a training set of 5,760 prompts organized along five dimensions. Experiments with LTX-2.3 on JavisBench and VABench show improvements over the base model and GDPO in generation quality, text alignment, and audio-video synchronization, though gains vary across metrics and fine-tuning settings.

Critical strengths

The shared-anchor design is well motivated: keeping one modality’s full trajectory fixed makes synchronization rewards easier to compare and gives a clearer learning signal to the tower being updated. Alternating frozen-tower updates also makes large-scale post-training practical, with full-parameter training of the 22B model demonstrated on eight A800 GPUs, while separate noise strengths and loss weightings accommodate the different learning dynamics of audio and video. The method is clearly presented, and Appendix A offers a theoretical interpretation under idealized assumptions. Results on JavisBench and VABench show improvements over the base model and GDPO across most quality, alignment, and synchronization metrics, while the dataset and alternation-schedule ablations provide useful supporting evidence. The structured 5DAV prompt set is also a useful contribution for studying performance across different types and difficulty levels of audio-video generation.

Critical weaknesses

My main concern is that the experiments do not clearly separate the contributions of shared-anchor rollouts, tower freezing, video-step reweighting, and ANC. Although the paper studies the alternation schedule, controlled ablations of these components are still needed to explain where the gains come from. The connection between the idealized Gibbs analysis and practical training also remains unclear, and stability measurements would help assess this gap. Comparisons with other modality-aware post-training methods would further strengthen the evaluation. Finally, while the authors acknowledge reward hacking and introduce a CLAP guardrail, its effectiveness is not directly tested through ablations or human evaluation, and some implementation details, including the GRPO clipping parameter and guardrail threshold, remain unspecified.

Questions for authors
  1. Appendix B provides most of the training settings, but could you specify the GRPO clipping parameter ε and show how sensitive the results are to each tower’s noise strength and KL coefficient?
  2. Could you add controlled ablations for shared-anchor rollouts, tower freezing, video-step reweighting, and ANC to clarify each component’s contribution?
  3. Under the same training budget, how much do peak GPU memory usage and total training time decrease compared with updating both towers jointly?
  4. Could you report the CLAP guardrail threshold and compare results with and without it, ideally with a small human study or independent evaluator check for reward hacking?
  5. I would suggest adding a comparison with a recent modality-aware post-training method, such as OmniNFT or Adaptive Reward Routing, where a matched setup is feasible.
  6. Is ANC used during evaluation, and how does it affect generation quality and diversity?
  7. Do you plan to release the complete 5DAV prompts, training code, and configuration files, and if so, when?
5 comments

Dear Reviewer h4JM,

We thank you for your thoughtful review and for recognizing the motivation of shared-anchor training, the practicality of alternating frozen-tower updates, and the value of 5DAV. We appreciate your suggestions on controlled ablations, implementation details, and evaluation. We address each question below.

Q1: GRPO Clipping and Hyperparameter Sensitivity

We use a GRPO clipping parameter of ε = 0.2 for both the audio and video towers. We have explicitly included this setting in Appendix B. The following tables report the sensitivity of AV-GRPO to each tower’s noise strength and KL coefficient.

Sensitivity to the video tower’s noise strength and KL coefficient. All experiments are trained for 120 steps under identical conditions and evaluated on JavisBench-mini.

SettingNoise StrengthKL CoefficientSampling StatusVQ ↑CLIP ↑DeSync ↓
AV-GRPO(full) Default0.020.01valid videos5.8060.3180.733
Lower noise strength0.010.01valid videos5.8160.3140.741
Higher noise strength0.040.01noise-only videos---
Lower KL coefficient0.020.005overexposed videos--
Higher KL coefficient0.020.02valid videos5.8120.3150.739

The current setting achieves a favorable trade-off among the tested configurations: excessively high noise strength disrupts denoising during sampling, and an excessively low KL coefficient leads to overexposed videos during training. As long as these thresholds are not exceeded, training remains relatively robust to the choice of hyperparameters.

Sensitivity to the audio tower’s noise strength and KL coefficient. All experiments are trained for 120 steps under identical conditions and evaluated on JavisBench-mini.

SettingNoise StrengthKL CoefficientSampling StatusAQ ↑CLAP ↑DeSync ↓
AV-GRPO(full) Default0.80.002valid audio5.3480.4340.733
Lower noise strength0.40.002valid audio5.3310.4290.730
Higher noise strength1.20.002noise-only audio---
Lower KL coefficient0.80.001valid audio5.2980.4240.746
Higher KL coefficient0.80.004valid audio5.3350.4320.737

The current setting achieves a favorable trade-off among the tested configurations: excessively high noise strength disrupts denoising during sampling, and an excessively low KL coefficient still allows the model to generate valid audio, but makes training unstable. As long as these thresholds are not exceeded, training remains relatively robust to the choice of hyperparameters.

Q2: Controlled Component Ablations

Controlled component ablations. Each variant changes one component of the full method. All experiments are conducted on eight A800 GPUs, trained for 480 steps under the same training configuration, and evaluated on JavisBench.

VariantJavisScore ↑DeSync ↓AV-align↑
AV-GRPO LoRA0.2130.5540.382
w/o shared anchors0.2010.6740.367

Without shared anchors, the fairness of synchronization-reward evaluation is compromised. When both audio and video vary simultaneously, the model struggles to learn an effective optimization signal for synchronization, causing a substantial degradation in the audio–video synchronization capability of the trained model compared with the shared-anchor setting.

VariantVQ ↑AQ ↑CLIP ↑CLAP ↑AVHScore ↑DeSync ↓
AV-GRPO LoRA5.8165.6330.3300.4490.2350.554
w/o tower freezing5.8565.4280.3170.4320.2190.661

Without tower freezing, for a given audio/video sample with a high advantage, the advantage gradient is propagated to both towers simultaneously, even though the other tower is not directly responsible for generating that modality. This leads to somewhat confused credit assignment and hinders sustained model improvement. If shared anchors are further removed, a case may arise where an audio–video sample has poor video but good audio, yet the gradient still updates both towers, resulting in incorrect credit assignment. In addition, tower freezing provides the advantage of reducing memory usage, as shown in the table in Q3.

Then, under our default training configuration, removing ANC causes the training rollouts to produce noise-only outputs, while removing video-step reweighting leads to overexposed videos during training. These results highlight the roles of ANC in stabilizing rollout sampling and video-step reweighting in maintaining stable optimization. The resulting failures prevent a reliable quantitative assessment of anchoring and freezing alone under the same configuration. We have added qualitative examples in Appendix H to illustrate these failure modes and clarify the roles of these supporting components.

VariantObserved Issue
AV-GRPONone
w/o video-step reweightingoverexposed videos
w/o ANCnoise-only videos or audios

Q3: Training Memory and Time

Peak GPU memory usage under matched training settings and eight NVIDIA A800 80 GB GPUs.

MethodTraining StepsPeak GPU Memory (MiB) ↓Total Training Time (hours) ↓
AV-GRPO(lora) w/o tower freezing48064055132
AV-GRPO(lora)48049623133
AV-GRPO(full) w/o tower freezing-OOM-
AV-GRPO(full)48079858135

Q4: CLAP Guardrail and Human Evaluation

The CLAP guardrail threshold is η_CLAP = 0.4.

Effect of the CLAP guardrail. All experiments are conducted on eight A800 GPUs, trained for 480 steps under the same training configuration, and evaluated on JavisBench-mini.

MethodAQ ↑CLAP ↑DeSync ↓
LTX-2.35.1400.4180.775
AV-GRPO(full) w/ CLAP guardrail5.7560.4690.641
AV-GRPO(full) w/o CLAP guardrail6.4220.3670.636

Human preference for audio–text alignment. We randomly select 100 prompts from JavisBench and compare AV-GRPO with and without the CLAP guardrail. Three raters independently evaluate each audio pair with randomized presentation order and hidden model identities. For each pair, raters choose among three options: prefer the first audio, prefer the second audio, or declare a tie. The final outcome per prompt is determined by majority vote among the three raters. A model is preferred if at least two raters prefer it; all other outcomes are counted as ties. Results are aggregated across the 100 prompts as shown below.

ComparisonWith Guardrail Preferred (%)Tie (%)Without Guardrail Preferred (%)
Audio–text alignment on 100 JavisBench prompts9190

Q5: Comparison with Modality-Aware Post-Training

Thank you for suggesting this comparison. As of November 7, 2026, we could not locate an official public training implementation of Adaptive Reward Routing. We therefore added a comparison with OmniNFT using its official implementation.

We trained OmniNFT and AV-GRPO with LoRA on the same LTX-2.3 backbone and our 5DAV dataset for 480 training steps. We matched the prompt batch size, number of candidates per prompt, video resolution and duration, and evaluation settings. We retained OmniNFT’s modality-wise advantage routing, layer-wise gradient surgery, and region-wise loss reweighting.

All training experiments were conducted on eight NVIDIA A800 80GB GPUs. Under the matched LoRA setting, AV-GRPO outperforms OmniNFT on AQ, CLIP, CLAP, AV-IB, AVHScore, and DeSync, while OmniNFT achieves a higher VQ, indicating that AV-GRPO provides stronger text alignment, cross-modal coherence, and synchronization. We also attempted full-parameter fine-tuning of OmniNFT on the same hardware, but repeated attempts with different memory-management configurations exceeded the available GPU memory under our experimental settings. AV-GRPO’s trajectory-locked, alternating frozen-tower optimization reduces the memory required for parameter updates by optimizing only one modality tower at a time. This design enables full-parameter post-training of the 22B LTX-2.3 model on the same eight-GPU node. We therefore include AV-GRPO’s full-parameter results to demonstrate how this memory-saving strategy translates into additional performance gains within the same hardware constraints.

Comparison with OmniNFT on JavisBench. All trained variants use LTX-2.3 and 5DAV. The two LoRA variants provide the direct comparison between methods.

MethodFine-TuningTraining StepsVQ ↑AQ ↑CLIP ↑CLAP ↑AV-IB ↑AVHScore ↑DeSync ↓
LTX-2.3——5.8555.0970.3180.4080.2120.2010.757
OmniNFTLoRA4806.0415.5120.3240.4320.2300.2240.663
AV-GRPOLoRA4805.8165.6330.3300.4490.2360.2350.554
AV-GRPOFull4805.9025.7980.3270.4680.2470.2410.607

Q6: ANC during Evaluation

ANC is used only during training rollouts and is disabled during evaluation and inference. During training, stochastic sampling produces different target-modality candidates conditioned on the same anchor trajectory. ANC limits the standard deviation of the stochastic noise increment at each denoising step to stabilize training rollouts. Under our training sampling settings, removing ANC causes denoising failure, producing noise-only audio and video samples.

All reported evaluation results use the standard inference procedure with ANC disabled. Consequently, ANC does not directly modify the evaluation-time sampling process. Its contribution is to maintain usable training rollouts, allowing the model to learn from stochastic exploration.

Q7: Release Plan

Yes. We are committed to publicly releasing the complete 5DAV dataset, including all 5,760 prompts and their labels, together with the training code and configuration files for both LoRA and full-parameter fine-tuning. All materials are ready for release, and we will provide the public repository link once the anonymous review period ends.

Reviewer_4gdEOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The proposed method, AV-GRPO, post-trains a joint audio-video generator by fixing one modality's denoising trajectory, sampling a group for the other modality, and updating only that tower. Alternating the two phases lets candidates within each group compare against the same audio or video anchor. The proposed method also uses modality-specific noise and objectives, adaptive noise clipping, and a CLAP guardrail. The authors construct 5DAV, a set of 5,760 prompts covering five semantic and difficulty dimensions, and evaluate LoRA and full tuning of LTX-2.3 22B on JavisBench and VABench. Most metrics improve over the base model and GDPO. Appendix A analyzes a simplified theoretical version based on alternating Gibbs kernels.

Critical strengths
  1. Compared with fixing only the final sample, reusing the complete anchor trajectory can control the cross-modal inputs at every denoising step. Candidates in a group are then compared against the same anchor, and only one tower requires a backward pass. This addresses one reason synchronization rewards can be hard to interpret.

  2. The full-tuning model improves JavisBench AQ from 5.097 to 5.798 and AV-IB from 0.212 to 0.247. VABench DeSync falls from 0.800 to 0.542 (as shown in Tables 1-2). Improvements also appear in JavisScore and lip synchronization, rather than only the optimized synchronization score. Both LoRA and full tuning are evaluated, and the paper acknowledges the individual metrics that get worse.

Critical weaknesses
  1. Section 3.4 only tests the dataset and alternation interval, while many other parts are changed at the same time. This makes it hard to know which parts actually improve the results. The paper also lacks an ablation study of the CLAP guardrail. For efficiency, it would be helpful to compare training time and GPU memory usage with GDPO under the same settings, including the cost of sampling anchor trajectories.

  2. Appendix A.12 assumes that updating one tower does not affect the other tower. However, this may not be true when the two towers interact, even if one is frozen. The theory also uses a different sampling process from the actual method. The authors should explain how these assumptions relate to the real experiments in Tables 1–2.

Questions for authors
  1. How much improvement comes from anchoring and freezing alone, without ANC, video reweighting, and the CLAP guardrail? Could the authors provide their thresholds and compare training time and GPU memory with GDPO under the same settings?

  2. Were Tables 1–2 produced using joint denoising or alternating resampling? How are the reference kernels related to the same joint distribution? Do the theoretical results still hold if the assumptions in Appendix A.12 are not fully satisfied?

4 comments

Dear Reviewer 4gdE,

We thank you for your careful review and constructive feedback, and for recognizing the benefits of complete anchor trajectories and the improvements in quality and synchronization. Your comments help us clarify the contribution of each component and the relationship between our theoretical analysis and practical training. We address your concerns point by point below.

W1/Q1: Component Contributions, Thresholds, and Training Efficiency

Under our default training configuration, removing ANC causes the training rollouts to produce noise-only outputs, while removing video-step reweighting leads to overexposed videos during training. These results highlight the roles of ANC in stabilizing rollout sampling and video-step reweighting in maintaining stable optimization. The resulting failures prevent a reliable quantitative assessment of anchoring and freezing alone under the same configuration. We have added qualitative examples in Appendix H to illustrate these failure modes and clarify the roles of these supporting components.

We evaluate the effect of the CLAP guardrail separately on JavisBench-mini. All training configurations are kept identical across experiments. Removing it heavily increases AQ and slightly improves DeSync, but reduces CLAP from 0.469 to 0.367.

VariantAQ ↑CLAP ↑DeSync ↓
LTX-2.35.1400.4180.775
AV-GRPO(full) w/ CLAP guardrail5.7560.4690.641
AV-GRPO(full) w/o CLAP guardrail6.4220.3670.636

The ANC threshold is τ = 0.5, bounding the standard deviation of the stochastic noise increment at each denoising step. The CLAP guardrail threshold is η_CLAP = 0.4. Video-step reweighting follows the timestep-dependent weights in Eq. (12) and requires no additional threshold.

For training time and GPU memory, we compare GDPO and AV-GRPO on eight A800 80GB GPUs with matched video specifications, prompt batch sizes, candidate counts, and memory-management settings. The following table reports peak per-GPU memory and total training time.

MethodTraining StepsPeak GPU Memory (MiB) ↓Total Training Time (hours) ↓
GDPO(lora)48064531120
AV-GRPO(lora)48049623133
GDPO(full)-OOM-
AV-GRPO(full)48079858135

W2/Q2: Theoretical Assumptions and Practical Sampling

  1. “Were Tables 1–2 produced using joint denoising or alternating resampling?”

Tables 1–2 use standard joint audio–video denoising. Modality-conditioned resampling and alternating tower updates are used during training to construct and optimize anchored rollout groups. Evaluation requires no additional alternating resampling.

  1. “The theory also uses a different sampling process from the actual method. The authors should explain how these assumptions relate to the real experiments in Tables 1–2.”

The theoretical chain repeatedly applies the ideal conditional kernels, using each sampled modality as the condition for the next resampling step. Actual training obtains anchors from joint sampling, fixes the complete anchor trajectory within each group, refreshes anchors across groups, and performs finite-step GRPO updates.

The connection is the conditional optimization problem: both optimize one modality conditioned on the complete counterpart trajectory. Lemma 2 establishes the ideal optimum for every fixed anchor, without requiring anchors to follow the Gibbs chain’s stationary distribution. Consequently, refreshing anchors does not change these pointwise conditional optima.

The Gibbs-chain convergence analysis provides a further result for repeated application of the ideal kernels. Corollary 1 explicitly restricts its inference statement to the same full-trajectory alternating resampling protocol. Tables 1–2 evaluate how the benefits of anchored conditional training transfer to standard joint generation. We will make these protocol distinctions explicit in Section 2.2, Appendix A.12, and the evaluation description.

  1. “How are the reference kernels related to the same joint distribution?”

H0 specifies that, for a fixed prompt, the reference kernels are the two conditionals of a common reference distribution (P_0) over paired audio–video trajectories:

$ K_v^0(\cdot\mid\tau^A)=P_0(\cdot\mid\tau^A),\qquad K_a^0(\cdot\mid\tau^V)=P_0(\cdot\mid\tau^V). $

Under this compatibility condition and the shared KL-regularized objective, the optimal conditional kernels are the corresponding conditionals of

$ P^*\propto P_0\exp(r/\beta). $

This establishes when the two modality-wise objectives share a common joint optimum while preserving audio–video dependence through conditioning. In implementation, both reference policies use the pretrained model with the corresponding fixed anchor trajectory; H0 states the compatibility condition for this common-joint interpretation.

  1. “Appendix A.12 assumes that updating one tower does not affect the other tower. However, this may not be true when the two towers interact, even if one is frozen.”

H2 concerns preservation of the other tower’s conditional kernel evaluated at the same fixed conditioning trajectory. Cross-modal dependence alone does not establish a violation: an unchanged conditional kernel can produce different outputs when its conditioning trajectory changes.

AV-GRPO retains cross-modal interactions and uses both tower freezing and complete trajectory locking. The latter ensures that candidates within each group are compared against the same cached counterpart trajectory throughout rollout and optimization. This is the operational control implemented by the method. H2 additionally specifies exact conditional-kernel realization and non-interference for the oracle analysis in Corollary 1; it is a sufficient condition for that additional result, rather than an assumption that cross-modal interactions disappear.

  1. “Do the theoretical results still hold if the assumptions in Appendix A.12 are not fully satisfied?”

The main theorem remains valid under its own stated assumptions; H2 is required for the additional training correspondence in Corollary 1. Specifically, Theorem 2 assumes H0, bounded reward, (\beta>0), and the stated contraction condition. It establishes joint optimality, conditional compatibility, and convergence of the ideal Gibbs chain without invoking H2.

Appendix A.12 introduces H2 to connect parameter updates to exact realization and preservation of those ideal kernels. If practical updates do not fully satisfy H2, that additional correspondence is not established by Corollary 1, while the preceding conditional-optimality, compatibility, and ideal-chain results remain intact.

Dear Reviewer 4gdE,

Thank you again for your constructive feedback. We wanted to follow up and ask whether any concerns remain after our response. We would be happy to clarify any specific point that would help your assessment.

We added a CLAP guardrail ablation and a matched comparison with GDPO. Compared with GDPO at 480 LoRA steps, AV-GRPO used 23.1% less peak GPU memory per GPU, with total training time increasing from 120 to 133 hours. We also added a CLAP guardrail ablation. Appendix H also illustrates the failure modes that limit a reliable estimate of the gains from anchoring and freezing alone under the default settings.

The link between Appendix A and practical training is the fixed-anchor conditional problem: both optimize one modality given the other modality’s complete denoising trajectory. Lemma 2 characterizes the optimum of the idealized objective for each fixed anchor, so it does not require the training anchors to come from a stationary Gibbs chain. Theorem 2 establishes convergence of the ideal chain without H2. H2 is needed for Corollary 1’s additional claim that exact, non-interfering tower updates realize and preserve those ideal kernels. Tables 1–2 separately evaluate the trained model using standard joint denoising.

We hope this clarifies how the analysis relates to the reported experiments.

Thank you for your time and consideration.

Thank you to the authors for their rebuttal. It has addressed most of my concerns. The authors provide several additional experiments that help clarify the contributions of the proposed components. However, they also suggest that the experimental evaluation in the initial submission was somewhat limited. Given the importance of fairness across submissions at the deadline, I will keep my original rating for now and am open to discussing it with other reviewers.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.