- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The authors propose AV-GRPO, a modality-anchored online diffusion reinforcement learning framework, together with 5DAV, a fully decoupled and difficulty-controllable training dataset
Critical strengths
The method seems feasible. The figures are enough for a paper.
Critical weaknesses
-
The discussion of related work is quite limited. For example, joint audio–video generation is briefly discussed around Line 452, but the coverage is far from comprehensive. There are substantially more relevant works in this area, covering both closed-source and open-source models, than the 6 papers currently cited. More importantly, the paper provides very little discussion or comparison of these existing approaches, which makes it difficult to understand how the proposed method is positioned relative to prior work.
-
In Line 93, the paper introduces the 5DAV dataset with 5,760 prompts. However, there is no table comparing 5DAV with existing datasets in terms of scale, supported functionalities, video duration, or other important properties. And, Section 3.1 does not have any appendix reference for more details on this dataset. In addition, the dataset construction process is barely described in Line 263 or elsewhere. Without more details on how the prompts and samples are collected, generated, filtered, or quality-controlled, it is difficult to assess the quality and actual value of this dataset.
-
There are also several places where the notation and presentation are quite confusing. For example, in Lines 268–300, the paper uses expressions such as “Semantic Hierarchy (W)”. I could not understand what the letter in parentheses represents. It does not appear to be an abbreviation, an initial, or a mathematical symbol defined elsewhere in the paper. This notation feels arbitrary and should either be clearly defined or removed.
-
I am similarly confused by Figure 2. The text refers to “rows 2–3” and “rows 4–5”, but I can only identify either four rows, or six rows under a different interpretation of the layout. Neither interpretation seems consistent with the description. Even after trying to infer the intended correspondence, I still cannot tell which model produces the upper row and which produces the lower row. More importantly, it is difficult to visually distinguish whether the upper or lower result is supposed to be better. I suggest clearly labeling the model associated with each row and using red boxes or other visual annotations to highlight the relevant differences.
-
Figure 1 has a similar presentation issue. Its caption contains only one sentence and does not sufficiently explain the figure. In particular, the audio examples on the right are labeled “1, 2, ..., G”. When I tried to understand what “G” means here, I found that the same symbol is used with a completely different meaning elsewhere in the paper, e.g., Line 148 states that “Flow-GRPO draws trajectories”. This makes the figure unnecessarily ambiguous.
-
I am also confused by Table 1. The AV-Align column contains almost no meaningful comparison: only the LTX-based model has a reported value, while the remaining entries are shown as “–”. More generally, the table contains a large number of missing entries. For open-source models evaluated on open-source data, I do not see a clear reason why so many results should be left unavailable, especially when these missing values substantially reduce the completeness and usefulness of the comparison.
-
Table 1 also appears to contain missing references and possibly multiple naming errors. For example, I searched the paper for “See&Hear” and “FolleyCrafter” but could not find corresponding discussions or citations elsewhere in the manuscript. I strongly suspect that these may be typos, such as “FolleyCrafter” instead of “FoleyCrafter” and “See&Hear” instead of “Seeing&Hearing”. In the latter case, however, the corresponding paper of “See&Hear” does not appear to be cited at all.
These are not isolated issues. I encountered many similar inconsistencies and confusing descriptions throughout the manuscript. In its current form, the paper appears to have serious proofreading, notation, citation, and presentation problems. These issues significantly affect readability and make it unnecessarily difficult to understand and evaluate the technical contribution. I do not think the current presentation meets the standard required for acceptance.
Questions for authors
See above
3 comments
W3: Notation for the Five Dataset Dimensions
We have replaced the single-letter labels with abbreviations derived from the corresponding dimension names, such as “Semantic Hierarchy (SH).”, to improve clarity.
W4: Model Labels and Visual Comparisons in Figure 2
We have revised the description to clarify that, for each prompt, the upper video is generated by the original LTX-2.3 model, while the lower video is generated by AV-GRPO (full). We have also added Appendix G, which details, for each case, how AV-GRPO (full) outperforms LTX-2.3 in both the generated video and audio, with visual annotations highlighting the key differences between the two outputs.
W5: Figure 1 Caption and the Meaning of G
We have expanded the caption of Figure 1 to make it clearer.
The G in the figure and the G in Line 148 both refer to the number of videos or audio samples drawn per rollout group. Since each video/audio sample corresponds to one trajectory, the two uses of G are consistent. G can be chosen as needed and does not have to be fixed. In AV-GRPO, we set G=8; that is, for each prompt, we generate G=8 videos or audio samples along with their generation trajectories.
W6: Missing Entries in Table 1
We would like to clarify that the missing entries for the cascaded baselines in Table 1 are not results omitted in our paper. They are inherited from the reporting protocol of the original JavisBench paper and the existing comparison tables. Specifically, in JavisDiT[1] (Table 1) and JAVISDIT++[2] (Table 1), the papers that introduce JavisBench, these entries are already missing for the same cascaded baselines. Likewise, OmniNFT [3] (Table 1) reports the same missing entries for these baselines. JavisDiT leaves modality-dependent metrics unreported for A2V/V2A methods because these methods receive one modality from an upstream model, so metrics for that modality do not directly measure the generation capability of the method being compared. Other scores, such as VQ/AQ for these cascaded baselines, are also unavailable in the source tables. Our Table 1 preserves exactly the same reporting coverage as JavisDiT and OmniNFT; we do not additionally omit any results that were available in the original benchmark paper or in these prior comparison tables.
[1] JavisDiT: Joint Audio-Video Diffusion Transformer with Hierarchical Spatio-Temporal Prior Synchronization
[2] JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generation
[3] OmniNFT: Modality-wise Omni Diffusion Reinforcement for Joint Audio-Video Generation
AV-Align is an additional metric that we include beyond the main JavisBench. It provides additional evidence of AV-GRPO’s synchronization improvements and complements the broader comparisons based on JavisScore and DeSync. We now additionally report AV-Align values for two powerful models as reference.
W7: Method Names and References
“See&Hear” follows the abbreviated name used in OmniNFT and JavisDiT++, and refers to “Seeing and Hearing: Open-domain Visual-Audio Generation with Diffusion Latent Aligners.” We have also corrected “FolleyCrafter” to “FoleyCrafter” and added references for all methods in Table 1.
Comment
Thank you for your response.
However, it seems that other reviewers still share similar concerns about the manuscript. Therefore, I will maintain my current score for now and continue to follow the discussion among the reviewers.
Dear Reviewer 2xB4,
We thank you for your careful reading and detailed comments on the presentation of our paper. We apologize for the unclear notation, figure descriptions, and incomplete discussion of related work. Your feedback has helped us identify where clearer explanations and more consistent presentation are needed. We respond to each concern below.
W1: Coverage and Positioning of Related Work
We have expanded Section 4.1 to include additional representative open-source and closed-source audio–video generation models. Beyond adding references, we now discuss their generation paradigms and summarize their relevant capabilities and design choices based on publicly available descriptions. We also clarify AV-GRPO’s position within this literature. AV-GRPO builds on an existing joint audio–video generator and introduces reward-guided post-training to improve generation quality, text alignment, and cross-modal synchronization while retaining standard joint denoising at inference. The expanded discussion makes the relationship between our contribution and existing audio–video generation approaches more explicit.
W2: Dataset Comparison, Construction, and Quality Control
5DAV is a training dataset designed specifically for audio–video post-training optimization. It contains 5,760 text prompts organized into 288 combinations of five explicitly specified attributes, with 20 prompts per combination. Its design enables balanced coverage of these combinations, targeted sampling by synchronization difficulty and instruction granularity, and controlled subset ablations. In our experiments, Given a text prompt from 5DAV,each generated video contains 97 frames at 24 FPS.
Comparison of dataset annotations and controllability. 5DAV combines generation prompts with explicit synchronization-difficulty and instruction-granularity levels, balanced across 288 five-attribute combinations.
“Not specified” means that the published dataset description does not specify the corresponding level-based annotation or balanced combination design.
We have added Appendix F detailing the construction and quality control of 5DAV. For each combination of five explicitly specified attributes, human annotators manually author candidate prompts. To ensure semantic richness and simulate diverse real-world user inputs, annotators employ rich sentence structures and varied expressions rather than fixed templates, yielding 80 candidates per category. Candidates are then manually filtered for label validity, category compliance, audio–visual depictability, linguistic quality (checking for grammar, contradiction, and ambiguity), and exact/near-duplicate removal. Within each category, annotators group candidates by scene, entity, action, and sound description to perform diversity-aware selection, retaining 20 semantically and lexically diverse prompts (nominal 25% acceptance) and authoring additional candidates if fewer than 20 pass. All selected prompts then undergo a final manual quality control, where annotators rate category compliance, audio–visual depictability, temporal-window feasibility, and linguistic quality; low-scoring prompts are rejected or revised, and disagreements are adjudicated by a third senior annotator. Each final prompt is released with its five-dimensional labels. 5DAV stores no audio–video clips, since the RL rollouts are generated online by the policy during training.
The practical value of this design is supported by the dataset experiments in Section 3.4.1. We compare 5DAV with VGGSound under the stated training protocol.