- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
AV-DMD adapts DMD2 to four-step joint audio-visual generation. It reads video and audio score differences from separate noise schedules while keeping each joint score-network input internally synchronized, then trains one shared fake score with branch-specific losses on the corresponding states. On LTX-2.3, VBench Overall rises from 0.6417 to 0.6740 and JavisBench-mini CLAP from 0.372 to 0.416 against matched Joint DMD, with improved lip-sync scores but lower dynamic degree.
Critical strengths
- The two synchronized joint states permit modality-specific score estimation while retaining one shared fake score and the unchanged four-step inference path.
- Matched Joint DMD comparisons across VBench, JavisBench-mini, and AVBench identify gains in Semantic and CLAP and report regressions rather than only aggregate scores.
- The digital-human analysis repeats SyncNet comparisons on the same successfully tracked clips, strengthening the lip-sync result against a face-track coverage confound.
Critical weaknesses
- VBench Dynamic Degree falls from 77.8 to 62.5, Quality from 0.8419 to 0.8345, and AV-Align barely changes (0.2420 to 0.2431). The submitted materials provide no playable videos or user study to assess this perceptual trade-off.
- The synchronization constraint is not tested against mixed-noise evaluation, and the time/noise conditioning of the second score pass is unclear. This leaves the central in-distribution rationale insufficiently verified.
- The manuscript has no explicit limitations or failure-case discussion.
Questions for authors
- Can the authors provide matched-prompt playable examples, including failures, and a blinded user study of motion, audio quality, and synchronization?