- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper addresses few-step distillation of a unified audio-video diffusion model that jointly generates synchronized video and audio from one shared model. The authors argue that the prevailing modality-symmetric recipe undermines the temporal structure of audio, and propose Modality-Aware Asynchronous Distillation (MAAD), which decouples the objective and step schedule of each modality within one shared student through asynchronous denoising and student-context trajectory alignment (SCTA). On MiniMax-H3 (33B), MAAD outperforms released four-step distillations on all nine singing metrics, improving MuQ FAD from 11.37 to 4.58, and matches or exceeds the 50-step teacher on four of six song-quality scores.
Critical strengths
-
Reasonable motivation: The motivation for treating the two modalities differently is sensible to me, and it provides a coherent starting point for the proposed design.
-
Strong four-step performance on singing metrics: On the 33B MiniMax-H3 backbone, MAAD is the best four-step model across all nine Sing350 singing metrics, improving MuQ FAD from 11.37 to 4.58, and it matches or exceeds the 50-step teacher on four of six song-quality scores. Achieving this quality at four steps is the main consideration supporting acceptance.
-
Low additional cost of asynchronous denoising: The asynchronous denoising scheme adds only about 6% extra computation, so the gains in audio quality do not come at a large efficiency price.
-
Helpful demo: The demo video and website are helpful and support the credibility of the reported results.
Critical weaknesses
Major weaknesses
W1. The ablation may attribute the gain to extra audio steps rather than to the claimed mechanism: In Table 4, the asynchronous component alone raises the SongBench score from 4.784 to 4.926, whereas adding the remaining components of MAAD raises it only from 4.926 to 4.977, a much smaller increment than that of async-only over DM-only. This raises the question of whether the main improvement comes from giving audio more steps rather than from the claimed effect that the symmetric recipe erodes the temporal structure of audio. Because it bears directly on the paper's central claim, this is the single consideration that most affects my assessment.
Minor weaknesses
W2. Limited backbone coverage: The method is validated on a single backbone. Additional backbones would make the effectiveness of the proposed method more convincing.
W3. Reporting of fine-grained metrics may be selective: Table 2 highlights only the dimensions on which MAAD is strong (Coherence and Melody), whereas Table 7 shows that for the Instrumental dimension the Traj./DM setting is the highest, at 5.1538 versus 5.0940 for MAAD. The metric selection is therefore not fully comprehensive.
W4. Outdated references: Some arXiv references should be updated to their published versions, for example MusicRL (ICML).
Questions for authors
I am overall positive about this paper. My specific questions follow from the weaknesses listed above. The main one is whether the improvement in Table 4 should be attributed to the additional audio steps rather than to the claimed erosion of the temporal structure of audio under the symmetric recipe.
8 comments
Response to Reviewer Mysh (2/2)
W2. Does MAAD transfer to another backbone? We appreciate this suggestion. We agree that a second backbone, especially one with a different architecture, makes the evidence more convincing. We therefore applied MAAD to LTX-2.5, which has separate video and audio streams coupled by cross-attention, whereas H3 is single-stream. An audio-only step on LTX-2.5 runs only the audio stream, which reads the cached video keys and values. We used the same schedule as on H3 (4 video / 20 audio updates; 4.12 token-normalized NFE) and compared with a symmetric DM/DM 4/4 student trained under the same setup.
| VABench | Visual Reality↑ | Expressiveness↑ | Visual QA↑ | Alignment↑ | DeSync↓ | ViCLIP↑ | CLAP↑ | AudioBox↑ | DNSMOS↑ |
|---|---|---|---|---|---|---|---|---|---|
| Teacher | 4.470 | 4.333 | 0.624 | 4.436 | 0.563 | 0.193 | 0.324 | 3.579 | 1.484 |
| DM/DM 4/4 | 4.182 | 4.315 | 0.646 | 4.291 | 0.870 | 0.187 | 0.291 | 3.400 | 1.378 |
| MAAD | 4.151 | 4.350 | 0.642 | 4.347 | 0.612 | 0.198 | 0.324 | 3.468 | 1.508 |
| Sing350 | MuQ FAD↓ | CLAP FAD↓ | Onset F1↑ | Vocal↑ | Melody↑ | Instr.↑ | Musicality↑ | Coherence↑ | Clarity↑ |
|---|---|---|---|---|---|---|---|---|---|
| Teacher | – | – | – | 4.950 | 4.601 | 5.147 | 3.745 | 3.815 | 3.664 |
| DM/DM 4/4 | 21.74 | 0.245 | 0.651 | 3.941 | 4.803 | 4.353 | 3.294 | 3.281 | 3.188 |
| MAAD | 17.82 | 0.196 | 0.711 | 4.688 | 4.935 | 4.572 | 3.491 | 3.423 | 3.332 |
The results show that MAAD transfers to LTX-2.5. On Sing350, it improves all nine metrics, e.g., Vocal 3.941 → 4.688 and MuQ FAD 21.74 → 17.82. On VABench it improves seven of nine, including DeSync (0.870 → 0.612 s), while Visual Reality and Visual QA are lower (4.151 vs. 4.182; 0.642 vs. 0.646). Both students are further from the teacher than on H3, likely because the reduction in forward passes is larger: the guided LTX-2.5 teacher uses 120 passes, against 50 for H3.
W3. Is the reporting of fine-grained metrics selective? Thank you for the careful reading of Tables 7–8 (Tables 16–17 in new version). We agree that the paper should show the full picture. Table 2 (Table 4 in new version) reports the means over all dimensions plus Coherence and Melody, the dimensions most tied to long-range structure; all twelve dimensions are in Tables 7–8 (Tables 16–17 in new version). The full picture is a trade-off that is consistent with our analysis: DM favors component-level audio quality, whereas trajectory supervision better preserves long-range musical organization. DM on audio gives the best Vocal and Instrumental scores (Traj./DM first, DM/DM second). Trajectory supervision (MAAD) gives the best Melody, Structure, Arrangement, Mixing and Musicality, and all five SongEval dimensions. Relative to Traj./DM, MAAD is lower on Vocal (−0.0709) and Instrumental (−0.0598) and higher on Structure (+0.2780) and Melody (+0.1610). With audio-only steps, trajectory supervision keeps component-level quality close to DM and avoids the large drop of Traj. only in Table 4 (Table 13 in new version). Future work will investigate improved trajectory optimization to match DM on these dimensions while retaining the benefits for long-range musical organization. We have stated this trade-off in Sec. 5.4 (Sec. 5.4 in new version) and refer to Tables 17–18 (Tables 16–17 in new version) in the caption of Table 4 (Table 4 in new version).
W4. Outdated references. Thank you for pointing this out. We have cited MusicRL (ICML 2024) in Related Work (Sec. 2 in new version), alongside other audio generators trained with objectives beyond denoising (e.g., DMOSpeech and Presto), and update the arXiv references that have since been published.
We thank the reviewer again and hope the equal-step analyses address the main concern.
Dear Reviewer Mysh,
Thank you again for your constructive feedback. We have incorporated the following updates into the revised manuscript, with changes marked in red:
- Mechanism: Matched-step ablations show that the audio objective improves musical structure, with larger Melody gains on prompts with stronger long-range dependencies (Tables 3 and 5).
- Transfer: On LTX-2.5, MAAD improves all nine Sing350 metrics and seven of nine VABench metrics over DM/DM 4/4 (Table 11).
- Reporting: We explicitly discuss the Vocal/Instrumental versus musical-structure trade-off across all twelve dimensions, added MusicRL, and updated published references.
We hope these results address your concerns about the mechanism, transferability, and reporting,, and we would appreciate it if you could reconsider the score.
Response to Reviewer Mysh (1/2)
We thank the reviewer for the positive assessment, and for finding the motivation sensible and the four-step singing results strong.
W1. Is the gain due to extra audio steps rather than the claimed mechanism? Thank you for this key question, which bears directly on our central claim. Table 4 (Table 13 in new version) compares the two parts only through mean scores, which can hide gains concentrated in a few dimensions. We therefore compare the three Sing350 students dimension by dimension: DM/DM 4/4 → 4/20 (DM only → async only in Table 4 (Table 13 in new version)) changes only the steps, and DM/DM 4/20 → MAAD 4/20 changes only the objective. Parentheses: relative change from the row above.
The results show that both contribute, to different aspects: additional audio steps notably improve vocal clarity, while trajectory supervision improves melody and musical structure, the long-range dependencies our mechanism concerns. The limited audio quality of existing four-step models led us to give audio more steps, and the extra steps improve every SongBench dimension, including Vocal (+5.21%). At equal steps, DM/DM 4/20 keeps slightly clearer vocals, but MAAD improves Melody, Structure, Arrangement, Mixing and Musicality, and all five SongEval dimensions. Its Structure (+0.1666) and Arrangement (+0.0603) gains exceed those from extra steps (+0.0708 and +0.0282), so the small mean gain (+0.0518) combines larger gains in these musical dimensions with slight decreases in Vocal and Instrumental. The objective also raises Dynamic Degree from 8.9% to 51.1% (teacher 43.7%).
Does the objective's gain follow long-range structure? If the symmetric objective erodes the temporal structure of audio, replacing it should help most on prompts whose outputs depend most on long-range order. To measure this dependence, we cut the first 5 s of each teacher output into five 1 s blocks, randomly reorder them (eight random orders), and record how much the teacher's score drops (SongEval Coherence for audio, AMT for video; audio crossfades are matched so that only the order differs). We then split the 350 prompts into thirds by this drop (Low/Middle/High, separately for audio and video) and, within each third, compare MAAD with DM/DM at equal steps (4 video / 20 audio updates).
The results show that, from low to high sensitivity, MAAD's Melody gain rises from +0.0087 to +0.1429 (95% CI of the High − Low difference: [+0.013, +0.243]) and its Coherence gain from +0.0306 to +0.0415, while video differences stay close to zero. Since both models use the same steps, this trend reflects the audio objective, consistent with the mechanism. Finally, the two parts are complementary: without extra steps, trajectory supervision lowers SongBench/SongEval to 4.32/3.59 (Traj. only in Table 4 (Table 13 in new version)), which is why MAAD pairs it with audio-only steps.