- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
UniAVC unifies audio-video generation and editing in one model. It first trains unimodal branches, then freezes them while learning cross-modal bridges from paired data. Asymmetric noise schedules and clean-branch supervision support joint generation, bidirectional cross-modal generation, selective editing, and branch refinement. Experiments on JAVEditBench, InsAVE-80K, Verse-Bench, and VGGSound report competitive performance across these tasks, with gains on several editing and cross-modal generation metrics. These results illustrate the breadth of the unified framework.
Critical strengths
-
The system addresses a consequential workflow problem with broad capability coverage. One checkpoint supports joint generation, editing, and both cross-modal generation directions. In Table 2, UniAVC improves JAVEditBench instruction compliance and video fidelity to 4.42 and 4.48, respectively, and achieves an InsAVE-80K FVD of 150.86. Table 3 also reports strong cross-modal generation results. This makes the contribution practically interesting even without leading every individual metric.
-
The separation of unimodal learning from paired alignment, together with explicit clean-branch supervision, is well motivated. Training the bridges on paired data after establishing unimodal competence is a plausible way to use limited paired supervision. Pinning the preserved branch also provides a concrete mechanism for selective editing, rather than relying on approximate reconstruction of that branch. Table 4 provides initial evidence that the progressive recipe and specialized inference modes improve editing quality.
Critical weaknesses
-
The central scheduling claim is not sufficiently isolated by the ablations. Table 4 compares the asymmetric schedule (2,9) against (1,1), changing both asymmetry and the amount of schedule shifting. It does not establish whether the gain comes from asymmetry rather than a stronger shared shift. Moreover, clean-branch supervision directly trains the boundary conditions required for selective generation, but its contribution is not separated from the interior schedule. The single-stage/progressive comparison also needs a clearer account of matched data, updates, and trainable parameters. These distinctions matter because the paper attributes its benefits specifically to asymmetric scheduling and the staged allocation of paired supervision.
-
The empirical protocol needs reconciliation before the comparative and data-efficiency claims are reliable. The MiniMax-H3 aggregate JAVEditBench results differ between Tables 2 and A2: for example, AVS is 0.2307 versus 0.3009, and AVQ is 3.61 versus 3.85. Appendix A1.1 also describes alignment training as using the held-out split of InsAVE-80K, while Section 4.1 describes evaluation on its held-out split. This wording does not prove overlap, but leaves train/test separation unclear. Until the results and split definitions are reconciled, the reported advantages are difficult to audit.
Questions for authors
-
Can you compare asymmetric scheduling with shared shifted schedules such as (2,2) and (9,9), and with independently sampled modality noise levels, while keeping clean-branch subtasks and the rest of training fixed? Separately, what changes when clean-branch supervision is removed? These controls would determine whether the claimed scheduling contribution is supported.
-
Please specify the exact InsAVE-80K training/evaluation split identifiers and whether source clips are disjoint. Please also clarify the matched data/update/parameter budgets in the single-stage versus progressive ablation, and reconcile the sampling probabilities with the numbers of samples seen in Table A1. This would determine how much confidence to place in the paired-data-efficiency claim.
-
Which MiniMax-H3 results in Tables 2 and A2 are correct, and do both tables use the same checkpoint, examples, and inference protocol? For Table 2, please specify whether UniAVC uses self-refinement and how its total denoising budget compares with the baselines.
7 comments
Response to Reviewer FtdY (2/2)
[W1 & Q2] Single-stage budget.
The single-stage baseline is initialised from the same LTX-2.3 checkpoint with all parameters trainable and finetuned in one stage over the union of the unimodal and paired corpora, with sampling probabilities re-calibrated to match the numbers of samples seen in Table A1, so both paradigms consume identical data exposures and differ only in curriculum (§4.3, p.8, L425–L430). A unimodal sample contributes loss on its own modality while gradients still update the whole model. With the budget fixed, we attribute the performance gap to the optimisation conflict between unimodal and cross-modal objectives inside a mixed batch (§4.3, p.8 L431 to p.9 L474), together with the scarcity of paired supervision (§1 C1, p.2, L77–L87). Given the target to tackle pair-wise data scarcity, we demonstrate that our progressive paradigm is particularly effective for this adverse data situation, and leave paired-data scaling studies for future work (§A2.4, p.18, L968–L970).
[W2 & Q2 & Q3] InsAVE-80K and Table 2 versus Table A2.
Both are human errors on our side, and we thank the reviewer for catching them. The official InsAVE-80K splitting is 79K training and 1K evaluation pairs, with source clips disjoint between the two. Alignment finetuning uses only the 79K training pairs, and the 1K evaluation pairs are used solely for assessment (§A1.1, p.16, L850–L852). On the tables, Table A2's per-task columns had been carried over from an earlier preliminary run instead of the run behind Table 2. Table A2 is now corrected and agrees with Table 2 throughout (p.21), and the corrected rows can also be seen below:
| Metric | Method | Overall | Sub. Edit | BG. Edit | Speech | Sub. Add | Sub. Rm |
|---|---|---|---|---|---|---|---|
| VQ$\uparrow$ | MiniMax-H3 | 0.0605 | 0.0609 | 0.0618 | 0.0614 | 0.0618 | 0.0557 |
| VQ$\uparrow$ | UniAVC | 0.0599 | 0.0637 | 0.0634 | 0.0631 | 0.0619 | 0.0423 |
| AQ$\uparrow$ | MiniMax-H3 | 2.42 | 2.32 | 2.31 | 2.69 | 2.57 | 2.25 |
| AQ$\uparrow$ | UniAVC | 2.59 | 2.59 | 2.52 | 2.82 | 2.57 | 2.40 |
| AVS$\uparrow$ | MiniMax-H3 | 0.2307 | 0.2300 | 0.2046 | 0.2998 | 0.2111 | 0.1643 |
| AVS$\uparrow$ | UniAVC | 0.2975 | 0.3203 | 0.2829 | 0.3229 | 0.2672 | 0.0579 |
| IC$\uparrow$ | MiniMax-H3 | 4.11 | 4.08 | 4.37 | 4.40 | 4.12 | 3.48 |
| IC$\uparrow$ | UniAVC | 4.42 | 4.35 | 4.47 | 4.53 | 4.40 | 4.36 |
| VF$\uparrow$ | MiniMax-H3 | 4.17 | 4.10 | 4.33 | 4.23 | 3.88 | 4.29 |
| VF$\uparrow$ | UniAVC | 4.48 | 4.65 | 4.23 | 4.53 | 4.36 | 4.56 |
| AVQ$\uparrow$ | MiniMax-H3 | 3.61 | 3.75 | 3.67 | 3.83 | 3.58 | 3.08 |
| AVQ$\uparrow$ | UniAVC | 3.85 | 4.15 | 3.97 | 4.00 | 4.16 | 2.76 |
All comparisons and observations were always based on Table 2, so they remain valid and consistent.
[Q3] Table 2 protocol and denoising budget.
In Table 2, UniAVC uses Full mode with self-refinement enabled. We leverage human inspection to subjectively determine which stream is satisfactory and which necessitates refinement. When needed, a single refinement round is applied to only the target stream while the accepted one is preserved, adding a cost of one extra denoising trajectory on that branch, equivalently a 50% increase in total NFE at the default (§A2.3, p.18, L956–L958). We present self-refinement as an optional trade-off, and note that the incurred latency is already acknowledged as a limitation of our model (§A2.4, p.18, L963–L965). However, we highlight that the Base row in Table 4, with neither inference mode enabled, already surpasses the SOTA baselines such as MiniMax-H3 on instruction compliance (4.22 vs 4.11), video fidelity (4.31 vs 4.17), synchrony (0.2643 vs 0.2307) and overall audio-video quality (3.68 vs 3.61) while evaluating at a similar denoising budget.
We thank the reviewer once again for their careful reading and constructive feedback, which has significantly strengthened the paper.
Reference
[1] Unison: Harmonizing Motion, Speech, and Sound for Human-Centric Audio-Video Generation - ECCV 2026
Thank you for the additional experiments and clarifications. The shifted symmetric schedules and clean-branch ablation meaningfully strengthen the evidence, and the split clarification and corrected Table A2 address my original concerns about those reporting inconsistencies. I also appreciate the Base results and the Qwen3-Omni selector evaluation.
However, the interpretation of the headline comparison remains insufficiently established. Table 2 uses human-selected refinement with additional denoising, whereas the Base and automatic-selector results describe different operating conditions. Their promising scores do not fully resolve how selection and computation affect the reported advantage. Please specify which results use human or automatic selection, what information the selector receives, and the total inference cost, including selection and refinement. Identifying which baseline comparisons use comparable budgets would help establish the practical advantage of the method.
The training attribution also remains unresolved: matching data exposures does not isolate curriculum order when the single-stage baseline updates all parameters but the progressive recipe freezes unimodal branches during alignment. This supports a comparison of complete recipes, rather than a curriculum-only conclusion. These remaining issues affect the interpretation of the main comparative gains and the specific methodological contribution. I therefore maintain my rating of 2, while remaining open to reconsideration after clarification.
Response to Reviewer FtdY (1/2)
We thank the reviewer for the constructive feedbacks and questions. We hereby address each of them below.
We agree that the original comparison confounded asymmetry with shift magnitude. We therefore conduct additional experiments following your suggested settings to isolate the two factors, holding clean-branch subtasks and everything else fixed. Three findings stand out:
Beside, we argue that independently sampled noise levels form a separate training regime rather than a variant of our schedule: uncoupled draws break the deterministic link between the noise pair and the step counter that our sampler and inference modes rely on (§2, p.4, L171–L189), and let the leading modality flip freely, spending about half of training on video-first states that our (9,2) control shows to be detrimental. Additionally, Unison [1] also reports that fully decoupled timesteps destabilise optimisation without a curriculum. All three controls are reported below and have been added to Table 4 of §4.3.
We conduct another training for the model without clean-branch subtasks while holding all other settings fixed. Since these boundary-state samples are what place intent-aware editing and self-refinement inside the training distribution, neither mode is available to this established variant (3), so we compare it against our Base variant (2) with clean-branch subtasks included during training but using identical basic inference mode.
AVS and AVQ show significant drops under Base mode, indicating that clean-branch subtasks implicitly help strengthen cross-modal alignment learning without affecting unimodal qualities (VQ, AQ). The Full variant (1), with both inference modes enabled, marks the ceiling that removing the subtasks places out of reach. The schedule and clean-branch supervision are therefore jointly necessary: the schedule provides the asymmetric trajectory while the subtasks make the boundary states trainable (p.3, L138–L140).