- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a voice-driven digital human video generation method called Avatar-Forever, aiming to achieve high quality, real-time performance, and long-term stability. The paper optimizes short-step generation efficiency and long-term generation stability by separating the parallel training framework. A recovery-oriented rolling training (RRT) method is proposed, utilizing DMD distillation to obtain an efficient four-step generator and simulating autoregressive error accumulation to train a long-term robust adapter. Furthermore, ForeverCache reduces inference computation by reusing historical features. Experiments on multiple datasets validate the superiority of this method in terms of generation quality, long-term stability, and inference efficiency, achieving a generation speed of 27.2 FPS at 768×512 resolution on a single H100 GPU.
Critical strengths
-
The proposed method is well-designed and practically applicable. Addressing the challenge of balancing efficiency and long-term stability in real-time video generation, this paper proposes a training framework that decouples these two optimization objectives. Specifically, Regression-Regression Training (RRT) simulates error accumulation through multi-step autoregression, which better meets the practical needs of long-video generation compared to training to recover from local perturbations.
-
Experimental results are substantial, demonstrating competitive system performance. The paper presents quantitative and qualitative comparisons on multiple digital human generation datasets and validates the method's effectiveness through ablation experiments and user studies. The proposed method achieves good results in terms of long-term generation quality and inference efficiency, especially its real-time single-card generation capability, showcasing strong practical application value.
Critical weaknesses
-
In the independent learning process, potential inconsistencies may exist between the training and inference distributions. While the RRT model is initially trained using 30 sampling steps, the final model is a 4-step generator extracted from the DMD. The autoregressive error distributions of these two models may differ significantly. This paper does not adequately analyze or experimentally verify this difference, nor does it sufficiently demonstrate that the learned long-term robustness can reliably transfer to the final generator.
-
The advantages of the independent learning mechanism and the rationality of the parameter combination have not been fully validated. Although the authors directly update the independently learned DMD parameters into the LoRA RRT, no perturbation analysis or parameter compatibility analysis was performed. Despite relevant ablation experiments, the necessity and superiority of the independent design itself cannot be fully demonstrated due to the lack of direct comparison with standard sequence fine-tuning or co-optimization strategies with similar learning overhead. I believe such related experiments are necessary.
Questions for authors
-
Difference between Training and Inference Distributions: RRT is trained using a 30-step expansion, but the final model employs a 4-step generation method. Does this difference affect the model's robustness? Can you provide corresponding experimental verification results?
-
Necessity of Independent Training: Compared to sequential training where RRT is fine-tuned after DMD distillation, does independent parallel training offer an overall advantage? Can you directly compare the computational costs of the two methods?
-
Importance of Parameter Combinations: Why can DMD parameter updates obtained through independent training be directly added to the RRT's LoRA? Have parameter perturbations or different weight combinations been observed to affect generation quality?
-
Long-Term Generation Stability: Current quantitative experiments primarily focus on time periods under 30 seconds. Can you provide quantitative data on identity consistency, visual quality, and audiovisual synchronization performance over longer periods (e.g., 1 minute, 5 minutes, 10 minutes) to support claims of unlimited generation capabilities?
6 comments
W2 + Q2. Independent versus sequential training: quality, compatibility, and cost
We compare both sequential orders with Decoupled: DMD→RRT (D2RRT), as requested, and RRT→DMD (RRT2D). Each configuration receives 2,000 DMD and 2,000 RRT updates at batch size 8, totaling 4,000 updates. Table D2 reports the results.
Table D2. Decoupled-training ablation (revised manuscript Table 3). Perceptual and automatic metrics are reported together; best values are bold.
| LLM Judge | Automatic Metrics | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Training configuration | Audio-Visual↑ | Visual↑ | Motion↑ | Overall↑ | IQA↑ | ASE↑ | Sync-C↑ | Sync-D↓ | FID↓ | FVD↓ |
| D2RRT | 3.901 | 3.897 | 3.723 | 3.847 | 4.828 | 3.601 | 6.283 | 8.686 | 47.750 | 1474.360 |
| RRT2D | 4.175 | 4.188 | 3.654 | 4.023 | 4.836 | 3.641 | 6.576 | 8.870 | 38.180 | 944.260 |
| Decoupled DMD + RRT | 4.250 | 4.267 | 3.750 | 4.105 | 4.851 | 3.684 | 6.694 | 8.032 | 34.978 | 901.811 |
Does sequential application of both objectives reach the same quality? The decoupled model performs best on all ten metrics. Compared with the closest sequential variant, RRT2D, FID/FVD decrease by 8.4%/4.5%, and all four perceptual scores improve. D2RRT achieves IQA=4.828, close to the decoupled model's 4.851, but its FVD is 1474.360 versus 901.811 for the decoupled model, a 38.8% relative reduction. In the corresponding visual comparisons, RRT2D has relatively clear appearance but still exhibits long-horizon color drift, whereas D2RRT shows flickering and jitter. The decoupled model better maintains appearance and temporal consistency. FVD measures video-distribution quality, while flickering is assessed qualitatively in the comparisons on our anonymous website.
Training cost and parallel development time. Based on the branch runtimes in Table D3, the RRT→DMD (RRT2D) sequential reference would take approximately 57 hours. Our independent branches can instead run concurrently and finish in approximately 37 hours when 16 A100 GPUs are available simultaneously. With the same total of 4,000 updates, this saves about 20 hours of waiting time (an estimated 35.5%) and allows distillation and recovery training to be iterated independently. This reduces wall-clock waiting time, not total GPU-hours; the estimate applies to RRT2D.
Table D3. Branch training runtimes and compute costs. Each branch uses 8 A100 GPUs instead of H100 due to limited computational resources during the rebuttal period; GPU-hours equal its measured runtime in hours multiplied by 8.
| Branch | Setting | Hardware | Batch size | Updates | Measured runtime | A100 GPU-hours |
|---|---|---|---|---|---|---|
| RRT | Per chunk: 30-step rollout, K=4 | 8×A100 | 8 | 2,000 | 36 h 44 min | 293.87 |
| Distillation | Four-step generator distillation | 8×A100 | 8 | 2,000 | 20.25 h | 162 |
Q3. How are parameter updates combined, and do merge weights affect quality?
We have completed an inference-time RRT merge-weight ablation for the final model, fixing the final DMD-distilled model and RRT update while varying only the external weight over 0/0.5/0.75/1/1.25. The external coefficient of the DMD update remains 1. is distinct from LoRA's internal scaling; omits the RRT update in this final configuration. This experiment changes inference-time composition weights, not training checkpoints, and is separate from the small-scale component-training ablation in Table 3.
Table D4. RRT merge-weight ablation with fixed final checkpoints. Perceptual and automatic metrics are reported together; best values are bold.
| LLM Judge | Automatic Metrics | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| RRT weight | Audio-Visual↑ | Visual↑ | Motion↑ | Overall↑ | IQA↑ | ASE↑ | Sync-C↑ | Sync-D↓ | FID↓ | FVD↓ |
| 0 | 4.15 | 4.12 | 3.53 | 3.95 | 4.75 | 3.42 | 6.37 | 8.29 | 41.33 | 1180.50 |
| 0.5 | 4.33 | 4.27 | 3.78 | 4.14 | 4.82 | 3.57 | 6.48 | 8.22 | 40.15 | 1070.66 |
| 0.75 | 4.36 | 4.42 | 3.90 | 4.24 | 4.84 | 3.67 | 6.58 | 8.09 | 37.49 | 940.88 |
| 1 | 4.42 | 4.50 | 4.08 | 4.32 | 4.88 | 3.73 | 6.85 | 7.94 | 33.33 | 858.06 |
| 1.25 | 4.36 | 4.45 | 3.96 | 4.27 | 4.85 | 3.71 | 6.77 | 8.11 | 35.33 | 983.23 |
Empirical effectiveness of parameter composition. With the DMD checkpoint fixed, adding the independently learned RRT update at improves all ten metrics over omitting RRT: Motion increases by 15.6%, Sync-C by 7.5%, and FID/FVD decrease by 19.4%/27.3%. These joint gains provide practical evidence that the recovery increment complements the distilled generator in motion, synchronization, and visual fidelity. The unit coefficient also retains its training-time strength. Performance falls at , however, indicating that the benefit depends on the update magnitude; the results support the tested composition rather than arbitrary parameter combinations being free of interference. We report this ablation in the revised appendix.
Q4. Identity consistency, visual quality, and audio–visual synchronization at 1/5/10 minutes
Evaluation at 1/5/10 minutes. We use 16 fixed pairs of EMTD first frames and ten-minute LibriVox audio clips. We compare Decoupled with DMD-only to assess how the complete framework performs over long horizons relative to distillation without recovery training. We include SoulX-FlashTalk as an external baseline for its strong overall generation quality and fast inference, which makes multi-input ten-minute evaluation practical given the substantially longer runtimes of other baselines. Each model generates one continuous ten-minute trajectory per input at , 25 fps. The shorter durations are prefixes of those trajectories. Scores average one-minute windows within each prefix and then input pairs equally. Dino-S is a DINOv2-based proxy for appearance consistency.
Table D5. Visual quality, appearance consistency, and synchronization over minute-scale prefixes. Best values within each duration are bold.
| Model | Prefix (min) | IQA↑ | ASE↑ | Sync-C↑ | Sync-D↓ | Dino-S↑ |
|---|---|---|---|---|---|---|
| SoulX-FlashTalk | 1 | 4.5996 | 3.6738 | 6.8824 | 8.1764 | 0.91544 |
| DMD-only | 1 | 4.7899 | 3.9251 | 5.5958 | 9.2131 | 0.86746 |
| Decoupled | 1 | 4.8599 | 3.9313 | 6.9478 | 7.8801 | 0.92759 |
| SoulX-FlashTalk | 5 | 4.6131 | 3.6904 | 6.7950 | 8.2576 | 0.91608 |
| DMD-only | 5 | 4.7860 | 3.8323 | 5.4879 | 9.2856 | 0.81365 |
| Decoupled | 5 | 4.8520 | 3.9057 | 6.8412 | 7.9551 | 0.92975 |
| SoulX-FlashTalk | 10 | 4.6150 | 3.6913 | 6.7585 | 8.2745 | 0.91604 |
| DMD-only | 10 | 4.7751 | 3.8129 | 5.3077 | 9.5200 | 0.80389 |
| Decoupled | 10 | 4.8575 | 3.9106 | 6.7834 | 7.9880 | 0.92047 |
Quality, consistency, and synchronization. Decoupled achieves the best means on all five metrics at 1, 5, and 10 minutes. At ten minutes, its IQA/ASE and Dino-S remain 4.8575/3.9106 and 0.92047, compared with 4.7751/3.8129 and 0.80389 for DMD-only; Sync-C/D also favor Decoupled (6.7834/7.9880 versus 5.3077/9.5200). It likewise outperforms SoulX-FlashTalk on all five means, with stronger gains in image quality and aesthetics and smaller margins in appearance consistency and Sync-C. The combined evidence shows that our advantage extends beyond 30-second generation to the measured ten-minute horizon across multiple inputs. The DMD-only comparison uses different efficiency-branch initializations and therefore evaluates complete configurations rather than isolating RRT alone. We have added these results to the appendix.
Thank you for recognizing the method's design, experimental coverage, and practical single-GPU value. We address W1/Q1 together through the four-step-history experiment and W2/Q2 through the training-quality and cost comparisons. Q3 and Q4 then cover merge-weight sensitivity and minute-scale generation.
W1 + Q1. Training–inference mismatch and four-step-history adaptation
Experiment. Starting from our final DMD + RRT model, we freeze the DMD branch and continue training the RRT branch for 1,000 updates, using four denoising steps per rollout chunk with K=4. We then evaluate the original and fine-tuned models on 30-second generation.
Table D1. Additional RRT adaptation using four-step histories. Best values are bold.
Results. Using four-step histories for additional adaptation does not improve the final generator: all ten metrics favor the original model. Visual/Motion scores decrease from 4.50/4.08 to 4.39/4.02; synchronization also deteriorates, with Sync-C falling from 6.85 to 6.03 and Sync-D rising from 7.94 to 8.44. FID/FVD increase by 33.0%/14.9%, alongside lower IQA and ASE. These results show that matching rollout and inference step counts alone is insufficient to improve perceptual quality or robustness under the tested fine-tuning recipe.
Why use 30-step rollouts? The rollout schedule primarily controls the quality of the historical video supplied as conditioning during RRT; it does not prescribe the final model's inference cost. We use 30 steps to obtain sufficiently high-quality history conditions for recovery learning. Simply reducing the original, non-distilled RRT branch to four sampling steps can produce lower-quality histories than four-step generation with the composed DMD-distilled model. Thus, matching the number of sampling steps alone need not match conditioning quality. This distinction motivates our 30-step rollout design; the fine-tuning experiment above provides a separate empirical test of using four-step histories in the final model.