acceptodds
Under review as a conference paper at ICLR 2027

Where and When to Force: Routed Forcing for Streaming Avatars

Abstract

Audio-driven streaming avatar generation requires real-time synthesis of speech-synchronized videos with dynamic and diverse motion. Self Forcing uses Distribution Matching Distillation (DMD) to distill bidirectional video diffusion models into causal, few-step generators for real-time streaming. However, DMD minimizes a reverse KL divergence, which is inherently mode-seeking: it causes the student to discard high-dynamic modes and collapse onto static outputs, compressing both dynamics and diversity of generated videos. We find that this collapse is region-heterogeneous: person regions involving pose and gesture variations suffer the largest diversity loss, the audio-driven mouth region shows a small loss, and the background remains nearly stable. Based on this observation, we propose Routed Forcing, which routes the distillation objective by semantic region and noise stage to improve dynamics and diversity while preserving visual quality. Specifically, (1) Where to Force: Semantic-Region Routing applies Data-Forcing Distillation (DFD), which supervises the student with real videos, to the person region where diversity collapse is most severe, while retaining DMD for the mouth and background to preserve lip synchronization and scene stability. (2) When to Force: Noise-Stage Routing activates DFD at high noise stages, where real video serves as effective supervision to inject diverse and dynamic motion patterns. At low noise stages, DMD is used to refine details, avoiding blur and artifacts from spatial differences between real video and student-generated video. Experiments show that Routed Forcing improves dynamics by up to 45% and diversity by 7–25% over Self Forcing, while preserving video quality and lip synchronization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Autoregressive Video Generation, Audio-driven Avatars

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JZmYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Routed Forcing, a training strategy for addressing the loss of motion and diversity in distilled streaming avatars. The authors observe that diversity loss is concentrated in the person region, while the mouth and background are less affected. The method applies Data-Forcing Distillation to the non-mouth person region at high teacher query noise levels and retains DMD elsewhere. Experiments on SpeakerVid and AVSpeech report improvements in motion, diversity, and visual quality while retaining the inference efficiency of the original student model.

Critical strengths
  • Separating diversity loss by person, mouth, and background provides more insight than using a single score for the whole frame. This suggests a practical reason to handle these regions differently during distillation.
  • The experiments feature controlled comparisons. Results show improvements across all four metric groups on both datasets. Component and noise threshold ablations further demonstrate the effectiveness of the proposed training rule in this setting.
Critical weaknesses
  • The methodological contribution beyond DFD is limited. DFD already uses real video supervision and combines its updates with DMD. Routed Forcing retains these updates but replaces random mixing with a predefined rule based on semantic masks and a fixed noise threshold. While the regional observation is interesting, the method remains a targeted refinement of an existing training strategy rather than a significant advancement in distillation.
  • The evaluation does not demonstrate that increased variation corresponds to meaningful motion diversity, and current metrics may increase due to artifacts rather than plausible gestures.
  • FID and FVD measure quality, but direct evaluations of motion naturalness, temporal stability, and identity consistency are also needed.
  • The reference point for diversity recovery is unclear. The method starts from LiveTalk, applies 1,000 additional DMD training steps, then adds 200 Routed Forcing steps. However, Figure 6 shows that extra DMD training reduces diversity. Table 2 omits diversity measurements for the original LiveTalk checkpoint and instead reports results for OmniAvatar-1.3B rather than the actual OmniAvatar-14B teacher model. These omissions make it difficult to determine whether the method truly improves upon the original balance between quality and diversity or merely recovers losses caused by intermediate training stages. Both reference models should be evaluated using the same diversity protocol.
  • The causal explanation outweighs its supporting analysis. The introduction presents static collapse as a direct result of reverse KL mode seeking. However, selecting fewer modes does not necessarily mean choosing less dynamic ones. Appendix A provides a conditional explanation assuming that motion occupies directions of high conditional variance. But none of the above assumptions are verified in actual video representations.
Questions for authors
  • What evidence demonstrates that semantic routing offers benefits beyond simply reducing overall DFD influence? Could the authors compare it to random masks of equal area and a global DFD baseline with matched update strength? These controls would clarify whether selecting specific regions provides advantages beyond just applying less DFD supervision.
  • How does Routed Forcing compare to the original LiveTalk checkpoint and the OmniAvatar-14B teacher under the same quality and diversity evaluation protocol? Figure 6 shows that an additional 1,000 DMD steps reduce diversity.
  • Does the proposed method enhance the balance between quality and diversity compared to the original checkpoint, or does it mainly recover losses caused by this intermediate training stage?
  • How is spatial routing managed when a hand or body part in the real video aligns with background in the student rollout? Person masks are extracted from real videos, but the reported aggregate person IoU does not address correspondence at hands and moving boundaries. Could the authors provide alignment metrics for these regions and compare them to person masks extracted from student rollouts?
8 comments

Thank you for your valuable comments and for recognizing the value of our regional diversity analysis and the experimental support for our training rules from controlled comparisons and ablations on two datasets. We hope the following discussion can address your concerns!

Q1: Does semantic routing offer benefits beyond simply reducing overall DFD influence? Could the authors compare it to random masks of equal area and a global DFD baseline with matched update strength?

A1: Following your suggestion, we added two controls. They use the same noise routing (Top-50%) and the same total amount of DFD as RF, changing only how DFD is applied spatially:

  • Uniform allocation (strength-matched): Distribute the same total amount of DFD as RF uniformly across the entire frame, without selecting locations based on semantics.
  • Random mask: Use a smooth random mask over the entire frame, with the same total amount of DFD as RF.
MethodFID↓FVD↓Sync-C↑Sync-D↓RAFT↑DINO-T↑CLIP↑DINO↑LPIPS↑L1↑
Self Forcing27.17320.914.0310.975.621.073.071.374.962.14
Uniform allocation26.35312.414.0310.966.751.193.351.475.632.24
Random mask26.41315.874.0410.956.721.183.311.485.612.23
Semantic mask25.43309.924.0710.947.801.253.551.586.092.29

Table 1. Spatial allocation comparison.

The results show that the benefits of semantic routing do not come from using less DFD: with the same amount, location determines the effect. Compared with uniform allocation and the random mask, the semantic mask yields clearly higher motion dynamics and diversity (RAFT: 6.75 / 6.72 → 7.80, +15.6% / +16.1%, LPIPS: 5.63 / 5.61 → 6.09, +8.2% / +8.6%), as well as better visual quality and lip synchronization. Uniform allocation and the random mask produce almost identical results (RAFT: 6.75 vs. 6.72). With the same amount of DFD, uniform distribution and random placement perform similarly when locations are not selected based on semantics, whereas allocating DFD to human regions produces a clear improvement. This is consistent with the regional heterogeneity observed in Figure 1b of the main paper: diversity collapse is concentrated in non-mouth human regions, the mouth is constrained by audio, and the background is largely static. Applying DFD to human regions therefore substantially improves motion dynamics and diversity, while avoiding its application to the mouth prevents interference with lip synchronization.

We will include these spatial allocation controls in the final version.

Q2: How does Routed Forcing compare with the original LiveTalk checkpoint and the OmniAvatar-14B teacher under the same quality and diversity evaluation protocol?

A2: We compared Routed Forcing, the original LiveTalk checkpoint, and the OmniAvatar-14B teacher under the same evaluation protocol. The results are shown in Table 2.

MethodParametersFID↓FVD↓Sync-C↑Sync-D↓RAFT↑DINO-T↑CLIP↑DINO↑LPIPS↑L1↑
Routed Forcing1.3B25.43309.924.0710.947.801.253.551.586.092.29
LiveTalk1.3B28.77357.283.1011.624.771.073.291.327.722.97
OmniAvatar teacher14B39.57471.414.0310.965.220.987.483.5412.974.82

Table 2. Comparison of visual quality, lip synchronization, motion dynamics, and diversity among Routed Forcing, the original LiveTalk, and the OmniAvatar-14B teacher.

Routed Forcing outperforms both the original LiveTalk and the OmniAvatar-14B teacher on all visual quality, lip synchronization, and motion dynamics metrics, while maintaining overall diversity comparable to the original LiveTalk. The OmniAvatar-14B teacher achieves the highest diversity scores. Its 14B parameter count is substantially larger than Routed Forcing's 1.3B, and its greater model capacity allows it to accommodate a richer set of generation modes.

We will include this comparison and the related analysis in the final version of the paper.

Q3: Does the proposed method enhance the balance between quality and diversity compared to the original checkpoint, or does it mainly recover losses caused by this intermediate training stage?

A3: Routed Forcing improves the quality–diversity balance of the original checkpoint, rather than merely recovering losses caused by the intermediate training stage. As shown in Table 2, compared with the original LiveTalk, Routed Forcing achieves better visual quality, lip synchronization, and motion dynamics while maintaining comparable overall diversity. In addition, Table 1 of the main paper shows that Routed Forcing outperforms Self Forcing, which uses only DMD, on every evaluation metric. Together, these two comparisons support the improvement in the quality–diversity balance achieved by Routed Forcing.

Q4: How is spatial routing managed when a hand or body part in the real video aligns with background in the student rollout? The reported aggregate person IoU does not address correspondence at hands and moving boundaries. Could the authors provide alignment metrics for these regions and compare them to person masks extracted from student rollouts?

A4: We measured the coverage of hand regions in student-generated videos by the GT person mask. We approximated the generated hand regions using the convex hulls of DWPose hand keypoints, and computed the fraction of hand pixels covered by the GT person mask. Averaging across different input conditions and frames gives 80.34%. This indicates high coverage of the hands by the GT person mask.

In addition, we replaced the person mask with a version extracted from student rollouts and retrained RF while keeping all other settings unchanged. The results are shown below:

Person Mask SourceFID↓FVD↓Sync-C↑Sync-D↓RAFT↑DINO-T↑CLIP↑DINO↑LPIPS↑L1↑
Real video (ours)25.43309.924.0710.947.801.253.551.586.092.29
Student rollout25.72309.884.0810.957.691.273.501.606.022.26

Table 3. Ablation on the source of the person mask.

The results show that the two mask sources yield very similar results, with relative differences of no more than 1.6% on any metric.

We will include the hand coverage results and the person-mask source ablation in the final version.

W1: The methodological contribution beyond DFD is limited: Routed Forcing retains the DFD and DMD updates and replaces random mixing with a predefined rule based on semantic masks and a fixed noise threshold.

Response: We identify and validate allocation rules specifying at which noise stages and in which spatial regions real-data supervision should be applied. DMD and DFD provide two supervision objectives, but the objectives themselves do not determine how supervision should be allocated. Our analysis and controlled experiments show that, even when the amount of DFD and the training budget are held fixed, different temporal and spatial allocations can produce different, or even opposite, effects.

(1) When to apply it: With the same amount of DFD, high- and low-noise allocation produce different effects.

We fix the amount of DFD and change only the noise stage at which it is applied: random allocation (the original DFD recipe), the high-noise stage (t≥0.832t \geq 0.832), and the low-noise stage (t<0.832t < 0.832). The three variants start from the same checkpoint and have the same training budget.

MethodFID↓FVD↓Sync-C↑Sync-D↓RAFT↑DINO-T↑CLIP↑DINO↑LPIPS↑L1↑
Self Forcing (without DFD)27.17320.914.0310.975.621.073.071.374.962.14
Random allocation (original DFD recipe)28.78325.333.9211.105.471.113.081.424.671.84
Low-noise stage31.33357.173.8211.114.431.122.981.434.131.65
High-noise stage26.61311.854.0111.027.501.223.561.605.922.22

Table 4. Noise-stage allocation comparison. All three DFD allocation variants apply supervision to the full frame, with approximately 50% of training updates allocated to DFD. Bold indicates the best result among the three DFD allocation variants.

High-noise allocation outperforms both random and low-noise allocation on all 10 metrics. Compared with random allocation, RAFT improves by 37.1% (5.47 → 7.50), LPIPS improves by 26.8% (4.67 → 5.92), and FID decreases from 28.78 to 26.61.

In contrast, low-noise allocation performs worse than Self Forcing without DFD on 8 of the 10 metrics, including visual quality, RAFT, and pixel-level diversity. This shows that introducing the same amount of real-data supervision does not necessarily yield improvements. The noise stage at which it is applied is critical.

(2) Where to apply it: With the same amount of DFD, region selection still provides additional benefits.

As shown in Table 1 in A1, with the same noise routing and total amount of DFD, the non-mouth human semantic mask achieves higher motion dynamics and diversity than uniform allocation and the random mask, as well as better visual quality and lip synchronization. Therefore, introducing the same amount of real-data supervision does not necessarily yield the same benefits. The spatial region to which it is applied is critical.

(3) The two allocation rules jointly improve quality, motion dynamics, and diversity.

Table 1 of the main paper shows the overall effects of combining the two rules. The original DFD recipe selects DMD and DFD with equal probability and improves over Self Forcing on only 8 of the 20 metrics across the two datasets. Routed Forcing, in contrast, improves all 20 metrics. On SpeakerVid / AVSpeech, RAFT improves by 38.8% / 45.6% and LPIPS-div improves by 22.8% / 24.5%, respectively, while visual quality improves and lip synchronization is maintained, with an absolute Sync-C increase of 0.04 on both datasets.

Therefore, our technical contribution consists of temporal and spatial supervision allocation rules supported by analysis and controlled experiments, with spatial masks and a timestep threshold serving as a simple implementation of these rules.

We will describe these two allocation rules and their contributions more explicitly in Section 1 of the final version and include the controlled experiments above.

Reviewer_mjqUOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Routed Forcing for real-time, audio-driven avatar generation. Existing methods often produce static videos with limited motion diversity. The proposed method uses real-video supervision mainly for the body, head, and hands at high-noise stages, while using standard distillation for the mouth, background, and fine details. This improves motion and diversity without reducing lip-sync accuracy, video quality, or generation speed.

Critical strengths

The observation of region-heterogeneous diversity collapse is interesting and useful. The paper shows that the collapse is concentrated mainly in the body, head, and hand regions, while the mouth and background are less affected.

Critical weaknesses
  1. The technical contribution appears incremental. The main method is a relatively straightforward combination of existing DMD and DFD objectives using spatial masks and a timestep threshold.

  2. The qualitative improvements are not very obvious in the presented figures. Compared with previous methods, the generated frames often look similar, making it difficult to assess the practical significance of the reported numerical improvements. Also, the paper does not provide separate supplementary material with additional video comparisons. Although the PDF contains appendices, static frames are insufficient for evaluating temporal dynamics, motion naturalness, and diversity.

Questions for authors

Could the authors provide additional video comparisons or a human evaluation to demonstrate that the improvements in motion dynamics and diversity are perceptually significant? The differences shown in the current static figures are relatively subtle.

3 comments

Thank you for your valuable comments and for recognizing the value of our observation of regionally heterogeneous diversity collapse. We hope the following discussion can address your concerns!

Q1: Could the authors provide additional video comparisons or a human evaluation to demonstrate that the improvements in motion dynamics and diversity are perceptually significant?

A1: (1) Additional video comparisons: We provide side-by-side video comparisons with Self Forcing and Data-Forcing Distillation in the supplementary material. We have also added comparison videos against seven additional baselines, covering model sizes from approximately 0.18B to 14B.

These comparisons show that Routed Forcing produces more dynamic motion than Self Forcing, including larger gestures. Routed Forcing also achieves better visual quality than Data-Forcing Distillation, whose outputs exhibit visible hand artifacts.

Among the other baselines, Ditto and LeapTalk show limited hand motion, while Hallo3 also exhibits limited gesture variation. FantasyTalking exhibits slow motion changes; AniPortrait and AvatarForcing exhibit rich gesture motion but with noticeable artifacts. OmniAvatar exhibits brightness flickering in some examples.

In contrast, our method generates richer, more natural movements with a 1.3B backbone while better preserving hand structure and motion clarity, yielding better overall visual quality in these comparison videos.

(2) Human evaluation: We conducted an anonymous pairwise-comparison user study. The results show that participants prefer Routed Forcing across all five dimensions in both comparisons, including motion richness and naturalness.

The study compares Routed Forcing with Self Forcing and Data-Forcing Distillation separately. Forty participants with valid responses completed a total of 800 pairwise evaluations using 40 input conditions randomly sampled from the test set. The evaluation dimensions were motion richness (excluding jitter, flickering, and limb deformation), motion naturalness, visual quality, lip synchronization, and overall preference. For each dimension, participants selected “left is better,” “about the same,” or “right is better.” We report the Good–Same–Bad (GSB) net preference rate:

GSB=NGood−NBadNGood+NSame+NBad×100%,\mathrm{GSB}=\frac{N_{\mathrm{Good}}-N_{\mathrm{Bad}}}{N_{\mathrm{Good}}+N_{\mathrm{Same}}+N_{\mathrm{Bad}}}\times100\%,

where Good, Same, and Bad indicate that Routed Forcing is rated better than, comparable to, or worse than the comparison method, respectively. Positive values indicate a preference for Routed Forcing.

ComparisonMotion RichnessMotion NaturalnessVisual QualityLip SynchronizationOverall Preference
Routed Forcing vs. Self Forcing+53.0+39.0+15.0+17.0+45.0
Routed Forcing vs. Data-Forcing Distillation+16.0+42.0+38.0+34.0+47.0

Table 1. Human evaluation results. Values are the GSB net preference rates (%)↑ for Routed Forcing relative to each comparison method.

Compared with Self Forcing, Routed Forcing achieves net preference rates of +53.0% for motion richness and +39.0% for motion naturalness, indicating that participants perceive richer motion and also prefer its naturalness. This is consistent with the reduction in motion dynamics observed during the Distribution Matching Distillation (DMD) training used by Self Forcing. Routed Forcing mitigates this degradation by introducing real-data supervision to non-mouth human regions at high noise levels.

Compared with Data-Forcing Distillation, Routed Forcing achieves net preference rates of +42.0%, +38.0%, and +34.0% for motion naturalness, visual quality, and lip synchronization, respectively. These results are consistent with the ablation study in Table 3 of the main paper: applying real-data supervision at low noise levels in Data-Forcing Distillation produces visible artifacts, while extending this supervision to the mouth region impairs lip synchronization. Routed Forcing restricts this supervision to non-mouth human regions at high noise levels, obtaining better ratings while increasing motion richness.

The overall net preference rates are +45.0% and +47.0% for the two comparisons, respectively, supporting an improvement in the overall viewing experience with Routed Forcing.

We will include the full setup and results of this human evaluation in the final version.

W1: The technical contribution appears incremental. The main method is a relatively straightforward combination of existing DMD and DFD objectives using spatial masks and a timestep threshold.

Response: We identify and validate allocation rules for when and where to apply real-data supervision. DMD and DFD provide two supervision objectives, but the objectives themselves do not determine how supervision should be allocated.

(1) When to apply it: With the same amount of DFD, high- and low-noise allocation produce different effects.

We fix the amount of DFD and change only the noise stage at which it is applied: random allocation (the original DFD recipe), the high-noise stage (t≥0.832t \geq 0.832), and the low-noise stage (t<0.832t < 0.832). The three variants start from the same checkpoint and have the same training budget.

MethodFID↓FVD↓Sync-C↑Sync-D↓RAFT↑DINO-T↑CLIP↑DINO↑LPIPS↑L1↑
Self Forcing (without DFD)27.17320.914.0310.975.621.073.071.374.962.14
Random allocation (original DFD recipe)28.78325.333.9211.105.471.113.081.424.671.84
Low-noise allocation31.33357.173.8211.114.431.122.981.434.131.65
High-noise allocation26.61311.854.0111.027.501.223.561.605.922.22

Table 2. Noise-stage allocation comparison. All three DFD allocation variants apply supervision to the full frame, with approximately 50% of training updates allocated to DFD. Bold indicates the best result among the three DFD allocation variants.

High-noise allocation outperforms both random and low-noise allocation on all 10 metrics. Compared with random allocation, RAFT improves by 37.1% (5.47 → 7.50), LPIPS improves by 26.8% (4.67 → 5.92), and FID decreases from 28.78 to 26.61.

In contrast, low-noise allocation performs worse than Self Forcing without DFD on 8 of the 10 metrics, including visual quality, RAFT, and pixel-level diversity. This shows that introducing the same amount of real-data supervision does not necessarily yield improvements. The noise stage at which it is applied is critical.

(2) Where to apply it: With the same amount of DFD, region selection still provides additional benefits.

The spatial allocation rule is based on the regional heterogeneity observed in Figure 1b of the main paper: diversity collapse is concentrated in non-mouth human regions, the mouth is constrained by audio, and the background is largely static. Following the noise-stage comparison above, we further test whether selecting supervision regions according to these differences remains beneficial when noise routing and the total amount of DFD are fixed.

To this end, we add two controls: uniform allocation and a random mask. The former distributes DFD mixture weights uniformly across the entire frame and adjusts their strength so that the sum of spatial weights in each frame matches that of Routed Forcing. The latter uses a smooth random mask over the entire frame and matches the same weight sum. Neither selects locations based on semantic region information, and both use the same Top-50% noise routing as Routed Forcing.

MethodFID↓FVD↓Sync-C↑Sync-D↓RAFT↑DINO-T↑CLIP↑DINO↑LPIPS↑L1↑
Self Forcing27.17320.914.0310.975.621.073.071.374.962.14
Uniform allocation26.35312.414.0310.966.751.193.351.475.632.24
Random mask26.41315.874.0410.956.721.183.311.485.612.23
Semantic mask25.43309.924.0710.947.801.253.551.586.092.29

Table 3. Spatial allocation comparison. Semantic mask corresponds to the non-mouth human mask used by Routed Forcing.

With the same amount of DFD, the semantic mask achieves higher motion dynamics (RAFT: 6.75 / 6.72 → 7.80) and diversity (LPIPS: 5.63 / 5.61 → 6.09) than uniform allocation and the random mask, as well as better visual quality and lip synchronization. Therefore, introducing the same amount of real-data supervision does not necessarily yield the same benefits. The spatial region to which it is applied is critical.

Therefore, our technical contribution consists of temporal and spatial supervision allocation rules supported by analysis and controlled experiments, with spatial masks and a timestep threshold serving as a simple implementation of these rules.

We will describe these two allocation rules and their contributions more explicitly in Section 1 of the final version and include the controlled experiments above.

If you have any further concerns about the reliability of the method, please let us know, and we will provide additional clarification.

Dear Reviewer mjqU,

With the discussion phase ending in about a day, we would greatly appreciate any further guidance or confirmation. We wanted to follow up on our previous response to your valuable suggestions. Your insights are very important to us, and we are eager to ensure that we have addressed all concerns appropriately.

Warm regards,

The Authors

Reviewer_uJ3fOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

To address the region-heterogeneous diversity collapse induced by Distribution Matching Distillation (DMD) in streaming audio-driven avatar generation, this work proposes the Routed Forcing training strategy. The strategy routes distillation objectives differentially across two dimensions: (1) Semantic-region routing: Data-Forcing Distillation (DFD) is deployed exclusively in the non-mouth person region where diversity loss is most prominent to inject real-world dynamic patterns, while DMD is preserved for the mouth and background regions to maintain lip synchronization and scene stability. (2) Noise-stage routing: DFD is activated at high noise stages to introduce diverse motion patterns, and switched to DMD at low noise stages to refine fine details, thus avoiding cross-trajectory blur and artifacts. Validated on two public datasets, the proposed method achieves substantial improvements in dynamics and diversity while preserving video quality and lip synchronization.

Critical strengths
  1. The idea of refining supervision signals and supervision intensity along both temporal and spatial dimensions is novel and insightful. In my view, it can provide new optimization guidance for the community working on streaming avatar generation.
  2. This paper is well organized and easy to follow.
Critical weaknesses
  1. Some state-of-the-art baselines are missing. The audio-driven avatar models compared in this paper are mainly baselines from 2024 to 2025, while streaming avatar has been a fast-growing research direction over the past year. Therefore, I consider it necessary to include comparisons with works published in the first half of 2026, such as SoulX-LiveAct (Zhen et al., 2026), Hallo-Live (Li et al., 2026), and Lip Forcing (Cho et al., 2026).

  2. Video demos are absent. For tasks like streaming avatar generation, providing demonstration videos is a fundamental requirement, as only video demos can intuitively reveal potential issues in generation quality. The lack of video demos may raise readers’ concerns about the actual performance of the generated videos, such as temporal continuity and generation stability.

  3. Subjective evaluation is missing. For dimensions such as dynamics and diversity, I argue that subjective human evaluation should be incorporated. Traditional computational metrics can hardly fully reflect the naturalness of generated motions; higher metric scores do not necessarily guarantee higher perceptual naturalness.

Questions for authors
  1. Add the baseline comparisons mentioned in the weaknesses section.
  2. (Not sure if the submission policy allows this) Add video demos covering different scenarios, identities and art styles, as well as side-by-side comparison demos with baselines.
  3. Add subjective human evaluation.
4 comments

Thank you for your valuable comments and for recognizing the novelty of our temporal and spatial supervision design, as well as the clarity of the presentation. We hope the following discussion can address your concerns!

Q1: Add the baseline comparisons mentioned in the weaknesses section.

A1: Thank you for pointing out these relevant works. Among the three, Hallo-Live and Lip Forcing address different tasks and cannot be directly compared on the same benchmark. SoulX-LiveAct addresses the same task as ours (first frame + audio → video), and we have conducted an additional comparison with it.

(1) Hallo-Live: A different task. Hallo-Live jointly synthesizes speech and video conditioned on text. Its paper does not report a mode driven by external audio, so it cannot be evaluated on a benchmark that requires lip synchronization with the given test audio.

(2) Lip Forcing: A different task. Lip Forcing performs video-to-video lip synchronization, regenerating only the region around the mouth. Head and body dynamics are inherited from the input video, whereas these are precisely the aspects targeted by Routed Forcing. It therefore does not constitute a comparison under the same benchmark setting.

(3) SoulX-LiveAct: Additional comparison. We evaluated SoulX-LiveAct under the same evaluation settings, with speed measured on a single GPU for all methods. The results are shown below:

MethodParametersFPS↑Latency (s)↓FID↓FVD↓Sync-C↑Sync-D↓RAFT↑DINO-T↑
LiveTalk1.3B24.820.3328.77357.283.1011.624.771.07
SoulX-LiveAct18B6.005.9626.50312.765.748.818.992.08
Routed Forcing1.3B24.820.3325.43309.924.0710.947.801.25

Table 1. Comparison with SoulX-LiveAct.

The results show that Routed Forcing achieves better visual quality (lower FID and FVD) with approximately 1/14 as many parameters. Its throughput is approximately 4.1 times that of SoulX-LiveAct (FPS: 6.00 → 24.82), while its latency is approximately 1/18 as high (5.96 s → 0.33 s).

SoulX-LiveAct achieves better lip synchronization and dynamics scores. This gap mainly stems from the difference in backbone scale. SoulX-LiveAct has 18B parameters in total, comprising a 14B backbone and an approximately 4B audio module. The audio module alone is already larger than our entire model. In contrast, Routed Forcing retains the LiveTalk architecture and uses causal OmniAvatar-1.3B.

We will include SoulX-LiveAct in the final version and add discussions of Hallo-Live and Lip Forcing to the related work section.

Q2: Side-by-side video demonstrations comparing the proposed method with the baselines are needed.

A2: We provide side-by-side video comparisons with the baselines in the supplementary material. These comparisons show that Routed Forcing produces more dynamic motion than Self Forcing, including larger gestures. Routed Forcing also achieves better visual quality than Data-Forcing Distillation. The outputs of Data-Forcing Distillation exhibit visible hand artifacts.

Q3: Add subjective human evaluation.

A3: To evaluate the perceptual naturalness of generated motion, we conducted an anonymous pairwise-comparison user study. The results show that participants prefer Routed Forcing across all five dimensions in both comparisons, including motion richness and naturalness.

The study compares Routed Forcing with Self Forcing and Data-Forcing Distillation separately. Forty participants with valid responses completed a total of 800 pairwise evaluations using 40 input conditions randomly sampled from the test set. The evaluation dimensions were motion richness (excluding jitter, flickering, and limb deformation), motion naturalness, visual quality, lip synchronization, and overall preference. For each dimension, participants selected “left is better,” “about the same,” or “right is better.” We report the Good–Same–Bad (GSB) net preference rate:

GSB=NGood−NBadNGood+NSame+NBad×100%,\mathrm{GSB}=\frac{N_{\mathrm{Good}}-N_{\mathrm{Bad}}}{N_{\mathrm{Good}}+N_{\mathrm{Same}}+N_{\mathrm{Bad}}}\times100\%,

where Good, Same, and Bad indicate that Routed Forcing is rated better than, comparable to, or worse than the comparison method, respectively. Positive values indicate a preference for Routed Forcing.

ComparisonMotion RichnessMotion NaturalnessVisual QualityLip SynchronizationOverall Preference
Routed Forcing vs. Self Forcing+53.0+39.0+15.0+17.0+45.0
Routed Forcing vs. Data-Forcing Distillation+16.0+42.0+38.0+34.0+47.0

Table 2. Human evaluation results. Values are the GSB net preference rates (%)↑ for Routed Forcing relative to each comparison method.

Compared with Self Forcing, Routed Forcing achieves net preference rates of +53.0% for motion richness and +39.0% for motion naturalness, indicating that participants perceive richer motion and also prefer its naturalness. This is consistent with the reduction in motion dynamics observed during the Distribution Matching Distillation (DMD) training used by Self Forcing. Routed Forcing mitigates this degradation by introducing real-data supervision to non-mouth human regions at high noise levels.

Compared with Data-Forcing Distillation, Routed Forcing achieves net preference rates of +42.0%, +38.0%, and +34.0% for motion naturalness, visual quality, and lip synchronization, respectively. These results are consistent with the ablation study in Table 3 of the main paper: applying real-data supervision at low noise levels in Data-Forcing Distillation produces visible artifacts, while extending this supervision to the mouth region impairs lip synchronization. Routed Forcing restricts this supervision to non-mouth human regions at high noise levels, obtaining better ratings while increasing motion richness.

The overall net preference rates are +45.0% and +47.0% for the two comparisons, respectively, supporting an improvement in the overall viewing experience with Routed Forcing.

We will include the full setup and results of this human evaluation in the final version.

Thank you for your response; some of my concerns have been well addressed. However, the provided video demos are not entirely satisfactory. For digital human tasks, in particular, the audio-visual synchronization is rather weak, and the image clarity is not particularly high.

I certainly understand the limitations associated with using a 1.3B parameter model, as you mentioned. Yet, this raises another issue: the proposed method has not been validated at a larger model scale. In my experience, there are significant differences in training characteristics between small-scale and large-scale DiT (Diffusion Transformer) models for real-time streaming video; therefore, it cannot be assumed that techniques effective for small models will reliably transfer to larger ones.

Overall, I consider this a "borderline" paper. While it offers a certain degree of innovation, the experiments conducted on small models do not fully demonstrate its practical utility, and the quality of the video demos is not particularly impressive.

Thank you for recognizing the innovation in our work. We hope the following discussion can address your concerns!

Q1: The proposed method has not been validated at a larger model scale.

A1: We adopt a teacher–student configuration commonly used in streaming avatar generation, with a 1.3B causal student for distillation. Works on streaming avatar generation, such as LiveTalk [1], JoyStreamer-Flash [2], AvatarForcing [3], Knot Forcing [4], and LeapTalk [5], also use 1.3B causal students, demonstrating that the 1.3B model scale is a common choice in related research. Therefore, we do not consider the absence of larger-scale validation a fundamental limitation of our work.

Another consideration is the low-latency requirement of streaming avatars. The 1.3B configuration offers lower latency and higher throughput. As shown in Table 2, on a single GPU, the 1.3B model achieves approximately 4.1 times the throughput of the 18B SoulX-LiveAct [6] (24.82 vs. 6.00 FPS), with approximately one eighteenth of its latency (0.33 vs. 5.96 seconds). These advantages are important for real-time interaction and give the 1.3B configuration practical value in balancing inference cost and responsiveness.

Given the limited rebuttal period, we have not been able to retrain our method on a larger backbone to evaluate its effectiveness at that scale. We will include larger-scale validation experiments in the final version.

Q2: The quality of the video demonstrations is not particularly impressive.

A2: Some audio clips in our test set contain rapid speech, presenting a challenging setting for audio-driven avatar generation. To provide a more direct assessment of our method, we have added comparison videos against seven baselines to the supplementary material, covering model sizes from approximately 0.18B to 14B.

We cannot compare against methods that lack publicly available weights or runnable code, such as JoyStreamer-Flash [2] and Knot Forcing [4]. Alongside models of the same scale, we include larger models with 5B and 14B parameters. The compared baselines are listed in Table 3.

MethodParameters
Ditto [7]0.18B
AvatarForcing [3]1.3B
LeapTalk [5]1.3B
OmniAvatar [8]1.3B
AniPortrait [9]2.5B
Hallo3 [10]5B
FantasyTalking [11]14B

Table 3: Baselines included in the supplementary video comparisons and their parameter counts.

These videos show that, compared with Self Forcing, our method generates richer movements and larger-amplitude gestures, mitigating motion degradation after distillation. Compared with Data-Forcing Distillation, our method maintains rich motion while achieving better visual quality, with fewer visible hand artifacts.

Among the other baselines, Ditto and LeapTalk show limited hand motion, while Hallo3 also exhibits limited gesture variation. FantasyTalking exhibits slow motion changes; AniPortrait and AvatarForcing exhibit rich gesture motion but with noticeable artifacts. OmniAvatar exhibits brightness flickering in some examples.

In contrast, our method generates richer, more natural movements with a 1.3B backbone while better preserving hand structure and motion clarity, yielding better overall visual quality in these comparison videos.

We hope these clarifications address your concerns and welcome any further questions.

References

[1] LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillation.

[2] JoyStreamer-Flash: Real-time and Infinite Audio-Driven Avatar Generation with Autoregressive Diffusion.

[3] AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising.

[4] Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animation.

[5] LeapTalk: Breaking the Latency-Quality Trade-off in Talking Head Generation.

[6] LiveAct: Towards Hour-Scale Real-Time Human Animation with Neighbor Forcing and ConvKV Memory.

[7] Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis.

[8] OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation.

[9] AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation.

[10] Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer.

[11] FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.