acceptodds
Withdrawn from ICLR 2027

DeForcing: Efficient Rolling-Free Training and Distillation Framework for Streaming Real-time Avatar Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Streaming real-time avatar generation is important for live video conferencing, virtual agents, and interactive digital humans, where visual responses must be synthesized causally and with low latency while preserving identity, motion co- herence, and audio–visual synchronization. We present **DeForcing**, a rolling-free training and distillation framework that converts a bidirectional diffusion avatar model into a streaming causal generator. For training, the framework uses op- tional causal warm-up (Causal SFT or CFG distillation) followed by DMD, with- out pre-generated ODE trajectories from expensive multi-step teacher inference or sequential self-forcing rollouts. Under noisy-cache inference, noise-aligned his- tory and a fixed block-causal mask make denoising invariant to temporal chunk- ing. The same model therefore supports autoregressive (AR) rollout for low- latency streaming, multi-token prediction (MTP) for parallel denoising of mul- tiple temporal chunks, and single-pass prefill for higher throughput, without re- training. We further combine hybrid local–global temporal attention across layers with attention-sink caching under a fixed KV-cache budget to reduce attention cost and enable long-horizon streaming. Instantiated in our audio-driven avatar system with reference-conditioned generation, the framework empirically supports the equivalence of these decoding schedules and demonstrates stable appearance and scene layout over long-horizon continuation. The resulting model maintains a fa- vorable balance among visual quality, audio–visual synchronization, and identity preservation relative to the bidirectional teacher.

Primary area:
generative models
Keywords:
causal diffusion models, diffusion distillation, rolling-free training, streaming avatar generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SZD7OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a method for real-time, streaming, avatar generation, using a causal, flow-matching based, model. The method takes as input a bidirectional teacher, and converts it to an auto-regressive model, and then distills this model using Distribution Matching Distillation. Special care is taken with respect to the context put into the KV-cache, which allows the method to have flexible chunk sizes in inference. The method shows good quantitative results in comparison with other approaches.

Critical strengths

In terms of quantitative metrics, the proposed method presents good results, outperforming LiveAvatar in particular, a reference method for the avatar streaming setup. The consideration of noise level in the cache context seems like a good issue to highlight, and it is natural that it should have a strong impact on the results. The introduction and motivation of the paper are well-done and clear.

Critical weaknesses

1/ My main criticism of the paper is that it does not clarify the key novelty or important contributions very well. This is due, in part, to heavy use of jargon everywhere which makes the paper difficult to read. While I understand that this is a domain with many different moving parts and techniques in play, and thus a certain amount of jargon is necessary, I felt the paper did not do a good job of highlighting why their method is different and better than other approaches.

2/ Another major criticism of the paper is that there are no video results (ie in video format) that I could see! This is a major drawback, because it is almost impossible to evaluate video generation from still images. In particular, one cannot see the temporal consistency and smoothness, over short or long horizons. I did not see a supplementary material or a link which would show this.

3/ In this same theme, concerning results, it feels like you are over-selling them (Line 372). You state that "DeForcing provides the strongest overall streaming trade-off on both benchmarks: it leads LiveAvatar ...". While this is true, it ignores SoulX-FlashTalk, which has better metrics on 5 out of 10 metrics/situations (counting short and long videos) than DeForcing. Thus I feel that it is a bit disingenuous. The conclusions are not so clear, and since there are no video results to view, it is difficult to tell whether the method is indeed better or not. In terms of qualitative results (compared with other approaches, the only ones which exist are in Figure 6, as far as I can tell. This is far from enough to confirm that the results are better visually.

Smaller notes: 4/ Line 73: Define the acronym Causal SFT

5/ Line 163: "Appendix A.2 gives the full gradient derivation". I don't know why you didn't just include equation 9 in the main paper, since it helps to understand what is done better. And I feel that "derivation" is a bit of an exaggeration here; you just copy the equation written in DMD, which does not really derive the gradient, in the sense that you do not show how the equations are obtained. Maybe think about including equation 9 in the main paper, or replace the phrasing "derivation".

Questions for authors

1/ If I have understood correctly, the core of the work is based on a careful consideration of whether the "history", ie the context which will be put into the KV cache is noised or not. It is presented as a novelty, or it seems to me, but both approaches are done in previous methods (self-forcing, causvid, live-avatar). It was not clear to me what the exact novelty is, can you explain this more clearly ?

2/ Section 2, related work. The Audo-drive avatar generation makes it seem like all other methods are non auto-regressive, when in fact you have at least SoulX-FlashTalk which is indeed Auto-Regressive (if I have understood). Also, you don't mention this paper in the related work, why ? Finally, the related work is very short, as you have missed at least the previous method. Please redo it.

3/ I found the section 3.2.2 extremely difficult to follow. You have added a whole new layer of jargon (AR rollout, prefill inference, MTP inference) for not much gain it seems to me. The people reading the paper are not necessarily experts in LLMs, so why this added jargon. Furthermore, the section is named "Flexible Chunk-Wise Inference" (and indeed this is a core contribution), and it is not said at all how these chunks are defined. If the flexibility is so crucial, shouldn't you discuss it more ? It is barely discussed in this section, and indeed it is difficult to figure out what is important.

4/ Line 201: "Clean- and noisy-cache formulations admit different combinations of causal initialization and DMD teacher choice. Our final pipeline uses noise-aligned causal initialization (Causal SFT or Causal CFG distillation), followed by rolling-free DMD with a bidirectional fake score estimator and frozen bidirectional teacher". It seems that this is at the core of the method, and is what much of the paper is leading up to. But I did not understand why this was novel, or what you are proposing which is central to the success of the paper. You should re-write this section to be much clearer.

5/ Concerning the execution time (Table 2), do you achieve real-time as you state in the title ? This is not clear, and you do not discuss it much. Real-time is 30 frames per second, do you achieve this, and if so, in which configuration ? If this is not the configuration of the results in Table 1, then this needs to be stated, or the results in Table 1 need to be updated. Otherwise, this is disingenuous.

6/ I found the jargon "unit" to be confusing. What you mean here is a latent temporal frame right ? I feel this would be much clearer, even if I understand why you wanted to avoid "frame". I feel "temporal latent frame" or "latent frame" is better.

7/ Line 308: "By the prefill-rollout equivalence established in Section 3.5 ...". This was extremely unclear to me, and seems central to the paper. Section 3.5 is very short, and I am not sure how this "equivalence" is established. It seem more like you just say it is so.

8/ Line 359: "MiniMax H3 Ref2VA". It is not explained what this is in the main paper, there is no way of knowing, and this stops us knowing what DeForcing-H3 is. Please rectify

Reviewer_tJsgOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes DeForcing, a rolling-free training and distillation framework. Built upon the noisy-cache inference paradigm, it renders denoising computation invariant to temporal chunking via noise-aligned history and a fixed block-causal mask. First, causal adaptation of the bidirectional model is performed via Causal SFT or CFG distillation, and the model is then compressed into a few-step generator through asymmetric distribution matching distillation. Combined with hybrid local-global attention and attention sink caching, the same model supports three inference modes without retraining, enabling low-latency streaming avatar generation. Multi-dimensional evaluation is conducted on the SekoBench benchmark, verifying the advantages of the proposed method in generation quality, audio-visual synchronization, and inference efficiency.

Critical strengths
  1. This paper summarizes that under the same-noise framework, different compatible inference paradigms can be adopted to balance computational efficiency and first-chunk latency. This observation holds considerable practical value.
  2. Within the relatively tight submission timeline, the authors conduct experiments on the MiniMax-H3 base model, validating the effectiveness of the proposed method on the latest larger-scale backbone, which is beneficial to the research community.
  3. This paper is well organized and easy to follow.
Critical weaknesses
  1. The main novelty is limited. The DeForcing method proposed in this paper is essentially a subset of Diffusion Forcing in practice. Specifically, Diffusion Forcing adopts a causal mask with independent per-history noise levels, while this work uses a causal mask with uniform historical noise levels, making it a strict subset. Admittedly, I agree that the original Diffusion Forcing may involve unnecessarily complex training schemes. However, the concept of "uniform historical noise levels" has also been proposed in other prior works, such as LiveAvatar (Huang et al., 2025), Ms.Forcing (Li et al., 2026), Self-Aligned Forcing (Wang et al., 2026), and AvatarForcing (Ki et al., 2026).

  2. In addition, I argue that using same-noise history for inference may introduce inter-chunk jitter in generated videos, especially in regions with weak attention weights such as backgrounds. Please supplement tests on cases with dynamic backgrounds and avatars that speak while walking.

  3. Video demos are missing. For tasks such as streaming real-time avatar generation, providing demonstration videos is a basic requirement, as only video demos can most intuitively reveal potential issues with the generation results.

Questions for authors
  1. Elaborate on the actual novelty of this work and add a detailed comparative discussion with prior works.
  2. Discuss the impact of same-noise history on video continuity, and whether it introduces more severe inter-chunk jitter compared to the Self-Forcing method. Please support the conclusion with experimental evidence.
  3. (Not sure if the submission policy allows) Please add video demos, including demonstrations under different scenes, identities and art styles, as well as side-by-side comparison demos with baseline methods.
Reviewer_QkpCOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

DeForcing adapts a bidirectional diffusion model for audio-driven avatars into a causal streaming generator. Its central proposal is a noisy KV cache in which historical and current latents share the same flow time, combined with a fixed block-causal mask. The authors claim this makes denoising invariant to temporal chunking, allowing one checkpoint to run autoregressively, in multi-unit chunks (termed MTP), or as a full-sequence prefill. They use the claimed equivalence to replace temporal autoregressive rollouts in DMD back-simulation with parallel prefill. The system also includes optional causal initialization, asymmetric DMD with a bidirectional teacher, hybrid local/global attention, and attention-sink caching. The authors evaluate Wan2.1-based DeForcing and a MiniMax H3 variant on SekoBench, reporting quality and synchronization metrics, qualitative long-horizon results, and DiT-only inference timings.

Critical strengths

The paper tackles a practical problem: streaming avatars must balance latency, synchronization, identity preservation, and long-term stability. Its proposed single-checkpoint trade-off between lower-latency autoregressive execution and higher-throughput chunked or prefill execution would be useful if the claimed equivalence holds.

The work also presents a fairly complete avatar system rather than an isolated denoising objective, including reference conditioning, audio adaptation, long-horizon continuation, and an H3-backbone transfer. These results make the application relevant, although they do not by themselves isolate the contribution of the proposed training method.

Critical weaknesses

The central partition-invariance claim is not specified or tested rigorously enough to support the paper’s main conclusion. The paper does not fully define what is stored in the cache, how historical latents are synchronized across flow times, or how cache updates and RoPE positions correspond across AR, chunked, and prefill execution. The reported “nearly identical” results are based mainly on representative outputs, without quantitative errors across seeds, partitions, or long-horizon cache eviction. This leaves the key premise difficult to assess.

The incremental contribution over the closest prior work is also not established experimentally. LiveAvatar and TalkingMachines already use noisy, timestep-aligned history; the paper’s distinction is the claimed use of partition invariance for rolling-free DMD. There is no controlled same-backbone comparison that isolates this contribution from causal initialization, teacher choice, or other system components. The quality evaluation is small (100 short and 25 long cases) and has no human assessment or uncertainty estimates. Although the PDF contains appendices and frame-based visualizations, I did not find supplementary generated-video clips in the submitted materials available for review. For a video-generation paper, static frames are not enough to judge temporal coherence, lip-sync quality, flicker, motion naturalness, or long-horizon drift; this makes the qualitative validation particularly difficult to assess. The reported real-time throughput also excludes audio encoding, VAE decoding, transfer, and application overhead.

Questions for authors
  1. Could you specify the exact cache contents and update rule at each flow step, including when and how historical latents are re-noised or otherwise aligned? Please also clarify how the RoPE positions and attention mask are made identical across AR, chunked, and prefill execution. This information would materially affect my assessment of the central claim.

  2. Do you have quantitative latent- or pixel-space differences across multiple seeds, examples, chunk partitions, sequence lengths, and sampling schedules, including with rolling-cache eviction? Such results could establish whether the claimed equivalence holds beyond the few representative cases shown.

  3. The submitted PDF includes appendices and static frame visualizations, but I did not find supplementary generated-video clips among the materials available for review. Do you have video samples for the short and long benchmarks, including representative failures and long-horizon segments? In this setting, the actual clips are important for assessing temporal coherence, lip-sync, flicker, motion naturalness, and drift; their absence materially limits my evaluation of the qualitative claims.

  4. Do you have a matched comparison against a LiveAvatar-style timestep-forcing setup or Self-Forcing, using the same backbone, data, and compute budget? A controlled comparison, including training time or GPU-hours, could change my assessment of novelty and practical benefit.

  5. Does the reported 1.182× real-time result include audio encoding, VAE decoding, and data transfer, or is it limited to DiT rollout as stated in the appendix? If end-to-end measurements are available, please report them; this would affect the scope of the efficiency claim.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.