acceptodds
Withdrawn from ICLR 2027

CineForcing: Streaming Multi-Shot Audiovisual Generation for Long Storytelling

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Humans experience the world as a continuous, synchronized stream of sound and vision. While modern generative models excel at synthesizing brief audiovisual clips, they struggle to sustain this continuity over long horizons. During extended generation or scene transitions, models suffer from identity drift: characters alter their appearance and voices lose their distinctive timbre. Tracking visual frames is insufficient, since acoustic identity cannot be recovered from pixels alone. We present **CineForcing**, a memory-conditioned streaming framework for long-horizon audiovisual generation. Instead of carrying an ever-growing buffer of raw tokens, CineForcing summarizes completed shots as compact committed memory states that pair a visual anchor with speech-filtered acoustic evidence from the same shot. Student generation is causal at two timescales: audiovisual blocks are generated sequentially within a shot, while each new shot reads only state committed by completed shots. Memory-conditioned supervised fine-tuning initializes the backbone; parallel audiovisual teacher forcing and two-pass distribution matching then train causal streaming with ground-truth and self-generated *within-shot* prefixes, respectively, under strict-past audiovisual memory. We evaluate CineForcing on 30-shot long-story audiovisual rollouts. On 100 stories comprising 3,000 shots, CineForcing achieves the highest Voice and CAVP scores and the lowest audiovisual desynchronization among the compared methods. It improves SpeechAcc from **0.4224 to 0.7583** over OmniForcing and, with four sampling steps, achieves a measured generation latency of **4.557 seconds per clip at 480p**. A matched inference-time ablation shows that audiovisual memory increases SpeechAcc from **0.0037** with both memory inputs zeroed to **0.7583**, while achieving the strongest late-shot visual consistency and synchronization among memory variants. Video demos: [cineforcing-demos.pages.dev](https://cineforcing-demos.pages.dev/).

Primary area:
generative models
Keywords:
streaming audiovisual generation, long-horizon generation, multi-shot consistency, audiovisual memory, diffusion distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_89KpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CineForcing, a memory-conditioned streaming framework for character-identity-preserved, multi-shot video generation with audio. The core idea is a simple FIFO-based memory design for both video and audio, integrated into a CausalForcing- or LongLive2-style distillation framework (teacher-forcing init + asymmetric DMD training). Experimental results show that CineForcing achieves comparable scores for visual/audio quality and identity preservation with lower latency (although some scores are lower than the baselines).

Critical strengths

To the best of the reviewer’s knowledge, this paper appears to be the first to introduce memory into a few-step causal flow-matching model for multi-shot joint audio-video generation.

Critical weaknesses
  • The discussion of existing memory-augmented video generation models is limited, which weakens the paper’s positioning. Some existing works (e.g., Corgi [1], FilmWeaver [2], OneStory, ShotStream, StoryMem) discuss how they select keyframes, but this paper does not provide a detailed comparison of their memory designs. This makes the novelty of the proposed memory design unclear. Furthermore, quantitative results for visual identity preservation are mixed: ShotStream and StoryMem tend to perform better on ViCLIP and Self-CIDS. These results do not sufficiently demonstrate the advantage of the proposed memory design in preserving character identity.

  • The proposed method focuses on multi-shot long-video generation (i.e., cinematic creation), but the proposed memory mechanism supports only one character. This limits its applicability to scenarios involving multiple characters within the stated application domain. It is also unclear whether the proposed training framework can be applied to a multi-character setup, as the proposed rollout strategy could become computationally heavy as the number of characters grows. These limitations create a gap between the paper’s application goals and the proposed method’s functionality.

[1] Corgi: Cached Memory Guided Video Generation, WACV 2025
[2] FilmWeaver: Weaving Consistent Multi-Shot Videos with Cache-Guided Autoregressive Diffusion, AAAI 2026

Questions for authors
  • Is it enough to support only one character for interactive cinematic generation, given the stated multi-shot setting?
  • Can the authors clarify the details of the proposed memory design and compare it with existing work?
Reviewer_fTYVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

CineForcing addresses audiovisual consistency in multi-shot story generation by separating local causal history from memory of completed shots. Visual anchors and speech-filtered audio are retained in a bounded memory bank, where the local K/V cache is reset at each shot. The proposed model is trained based on memory-conditioned SFT, causal teacher forcing, and self-rollout DMD. The experiments demonstrate improvements in dialogue accuracy, audiovisual synchronization, and voice continuity. Visual performance is less consistent, and human evaluation favors video-only memory for appearance.

Critical strengths

Compared with OmniForcing, CineForcing improves dialogue accuracy and audiovisual synchronization. The memory comparison also shows a benefit from retaining speech information across shots. Human judgments favor audiovisual memory for voice consistency, although video-only memory is preferred for character appearance.

Critical weaknesses
  1. The Related Work describes Duan et al. (2026) as using shot-paired visual and speech memory with causal teacher forcing and rollout distillation, which is essentially the CineForcing recipe, but the paper never compares against it. The only stated difference from the Self Gradient Forcing line is that historical K/V is detached (Sec. 3.3, Fig. 3b). The paper should explain what detaching historical K/V and the memory-state interface change relative to these two works.
  2. Table 3 shows how the trained model depends on visual and audio memory by zeroing the corresponding inputs at inference. This is a useful comparison, but it does not separate the contributions of the training stages. Comparing matched teacher-forcing and DMD checkpoints would clarify the benefit of self-rollout distillation, while separately trained memory variants would help assess the contribution of memory conditioning.
Questions for authors
  1. Is the video-only memory system in Fig. 5 and Fig. 6 a separately trained model or the zeroed-input intervention of Table 3? Please also report variance over seeds for late-shot ViCLIP, since the late-shot consistency claim rests on a very small gap.
  2. Do the timing results in Table 2 include decoding, speech separation, and memory encoding? Please specify the hardware and report the delay before the first output, including any additional delay at shot boundaries. Sec. 4.3 notes that full-clip latency differs from time to first chunk but does not report the latter.
Reviewer_BGVDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies appearance consistency, voice consistency, and audiovisual synchronization in streaming multi-shot generation. CineForcing separates within-shot causal K/V state from committed cross-shot audiovisual memory. Each memory item pairs one visual frame with the full separated speech track from the same completed shot. Training proceeds through Memory SFT, parallel audiovisual teacher forcing, and two-pass self-rollout DMD. At inference, the memory retains the first three shots and the four most recent non-sink shots. On 100 stories comprising 3,000 shots, the method improves several consistency, alignment, and dialogue metrics over OmniForcing, while reporting throughput above playback speed at 480p.

Critical strengths
  1. The state interface and causal implementation are concrete and coherent. The local cache is updated within shots and reset between shots, audiovisual evidence is committed only at shot completion, and the two-pass recovery explicitly detaches historical K/V gradients. This provides an implementable design for maintaining identity across cuts and makes training-time information visibility auditable. It is a central technical strength of the paper (Section 3 and Appendix F).

  2. The gains are supported by both automatic metrics and human evaluation, with their costs also reported. Relative to OmniForcing, SpeechAcc is 0.7583 versus 0.4224 and DeSync is 0.2627 versus 0.6275. The method receives 78% of the same-voice preferences in the human evaluation. The authors also report that video-only memory receives more appearance preferences (48% versus 33%). LTX-2.3 achieves a SpeechAcc of 0.8300, and StoryMem has higher visual consistency. The paper addresses the overall tradeoff in streaming generation. These results provide meaningful evidence for acoustic continuity and synchronization (Table 1 and Figure 5).

Critical weaknesses
  1. The distinctions from, and advantages over, the closest audiovisual memory methods have not been established. According to Section 2, Ripple uses cross-modal recurrent memory, while UnityShots uses visual memory and speaker-reference conditioning. The section describes Duan et al. (2026) as addressing shot-paired visual/speech memory and causal training. The audiovisual baselines in the main quantitative comparison are LTX-2.3 and OmniForcing, which generates shots independently; the other two methods generate video only. These comparisons support the reported improvements over the listed baselines. They do not establish how much methodological progress the proposed state interface and training pipeline offer beyond existing cross-shot audiovisual conditioning methods. Clarifying the mechanistic differences from these close neighbors is necessary and directly affects my assessment of the contribution and recommendation.

  2. Matched controls are missing for the key components, making the necessity of the full pipeline difficult to assess. Table 3 zeros inference inputs to the same trained checkpoint, and Appendix A clearly states that slots and positions are retained. This intervention effectively measures how the trained checkpoint depends on its memory inputs. It does not disentangle the training contributions of Memory SFT, speech memory, and two-pass DMD. The SpeechAcc of 0.0037 when both inputs are zeroed may also reflect fragility under an unusual conditioning regime. This intervention does not measure the performance of a memory-free system trained under matched conditions. The paper currently lacks stagewise comparisons with the backbone, data, and budget held fixed, as well as a control comparing separate visual/speech references with same-shot paired memory. It is therefore unclear whether the gains arise primarily from the proposed pipeline or from adding speech conditioning. Additional experiments could address this gap, which materially affects the central contribution.

Questions for authors
  1. Please clarify the additional mechanisms and scope relative to Duan et al. (2026), Ripple, and UnityShots. Which components are introduced here, and which are inherited from prior methods? If these close neighbors cannot be applied directly to the current setting, what are the specific obstacles?

  2. With the backbone, training data, sampling steps, and a comparable training budget held fixed, could you provide a stagewise comparison? Please compare Memory SFT alone, the addition of teacher forcing, and the further addition of two-pass DMD. This would directly affect my assessment of whether the complete training pipeline is necessary.

  3. Could you provide a simple baseline using visual and speech references trained under matched conditions, or a control that shuffles visual/speech pairing across shots? Please report adherence to new dialogue, identity consistency, and visual quality together to test the value of the paired state itself.

  4. Voice uses whole-shot audio and may be influenced by background sounds and speaker changes. Could you report per-speaker results on a subset that keeps the character fixed while varying the background and dialogue, with uncertainty quantified at the story level? The repeated choices by 10 participants over the same 100 sets should account for the corresponding dependence structure. If the data cannot be released, providing this breakdown and aggregate statistics would still help assess the conclusions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.