- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Weave Forcing proposes a training-free compositional memory routing framework aimed at improving cross-shot character and background consistency in interactive long video generation. The method first uses an LLM to decompose the current shot prompt into semantic components, such as characters and backgrounds, and retrieves the corresponding historical shots for each component. It then employs Contrastive Slot Attention to construct Semantic KV Masks, selectively combining relevant KV memories from different historical shots while suppressing irrelevant visual information. Additionally, the authors introduce Coverage Adaptive RoPE, which adjusts temporal position offsets and memory retention strategies according to the availability of historical references, balancing the preservation of historical content with the generation of new content.
Critical strengths
-
The paper presents a clear and well-motivated problem formulation. It identifies the limitations of existing historical memory retrieval strategies in compositional multi-shot video generation, where retrieving a single historical shot may miss relevant components, while reusing multiple complete shots may introduce irrelevant visual information. This motivation is clearly explained and supported by intuitive examples.
-
The proposed framework operates without additional model training and can be integrated into existing autoregressive video generation architectures through semantic memory routing, KV-cache masking, and temporal position adjustment.
Critical weaknesses
-
Recent video generation systems, such as Seedance 2.0 and MiniMax H3, already support multi-shot video generation using explicit character and scene references, along with previously generated video content. Such workflows can maintain character identity and scene consistency across shots through explicit visual references, without requiring semantic retrieval and recombination of historical KV caches. In contrast, Weave Forcing introduces a relatively complex historical memory retrieval and fusion mechanism to address a similar compositional consistency problem. However, the paper does not sufficiently justify the necessity of adopting a KV cache-based approach, nor does it demonstrate unique advantages over existing reference-conditioned video generation workflows in terms of generation quality, efficiency, or controllability. Consequently, the practical motivation and applicability of the proposed framework remain insufficiently justified.
-
The proposed LLM-based Semantic Memory Router primarily relies on textual descriptions of historical shots for retrieval, without verifying whether the corresponding characters or scenes were actually generated in those videos. Video generation models do not always follow textual instructions accurately. If a historical shot fails to generate a specified character or omits key visual elements, its KV cache may lack the required visual information, even if the router correctly identifies the corresponding shot based on its textual description. Therefore, correct retrieval at the textual level does not necessarily guarantee valid visual references, potentially leading to the repeated reuse of incorrect or incomplete historical memories in subsequent video generation.
-
Weave Forcing selects only one historical shot for each semantic slot, rather than aggregating all relevant historical shots, but this design choice is not adequately justified. Different historical shots featuring the same character may provide complementary visual information, such as the character's appearance from different viewpoints and in different poses. The current strategy may therefore discard valuable information, and the paper does not investigate whether aggregating multiple relevant historical memories could further improve cross-shot consistency.
-
Attention maps do not necessarily correspond to accurate semantic segmentation, especially when multiple subjects occlude one another, when characters and backgrounds are semantically correlated, or when small objects occupy only a few tokens. Inaccurate semantic masks may lead to the mixing of attributes across different characters. In fact, the authors demonstrate such a failure case in Figure 9 of Appendix E, where clothing attributes are incorrectly transferred between different characters.
-
Although Semantic KV Masking as a key innovation, the experimental results show that its contribution to subject consistency is relatively limited, while the most substantial improvement comes from semantic-level historical memory retrieval. This suggests that the proposed framework relies heavily on the external LLM for effective memory routing. Furthermore, the paper does not provide a dedicated ablation study to independently evaluate the contribution of Coverage Adaptive RoPE.
Questions for authors
The paper proposes Coverage Adaptive RoPE. Specifically, when no historical references or only partial historical references are available, the authors fix the temporal RoPE offset at . When both character and background references are available, the offset is determined based on prompt embedding similarity within the range . However, the paper does not sufficiently justify the choice of or explain why the range is selected for full coverage. It also lacks sensitivity analyses comparing different fixed offsets and adaptive offsets. Furthermore, since the paper simultaneously adjusts the retention duration of historical KV memories, the existing experiments make it difficult to independently assess the actual contribution of Coverage Adaptive RoPE. Please see weakness for other questions.
4 comments
Q5. Contribution of Semantic KV Masking and dependence on the LLM
We agree that, in the current quantitative results, removing semantic separation has a smaller effect on subject consistency than removing LLM routing. The two components serve different purposes: LLM routing determines which historical shot to retrieve, whereas the semantic KV mask determines which tokens within that shot can be reused. The component ablations in Section 5.3 show:
| Setting | Inter-shot subject | Inter-shot background | Text alignment |
|---|---|---|---|
| Without LLM routing | 0.442 | 0.734 | 0.230 |
| Without semantic separation | 0.501 | 0.778 | 0.220 |
| Full method | 0.513 | 0.811 | 0.237 |
Figure 5 further compares direct concatenation (Concat), weighted fusion, and semantic KV masking. Direct concatenation is more prone to introducing irrelevant background content, whereas semantic masking better isolates the requested people and scene. Quantitative metrics may not fully capture such localized visual differences, so the numerical changes should be considered alongside the qualitative results. Together, the ablation scores and visual comparisons support the effectiveness of semantic KV masking.
Q6. Offset settings in Coverage-adaptive RoPE
When neither the people nor the background have historical references, or when only part of the requested content has a reference, the available history may not fully match the current instruction. We therefore use a larger temporal offset, J = 45, to limit interference from mismatched content. When both types of references are available, we set the offset within [1, 6] according to prompt-embedding similarity s:
J = round(1 + 5 × clip((0.99 − s) / (0.99 − 0.5), 0, 1))
A higher similarity places the relevant historical reference closer to the current queries. To evaluate this design, we change only the offset setting while keeping routing, semantic masking, memory decay, and eviction unchanged:
| Setting | Intra-shot subject | Intra-shot background | Inter-shot subject | Inter-shot background | Text alignment |
|---|---|---|---|---|---|
| Uniformly large offset | 0.776 | 0.928 | 0.469 | 0.745 | 0.226 |
| Coverage-adaptive RoPE | 0.771 | 0.923 | 0.513 | 0.811 | 0.237 |
A uniformly large offset reduces inter-shot subject and background consistency. A uniformly small offset causes visual drift and unnaturally saturated colors in the no-coverage and partial-coverage cases, so these visibly unstable settings are excluded from the quantitative table. These experiments demonstrate the effectiveness of our Coverage-adaptive RoPE design. The results have been added to Section 5.3, Ablation Study.
Thank you to the authors for addressing all my questions. My concerns have been fully resolved, and I am therefore willing to raise my score to 3.
Dear Reviewer PCSx,
Thank you for taking the time to read our rebuttal. We are glad that our responses addressed your concerns, and we greatly appreciate your updated score.
Best regards,
The Authors
We thank the reviewer for the comments and respond to each point below.
Q1. Comparison with reference-conditioned video workflows
We use Self Forcing as an autoregressive baseline and build on its generation framework. We acknowledge that our method may not match the overall visual quality of recent reference-conditioned systems such as Seedance 2.0 and MiniMax H3. Its advantage lies in a different operating setting: the generator achieves 15.42 FPS in our evaluation (Appendix A.4), while autoregressive inference and KV-cache updates support continuous long-video generation.
Building on this efficient foundation, Weave Forcing improves cross-shot consistency in interactive long-video generation. For each new instruction, it selects historical KV memories for the requested people and background and masks unrelated content. This provides control over which earlier elements are recalled and combined in a new shot, while retaining the throughput and continuous-generation capability of the autoregressive backbone.
Q2. Risk of a mismatch between text-based routing and generated content
Our shot-level prompts explicitly introduce the people and scenes that later shots may need to recall. We also provide detailed instructions to the LLM for extracting semantic slots and selecting matching historical sources. In our evaluated cases, the LLM rarely misinterpreted these instructions, and we did not observe a missing target in a historical shot being repeatedly retrieved and reused incorrectly. CSA subsequently identifies relevant regions in the selected shot’s actual historical KV memory.
We agree that, if a historical shot fails to depict a target element, its KV memory may be an unreliable reference. Therefore, we designed detailed LLM prompts that specify how to identify the requested character and background slots, match them to earlier shots, and select the corresponding historical references, thereby reducing errors in text-based memory routing.
Q3. Why each semantic slot selects one historical shot
Selecting one source for each semantic slot makes its historical reference explicit and limits cache size and potential conflicts between appearances from different viewpoints or poses. Different slots can still select different historical shots, allowing the current shot to combine people and backgrounds introduced separately. We agree that multiple views of the same person may provide complementary information. Aggregating several historical memories for one slot is a worthwhile extension for future work.
Q4. Attention masks and multi-person attribute-binding errors
Contrastive Slot Attention (CSA) subtracts the attention response to non-target content from the response to the target slot to identify the semantic region to retain. Morphological opening and closing remove isolated responses and improve the mask’s spatial coherence. Nevertheless, attention maps do not guarantee accurate instance separation, particularly when people overlap, subjects and backgrounds are semantically related, or a small object occupies only a few tokens.
Figure 12 in Appendix E illustrates this limitation through a clothing-attribute transfer error. The failure has two coupled causes: an imperfect mask may retain tokens from an adjacent person, leaving their historical appearances insufficiently separated; and a prompt involving several people and attributes can create competition for attention, causing an attribute to be assigned to the wrong person. More discriminative instance-level masks and explicit multi-person attribute binding are promising directions for addressing this failure mode.