- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ATAS, a world-state-centric framework for long-form video generation. It maintains separate scene and event memories for static 3D information and temporally indexed dynamic observations, respectively, and uses state-update validation to control which generated observations are incorporated into persistent memory. The authors evaluate the framework through benchmark experiments, spatial revisiting, event recapture, ablations, and human evaluations across multiple video-generation backbones. While the problem is interesting, the writing and logical flow significantly hinder the understanding of the method and its main contributions.
Critical strengths
-
The paper addresses an interesting problem in long-form video generation. Separating static scene memory from temporal event memory is intuitive and reasonably motivated.
-
Separating video delivery from state updates is a useful design choice for preventing unreliable observations from contaminating subsequent generations.
-
The experimental evaluation covers multiple video-generation backbones, spatial revisiting, event recapture, ablations, and human evaluations.
Critical weaknesses
-
The writing and logical flow are poor, significantly affecting readability. The motivation, method design, and claimed contributions are not presented clearly enough. The title emphasizes the term world-state-centric, but the abstract does not explicitly define this term or explain how it distinguishes ATAS from conventional video-memory frameworks. Readers must infer the central concept from the subsequent introduction.
-
The figure captions do not adequately explain the figures. For example, Figure 1 contains multiple video frames, a scene representation, and colored connections, but its caption mainly restates the high-level idea of the paper without explaining the visual elements or how they relate to the proposed pipeline. This significantly reduces the usefulness of the figure. The paper would benefit from a clearer narrative connecting the problem formulation, design choices, and overall workflow.
-
The novelty of ATAS compared with existing video-memory and 3D scene-memory methods is insufficiently established. The individual contributions of event indexing and validation-controlled state updates remain unclear.
-
Important implementation details and evaluation protocols need clarification, including the models used for reconstruction and registration, the state-update rules, the definition of the reported aggregate score, and the statistical uncertainty of the results.
Questions for authors
-
What exactly does world-state-centric mean operationally, and how does it distinguish ATAS from existing persistent-memory frameworks?
-
Can the authors improve the figures and their captions to explain the visual elements and the information flow through the framework?
-
Can the authors clarify the implementation details and evaluation protocols to make the method and reported results easier to understand and reproduce?
6 comments
Response to Reviewer h7uk (2/3)
W3. Novelty and component contributions
We separate the agentic framework from the contributions of event indexing and validated memory updates.
1. Novelty. Our contribution is an agentic workflow that connects shot planning with registered scene geometry, retrieval of specific past events, and validated memory updates.
| Method | Agentic planning | Explicit 3D/4D geometry | Geometric camera control | Dynamic reshooting | Training-free integration |
|---|---|---|---|---|---|
| OneStory | ✗ | ✗ | ✗ | ✗ | ✗ |
| StoryMem | ✗ | ✗ | ✗ | ✗ | ✗ |
| VideoMemory | ✓ | ✗ | ✗ | ✗ | ✓ |
| SPMem | ✗ | ✓ | ✓ | ✗ | ✗ |
| Spatia | ✗ | ✓ | ✓ | ✗ | ✗ |
| GEN3C | ✗ | ✓ | ✓ | ✓ | ✗ |
| Vista4D | ✗ | ✓ | ✓ | ✓ | ✗ |
| ATAS | ✓ | ✓ | ✓ | ✓ | ✓ |
Training-free integration means using existing generators without method-specific fine-tuning. GEN3C and Vista4D already support dynamic reshooting. ATAS connects this capability to agentic planning, retrieval of specific past events, and validated memory updates across nonconsecutive shots. Recapture uses the scene geometry valid at the requested event time. The capability comparison is also provided in Appendix Table F.
2. Component contributions. We evaluate event indexing and validated memory updates with controlled Wan 3.0 ablations.
Event indexing. Adding indexed event memory to updated scene memory raises C-Top10 from 0.658 to 0.703 and Event score from 0.585 to 0.728, while LPIPS_C changes from 0.286 to 0.285 in Table 8. Below, we compare retrieval of similar events from the same scene with the RGB-D history, renderer, and reference budget fixed. Most recent selects the latest event, Semantic similarity matches by content, and ATAS uses the requested event identifier and time interval. The two accuracy metrics measure correct event and interval selection.
| Retrieval rule | Event retrieval accuracy (%) ↑ | Interval retrieval accuracy (%) ↑ | Event score ↑ |
|---|---|---|---|
| Most recent event | 47.3 | 29.6 | 0.683 |
| Semantic similarity | 84.1 | 79.8 | 0.722 |
| Event and time indexing (ATAS) | 100.0 | 100.0 | 0.737 |
Event and time indexing performs best on all three metrics in Table 5, retrieving the requested event and interval even when several events look similar.
Validated memory updates. Memory updates require two checks. Observation validation checks geometric alignment against held-out historical frames and static correspondences. The state-update check verifies that the reconstruction comes from the reviewed clip, belongs to the requested scene, and uses the current memory version. A clip can enter the film without passing these memory checks. The base row uses recursive compilation without either check. The next two rows add these checks cumulatively under identical requests, cameras, candidate budgets, and retry policies.
| Configuration | C-Top10 ↑ | LPIPS_C ↓ | Event score ↑ |
|---|---|---|---|
| Base pipeline with recursive compilation | 0.684 | 0.288 | 0.711 |
| + observation validation | 0.702 | 0.286 | 0.726 |
| + state-update check (full ATAS) | 0.719 | 0.284 | 0.739 |
Table 7 shows that each validation stage improves all three metrics, and full ATAS performs best, supporting validation before generated observations are reused in memory.
(see next comments for more response, thanks)
Response to Reviewer h7uk (3/3)
W4. Implementation details and evaluation protocols
Thank you for the suggestion. We have clarified the implementation in Secs. 3.3 and 3.5 (lines 252–263 and 283–304) and Appendix C, and the evaluation protocols in Appendix A.6–A.10.
Reconstruction and registration. Pi3X estimates depth and camera poses, and SAM 3 masks dynamic regions. Previously registered frames provide depth, camera poses, and camera intrinsics to constrain reconstruction. We align the result to the scene coordinate frame and validate it on held-out historical frames.
Memory updates. The planner marks regions expected to change before generation. Validated observations update these regions while preserving earlier geometry for historical queries. Ongoing motion is stored in event memory. Before each update, we also check that the reconstruction comes from the reviewed clip, belongs to the requested scene, and uses the current memory version.
Score aggregation. Avg. is the unweighted mean of VBench, ST-Bench, ViStory-Self, and ViStory-Cross. ST-Bench averages Aesthetic, PF-G, PF-S, C-All, and C-Top10. Event Recapture Score, abbreviated as Event score, averages five normalized scores for event identity, action and state preservation, temporal order, character and object consistency, and viewpoint realization within each request, then averages across requests.
Statistical uncertainty. The ± values denote standard errors, calculated as the sample standard deviation of evaluation scores divided by the square root of the total number of evaluated samples. We now state this computation in Appendix A.6 (lines 995–998) and identify the uncertainty as standard error in Table 1.
Q3. Implementation and evaluation reproducibility
For implementation details, score aggregation, and statistical uncertainty, please see our response to W4. Comparisons within each backbone share the generator, prompt package, duration budget, and evaluation protocol. Please also refer to Secs. 3.3 and 3.5, Appendix A.6–A.10, and Appendix C in the revised PDF.
We thank you again for your constructive feedback on presentation and methodology. We hope the clearer definition, redrawn teaser, revised captions, component analyses, and implementation details make the agentic contribution and supporting evidence easier to assess. We welcome any further comments or suggestions.
Response to Reviewer h7uk (1/3)
We thank Reviewer h7uk for the careful evaluation and constructive suggestions. We appreciate your recognition of the memory design as “intuitive and reasonably motivated” and of separating video delivery from state updates as “a useful design choice.” We have carefully considered all comments and provide our point-by-point responses below.
Thank you for the feedback. We have revised the abstract, introduction, and method description to make the central concept explicit and clarify the connection between our motivation, design choices, and contributions.
Core definition. The world state is the persistent record held in scene memory and event memory. It contains current and historical 3D scene geometry with validity intervals, and dynamic observations indexed by scene, event, and time. Agents query this record before generating shots and update it only with validated observations.
Motivation and flow. Scene revisits require shared geometry. Historical recapture requires selecting an earlier event and the scene geometry valid at that time. Validation determines which new observations can update memory.
We have made the following specific revisions.
We have redrawn Fig. 1 and rewritten the captions of Figs. 1 and 2 (lines 22–28 and 170–175) to explain their visual elements and information flow.
Fig. 1. The redrawn teaser shows Shot 1 and Shot 2 alongside the updated world state. Orange silhouettes indicate different times of the walking event, and camera icons identify the queried viewpoints. The two bottom panels separate event recapture and scene revisit, each showing the geometric projection and the generated sequence.
Fig. 2. The caption explains the sequence from shot planning and memory retrieval to rendering, generation, and memory updates. It also clarifies that a clip can enter the film even when its reconstruction is rejected.
World-state-centric means that agents plan and generate shots by querying and updating a shared persistent record of registered 3D scene geometry and past events, stored in scene memory and event memory.
Operational meaning. Agents turn screenplay instructions into requests specifying the scene location, target camera, and, when needed, a historical event interval. To replay an earlier action, they retrieve the matching event observations and scene geometry valid at that time, then render both from the requested camera to guide generation. Only validated observations update memory.
Distinction from existing frameworks. Compared with entity-reference memory in VideoMemory, ATAS provides geometric camera control and retrieval of specific past events and time intervals. Compared with Spatia and SPMem, it uses an agentic workflow to turn screenplay instructions into memory queries, rendering, generation, and validated updates. We clarify this distinction in the abstract (lines 36–43), Introduction (lines 71–88), and Sec. 3.1.
We have redrawn Fig. 1 and rewritten the captions of Figs. 1 and 2 to explain their visual elements and information flow. Please see our response to W2 and the corresponding figures and captions in the revised PDF.
(see next comments for more response, thanks)