acceptodds
Under review as a conference paper at ICLR 2027

SeerWorld: Towards Out-of-Sight World Evolution in Generative Video Models

Abstract

Interactive video world models offer a promising route toward world simulation by enabling camera-controlled exploration of evolving visual environments, while aiming to maintain consistency across changing viewpoints. Faithful world simulation, however, requires more than preserving what has previously been observed: entities should remain participants in the world's dynamics, interacting with events and changing in response even outside the observer's field of view. Current models struggle throughout this process: a mid-rollout prompt switch may fail to instantiate the new event; when the camera later revisits an earlier region, the subject and background may drift or disappear; and even when the subject is preserved, it often remains behaviorally static rather than reacting to or interacting with the event, making the resulting world evolution behaviorally and causally implausible. We formalize this challenge as Out-of-Sight Reactive Evolution and introduce the SeerWorld-Data along with SeerWorld-Bench. As a baseline reference solution, we develop SeerWorld, a segment-autoregressive video model trained on our proposed data, together with a reasoning-augmented generation framework, which autonomously reasons about how the off-screen world evolves and renders the resulting dynamics. Evaluation results show that our approach outperforms recent state-of-the-art models in event-conditioned interaction rendering while preserving revisit consistency, establishing a reference point for autonomous world evolution beyond the current field of view. The model and data will be made publicly available.

Primary area:
generative models
Keywords:
Out of Sight Dynamics, Video World Model, Reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_FjDcOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper tries to solve a problem, out-of-sight world evolution. When a subject is out of the camera view, it should still take part in and respond to the changes of the world. The paper proposes a full-stack pipeline spanning dataset curation, model architecture design, model training, and benchmark evaluation. SeerWorld mainly consists of a vision-language model (Qwen3.8-27B) and an autoregressive video generation model (LTX-2.5). The VLM serves to maintain world memory and infer plausible actions of out-of-sight subjects. The video generator is fine-tuned in two stages to learn reference anchors and reduce training-inference gaps for autoregressive generation.

Critical strengths

(a) The paper identifies the "out-of-sight world evolution" problem in current video world models and proposes a full-stack pipeline covering dataset construction, model design and training, and benchmark evaluation to solve the problem. Although the task setting is simplified in this paper, the dataset-model-evaluation paradigm is complete and reasonable. The dataset construction pipeline is inspiring. Leveraging powerful foundation generative models to obtain ground-truth data and performing multi-stage extraction and checking are useful to solve data scarcity problems in various controllable generation tasks.

(b) The quantitative and qualitative results, together with the video demos, verify the effectiveness of the proposed method.

Critical weaknesses

(a) Multi-turn and long-horizon video generation results are insufficient. In the "Limitations and Future Work" section, the paper recognizes that it currently focuses on a single event followed by a single subject reaction, while leaving long-horizon worlds in which multiple events interact, earlier reactions alter later conditions, or several entities respond over time, underexplored. It is reasonable to leave such more complex scenarios in future. However, the ability to generate multiple turns of "a single event followed by a single subject reaction" remains unclear in this paper. For example, in the "demo3_game.mp4" of supplementary materials, the player sees the enemy, then holds her sword. In the autoregressive manner, the enemy can also respond to the player's action as the next-turn out-of-sight evolution. However, the paper does not seem to show such multi-turn generation capabilities.

Questions for authors

(a) See Critical Weaknesses (a). Multi-turn and long-horizon generation are important in video world models. How long and faithful can the framework generate videos without obvious visual drift and unreasonable evolution?

(b) In the "Limitations and Future Work" section, the paper recognizes more complex camera trajectories, multi-event, and multi-reaction evolution as future directions. I am curious about the extensibility of the current framework. For example, the dataset construction pipeline requires a powerful commercial video generation model (e.g., Seedance 2.5) to create ground-truth video data. As the instructions (e.g., camera motions, number of entities, number of events) become more complicated, can the ground-truth data still be obtained with an acceptable success ratio and costs? Can the data annotation and checking pipeline be extended feasibly to annotate such complex videos?

6 comments
AuthorsOpenReview ↗

Response to Weakness (a)

Thank you for the valuable feedback, here are our responses to all the concerns (weaknesses and questions).

Weakness (a): Multi-turn generation

Thank you for the comment. As acknowledged in our limitations, this paper primarily studies a single out-of-sight event followed by one subject reaction, rather than explicitly focusing on extended multi-turn evolution. Nevertheless, the autoregressive design of SeerWorld permits the generated state to be propagated into subsequent turns. To directly evaluate this capability, we now conduct an additional multi-turn evaluation over all 120 SeerWorld-Bench cases:

subject → event → reaction → same event evolves → next reaction evolution

This evaluation directly covers the type of suggested continuation: after the subject reacts to an event, the camera revisits the same event entity and observes its subsequent evolution, before returning to the subject for another reaction.

Here is an example: In the game domain, the camera pans away from a marsh ranger and reveals a spectral knight materializing from the mist on the opposite bank. When the camera returns, the ranger has risen from her crouched pose and drawn her sword into a defensive stance. The camera then revisits the same spectral knight, which preserves its identity, raises its sword, and advances in response. On the next return, the ranger retains her drawn-sword state, steps behind a nearby tree, and lowers into a defensive crouch rather than resetting to her initial pose.

These multi-turn autoregressive videos contain 465 frames, corresponding to approximately 19.4 seconds at 24 fps. We use the same checkpoint as the full SeerWorld method reported in the paper, without any additional training specifically for this multi-turn experiment. Importantly, each later event or subject-reaction segment is conditioned on its corresponding previously generated state rather than the initial observation. The event therefore evolves from its previously generated event state, while the later subject reaction evolves from the previously generated post-reaction subject state. We evaluate the consistency and the casual plausibility. The main results are as follows:

Multi-turn metricSeerWorld
Multi-turn subject revisit success95.8%
Subject identity consistency87.5%
Multi-turn event86.7%
Multi-turn causal plausibility81.7%
Multi-turn reaction quality75.8%

These results show that SeerWorld can recover a previously observed subject across repeated revisits while largely preserving its identity and generating a causally appropriate subsequent reaction. In particular, the identity consistency shows that the subject is usually recognizable and consistent after multiple camera movements and state changes. The multi-turn event result shows that the event can usually evolve successfully in the next turn. The 81.7% multi-turn causal plausibility further demonstrates that later behavior can remain consistent with the preceding event and accumulated state, rather than simply reproducing the initial reaction.

Notably, despite the more difficult multi-turn setting, SeerWorld’s multi-turn causal plausibility remains higher than the single-turn causal-plausibility results of all baseline methods reported in the main paper (e.g., 32.9% causal plausibility for Evoke). We believe that these results demonstrate the multi-turn capability of SeerWorld to a certain degree. We will add these quantitative results and related discussion to the final version.

AuthorsOpenReview ↗

Response to Questions (a) & (b)

Question (a): Multi-turn generation question

Thanks for the question. In the additional experiment above, we evaluate multi-turn generation totaling 465 frames, or approximately 19.4 seconds at 24 fps. Within this tested horizon, SeerWorld achieves 95.8% multi-turn subject revisit success, 87.5% subject identity consistency, and 81.7% multi-turn causal plausibility. Across the complete rollout, it additionally obtains 89.2% background consistency.

These results indicate that the generated world can remain meaningfully faithful over several alternating subject–event revisits: the subject and background are generally preserved, and the later reaction usually remains causally consistent with the preceding event evolution. However, we do not claim drift-free generation over arbitrarily long horizons. Compared with the original single-reaction evaluation, both the causal plausibility and reaction quality decrease. This degradation reflects the accumulation of appearance, motion, and state errors across autoregressive turns.

Therefore, the current framework demonstrates meaningful multi-turn capability over the evaluated five-segment, around 20-second horizon. However, its reliability gradually weakens as additional states and transitions are introduced. In our qualitative long-horizon stress test, rollouts beyond approximately 30 seconds begin to exhibit noticeable visual drift and less accurate state evolution, which we attribute primarily to the accumulation of autoregressive errors in appearance, geometry, and motion as each generated segment becomes the conditioning context for subsequent segments. However, we believe SeerWorld can be extended to handle longer-horizon and more complex scenarios through targeted designs, such as constructing high-quality data with richer multi-entity and multi-event interactions and explicitly modeling multi-turn evolution during training. We will revise the paper to clarify that repeated state updates are already feasible within the current framework, while improving robustness over substantially longer horizons, more turns, multiple interacting events, and several evolving entities remains an important direction for future work.

$ $

Question (b): Extensibility of data construction

Thank you for the question. As discussed in our response to Critical Weakness (a), the current data and model already exhibit a certain degree of multi-turn capability. Supporting substantially more complex scenarios, however, would require extending the data-construction pipeline. In particular, story scripts should explicitly specify multiple entities, their evolving states, event dependencies, and camera trajectories.

Current commercial and open-source video generators, including Seedance and MiniMax-H3, can generate such complex scenarios. However, as complexity increases, the acceptance rate is likely to decrease and more fine-grained filtering will be required. The current annotation and checking pipeline can be extended accordingly by adding turn-level and cross-turn checks for entity identity, state persistence, event order, causal consistency, and camera motion.

The pipeline is not restricted to expensive commercial models. Our preliminary experiments with the open-source MiniMax-H3 model produce satisfactory results at relatively lower inference cost. Nevertheless, maintaining high data quality for substantially more complex multi-entity and multi-event scenarios remains an important research and engineering challenge and a promising direction for future work.

We greatly appreciate your positive assessment of our work and thanks again for your valuable comments which help improve our paper!

Thank you for your detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns. But I still have two questions:

(a) The new multi-turn evaluation provides encouraging quantitative results. But I also would like to see some visualization cases of multi-turn long videos, for example, some screenshots of a long video.

(b) The authors suggest that longer-horizon generation could be improved by constructing richer multi-entity and multi-event training data and explicitly modeling multi-turn evolution. However, I do not fully confirm that these approaches alone can effectively address error accumulation and visual quality degradation in autoregressive long video generation. These issues may also arise from the training-inference gap and accumulated errors in contexts, rather than simply insufficient training data length and complexity. I think there should be some further clarification and discussion.

Reviewer_mFHdOpenReview ↗
Rating
3
Confidence
3(+1)
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors formulate Out-of-Sight Reactive Evolution: the camera leaves a subject, an event occurs elsewhere, and on revisit the subject should show a causally plausible reaction. They build SeerWorld-Data, about 12k generated videos with segment-level annotations, and SeerWorld-Bench with 120 cases. As a reference solution, SeerWorld pairs a VLM brain, which records the world state and reasons about the off-screen reaction, with an LTX-2.5-based renderer trained with Identity-Anchored Attention and a two-stage autoregressive scheme. SeerWorld reaches 90.8% Core Task Success, far above recent video world models.

Critical strengths
  • Novel and meaningful task: The task goes beyond restoring previously seen content and asks whether the world keeps evolving outside the view. It jointly tests revisit consistency, event realization, and causal reaction, which existing benchmarks do not capture.

  • Useful data and benchmark: The dataset and benchmark cover diverse events, scenes, and reactions across photorealistic and game domains, with careful verification and train-test screening.

  • Clear and effective design: Separating world reasoning (VLM brain) from rendering is simple and interpretable. The ablations show that each part helps, and the brain prompt also transfers to another renderer (Evoke-Turbo).

Critical weaknesses
  • Synthetic data limits the approach to distillation: All training videos are generated by state-of-the-art commercial models (Seedance 2.0/2.5) from scripted scenarios, rather than collected from real data. The method is therefore essentially a distillation of these teacher models, and its performance is bounded by them. It cannot improve a state-of-the-art model in the same way, and it is unclear whether the learned behavior transfers to real-world videos. The teacher models themselves are also not evaluated on SeerWorld-Bench as a reference. In addition, Visual Integrity drops from 90.4 (LTX-2.5) to 83.8 after training.

  • Unclear generality as a data and benchmark contribution: As a dataset and benchmark paper, the data is used to train only one renderer (LTX-2.5), while baselines are evaluated without it and receive only the blind revisit prompt. It remains unclear how the data applies to other models, and how much of the gap comes from in-domain training data rather than the method. Finetuning other video world models on SeerWorld-Data would make this contribution more convincing.

  • Evaluation relies on a VLM from the same family as the brain: The reaction is reasoned by Qwen3.8-27B, and the main metrics, including Causal Plausibility, are also judged by Qwen3.8. This may favor reactions written in the judge's own style. The human study is also small (two raters, 50 cases, one baseline).

Questions for authors
  1. How do the results change if the VLM judge is replaced by a model from a different family, e.g., Gemini or GPT?
4 comments
AuthorsOpenReview ↗

Response to Weakness1

Thank you for the valuable feedback, here are our responses to all the concerns (weaknesses and questions).

Weakness1: Synthetic data & real-world evaluation

Thank you for this important comment. We agree that the use of synthetically generated training videos raises questions. We here therefore clarify more clearly from the following perspectives:

(1) Synthetic training data does not impose a direct behavioral upper bound. Although SeerWorld-Data is rendered by commercial video generators, SeerWorld is not trained by having these models execute our online out-of-sight evolution task. As described in Sec. 3.2, we provide Seedance with a complete, pre-scripted trajectory (including the event and final reaction) and generate the whole video in one bidirectional pass. Seedance itself does not expose the segment-level autoregressive interface required by our setting: it cannot receive a previously unseen event through a mid-rollout prompt switch, update the world state, and then autonomously infer the subject’s subsequent reaction. Moreover, approximately half of the generated videos fail our structural and semantic validation and are discarded (data verification see Section 3.2 of the main paper). Thus, the final dataset provides filtered offline trajectory supervision and Seedance itself cannot perform very well in our task.

To provide a teacher reference, we evaluate Seedance 2.0 on all 120 SeerWorld-Bench cases. Following the one-shot protocol used for Wan and LTX (appendix Table 7), Seedance receives the initial frame and a single composed prompt containing the initial scene, event, camera return, and unspecified reaction, which means it observes the future event and return instruction before generation begins.

MethodCore ↑Revisit ↑Event ↑Causal ↑React. ↑Visual ↑
Wan 2.28.340.087.517.118.377.9
LTX-2.327.597.594.243.342.188.8
LTX-2.534.295.090.850.842.990.4
Seedance 2.029.2100.0100.061.775.498.3

Despite excellent visual quality and perfect event/revisit execution, Seedance achieves only 29.2% Core Success. This is because its causal plausibility got many 0.5 scores instead of the full marks. The perfect event and revisit execution mainly due to its strong prompt following capability since we give the whole composed prompt to it. This demonstrates that Seedance is the upper bound for visual quality instead of out-of-sight reactive evolution.

(2) SeerWorld can be transferred to real-world videos. Specifically, we randomly selected 30 cases from the subset (which has clearly visible subjects ) of the SpatialVID dataset [1]. We extract the first frames and construct match prompts for real-world test. (Note that no SpatialVID videos are used for SeerWorld Training). The comparison results with the second-best video world model are as follows:

MethodCore ↑Revisit ↑Event ↑Causal ↑React. ↑
Evoke-Turbo20.090.066.725.025.0
SeerWorld86.796.790.088.365.0

(3) Visual Integrity trade-off. As shown in Appendix Table 7, the Visual Integrity decrease from 90.4 (base model LTX-2.5) to 83.8 (SeerWorld). We attribute this decrease primarily to two factors: (i) SeerWorld is finetuned on a specialized event–reaction dataset (i.e., SeerWorld-Data) than a very large-scale of data. This shifts capacity toward event and entity evolution, while slightly weakening general-purpose rendering fidelity. (ii) The LTX-2.5 base model generates the complete video bidirectionally in one pass and observes the intended future event and reaction from the beginning. SeerWorld instead performs segment-based autoregressive generation with mid-rollout prompt switches. This visual integrity trade-off (between bidirectional one-pass generation and autoregressive generation) is a commonly observed phenomenon in video generation. We will explicitly discuss this limitation and identify quality-preserving fine-tuning as future work.

[1] Wang et al. SpatialVID: A Large-Scale Video Dataset with Spatial Annotations. CVPR 2026.

AuthorsOpenReview ↗

Response to Weakness2, Weakness3, and Question1

Weakness2: Data generality

Thank you for this valuable suggestion. We agree that evaluating the transferability of SeerWorld-Data beyond the LTX-2.5 renderer provides stronger evidence for its generality. We address this concern from two complementary perspectives.

(1) Effect of the Brain reasoning prompt on existing baselines. In Sec. 4.4 of the main paper (Table 3 and Fig. 6), we additionally evaluate whether an existing baseline can benefit from the reasoning prompt produced by our Brain module. For Evoke-Turbo, replacing the blind revisit prompt with the Brain reasoning prompt improves Core Task Success from 21.7% to 45.0%, and Causal Plausibility from 32.5% to 53.3%. This controlled comparison changes only the revisit prompt and demonstrates that the Brain design transfers to another visual renderer. Nevertheless, Evoke-Turbo with the Brain prompt remains substantially below SeerWorld, showing that the improvement cannot be explained by prompt alone.

RendererCore Task SuccessCausal PlausibilityReaction Quality
Evoke-Turbo21.732.529.6
Evoke-Turbo + Brain reasoning45.053.350.8
SeerWorld90.894.285.0

(2) Fine-tuning additional renderers on SeerWorld-Data. We additionally fine-tuned two open-source video generators (Wan2.2-TI2V-5B and LTX-2.3) on SeerWorld-Data. Both models are trained with image-and-text conditioning and evaluated on the same 120-case SeerWorld-Bench. We evaluate each trained renderer with both the blind revisit prompt and the SeerWorld Brain reasoning prompt. The results are as follows (%):

RendererSeerWorld-DataPrompt TypeCore Task SuccessCausal PlausibilityReaction Quality
Wan2.2-TI2V-5B×Blind revisit8.317.118.3
Wan2.2-TI2V-5B✓Blind revisit17.534.235.0
Wan2.2-TI2V-5B✓Brain reasoning60.062.957.5
LTX-2.3×Blind revisit27.543.342.1
LTX-2.3✓Blind revisit48.358.350.8
LTX-2.3✓Brain reasoning78.385.078.3

Together with the component ablations in Table 2 of the main paper, these results demonstrate that both SeerWorld-Data and the proposed method provide substantial benefits for out-of-sight reactive evolution across different renderer families. We will clarify this more clearly in the final version.

$ $

Weakness3 & Question1: Additional evaluations

Thank you for the question.

(1) Following your suggestion, we re-evaluated SeerWorld and two representative world-model baselines using Gemini 3.1 Pro on SeerWorld-Bench. The main results are shown below (%):

MethodCore Task Success ↑Revisit Success ↑Event Success ↑Causal Plausibility ↑
LingBot-World-v218.373.362.520.8
Evoke-Turbo20.896.779.226.7
SeerWorld89.297.593.390.0

Although the absolute values of some individual metrics differ between Qwen3.8-27B and Gemini 3.1 Pro, the central conclusion and relative ranking remain stable. In particular, SeerWorld achieves an 89.2% Core Task Success rate under Gemini, close to the 90.8% obtained with Qwen, and substantially outperforms both LingBot-World-v2 and Evoke-Turbo (although Gemini is generally more stringent and assigns slightly lower scores to all considered methods, this does not affect the core conclusion).

(2) Besides, we conduct an additional human study comparing SeerWorld with LingBot-World-v2 on 50 randomly selected cases. SeerWorld achieves an 88% win rate for task completion, which jointly considers successful event realization, revisiting, and subject reaction. It also obtains a 56% win rate for visual quality (n=50).

These additional results further demonstrate the effectiveness of SeerWorld and its substantially stronger capability for out-of-sight reactive world evolution.

We greatly appreciate your positive assessment of our work and thanks again for your valuable comments which help improve our paper!

Thank you for preparing such a detailed rebuttal. I believe the discussion could substantially strengthen the paper, and it makes me more confident in my weak accept rating.

AuthorsOpenReview ↗

Thanks for reviewing our paper!

Thank you for your reply. We are glad that our rebuttal has appropriately addressed your concerns. Thank you again for your valuable comments on our work!

Reviewer_XxvpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies Out-of-Sight Reactive Evolution, focusing on whether subjects that have left the field of view can respond appropriately to newly introduced events. The authors construct a dataset and benchmark covering a sequence of subject exit, event occurrence, camera revisit, and subject reaction. The method uses a VLM to infer appropriate reactions from the generated video history and trains an LTX-2.5-based video renderer on the constructed dataset, incorporating Identity-Anchored Attention (IAA) to preserve subject identity. The reported results show substantial improvements over the compared methods on the proposed benchmark.

Critical strengths
  1. The paper is well-motivated and addresses a meaningful problem. Subjects should remain responsive to new events even when they are outside the camera’s field of view, and their states should reflect these events when revisited. This is a natural and worthwhile requirement for world modeling.

  2. The paper reports substantial performance improvements over existing baselines, demonstrating the effectiveness of the proposed system on the evaluated task.

Critical weaknesses
  1. Although the paper discusses related work on out-of-sight dynamics, such as LiveWorld, its quantitative comparisons mainly focus on general-purpose interactive video world models. Comparisons with methods that directly address out-of-sight state evolution are missing, leaving the added value over these closely related approaches insufficiently established.

  2. The contribution of IAA is not sufficiently supported by the current evidence. In Table 2, adding IAA increases Core Task Success from 82.5% to 84.2%, corresponding to a net increase of only two successful cases on the 120-case benchmark. Furthermore, IAA is primarily motivated by subject identity preservation, but the ablation does not report Subject Consistency or other directly relevant metrics, making it difficult to assess whether IAA improves identity preservation as intended.

Questions for authors
  1. The limitations section notes that the current evaluation focuses on a single event followed by a single subject reaction. If the camera revisits the same subject multiple times, can the system consistently retain and update the subject’s post-reaction state so that subsequent behavior remains consistent with prior events and state changes? Could the authors provide examples to assess this capability?
3 comments
AuthorsOpenReview ↗

Response to Weakness1

Thank you for the valuable feedback, here are our responses to all the concerns (weaknesses and questions).

Weakness1: Direct comparison with the out-of-sight state evolution method

Thank you for the valuable comment. In our original experiments, we mainly compared SeerWorld with recent, strong video world models such as LingBot-World and Alaya-Evoke, since these models provide powerful autoregressive video generation and interactive world-rendering capabilities. Following the reviewer’s suggestion, we further examined methods developed specifically for out-of-sight state evolution and added a direct comparison.

As discussed in our Related Work, the two most closely related methods are LiveWorld [1] and Hybrid Memory (HyDRA) [2]. HyDRA mainly addresses the out-of-sight evolution of moving entities in game-style environments through its hybrid memory mechanism. Specifically, when the camera moves away from a walking entity and later revisits it, HyDRA can render the entity at an updated location rather than leaving it frozen at its last observed position. This form of locomotion-based evolution is also covered by SeerWorld, but our setting further requires out-of-sight reactive evolution: the model must render a newly introduced event through prompt switching and then generate a new, causally appropriate reaction from the off-screen subject, such as escaping, taking cover, or raising a shield.

We attempted to adapt HyDRA to this setting and provided it with the complete event and reasoning/revisit prompts. However, both its memory mechanism and training data are primarily designed for propagating the location and motion states of already moving entities. In particular, its training data mainly consists of entities walking or moving through game-style environments, without newly introduced events or more complex reactive behaviors such as escaping, taking cover, or raising a shield. Consequently, HyDRA generally continued the entity’s existing motion and did not reliably render either the newly introduced event or a corresponding reaction. We therefore use LiveWorld [1], whose world-state representation and generative renderer can be adapted to our complete rollout protocol, as the principal quantitative comparison.

We evaluate two LiveWorld variants on SeerWorld-Bench. In addition to the standard blind-prompt setting, we provide LiveWorld with the same reasoning/revisit prompt generated by SeerWorld’s brain. The results are as follows:

MethodCore ↑Revisit ↑Event ↑Causal ↑React. ↑Visual ↑Subj. Cons. ↑Bg. Cons. ↑Motion ↑
LiveWorld14.295.869.222.519.656.278.396.798.8
LiveWorld + Brain prompt/reasoning33.395.873.340.031.262.986.396.798.9
SeerWorld (Ours)90.8100.097.594.285.083.899.298.399.5

The results show that LiveWorld performs well in Revisit Success (95.8%) and Background Consistency (96.7%), validating its ability to preserve and return to an established world state. Providing SeerWorld’s brain/reasoning prompt improves LiveWorld’s Core Task Success from 14.2% to 33.3%. However, even with access to the complete brain prompt, it achieves only 40.0% Causal Plausibility and 31.2% Reaction Quality, resulting in 33.3% Core Task Success. In comparison, SeerWorld achieves 90.8% Core Task Success, with substantially stronger Event Success, Causal Plausibility, and Reaction Quality. These results suggest that maintaining an out-of-sight world state is not by itself sufficient for out-of-sight reactive evolution: the model must also incorporate a newly realized event and render a coherent, event-conditioned change in the subject’s behavior. We will add this quantitative comparison and the corresponding discussion to the final version.

[1] Duan et al. LiveWorld: Simulating Out-of-Sight Dynamics in Generative Video World Models. ECCV 2026.

[2] Chen et al. Out of Sight but Not Out of Mind: Hybrid Memory for Dynamic Video World Models. arXiv 2026.

AuthorsOpenReview ↗

Response to Weakness2 and Question1

Weakness2: More details about IAA

Thank you for the comment. In Table 2 of the main paper, we focus on core task metrics (e.g., Core Task Success, Revisit Success). We therefore provide the complete comparison here:

ConfigurationCore ↑Revisit ↑Event ↑Causal ↑React. ↑Visual ↑Subj. Cons. ↑Bg. Cons. ↑
Stage 1 w/o IAA82.5100.096.788.880.882.195.497.1
Stage 1 + IAA84.299.298.388.783.983.399.297.9
Full method90.8100.097.594.285.083.899.298.3

The first two rows isolate the contribution of IAA. Although Core Task Success improves modestly from 82.5% to 84.2%, Revisit Subject Consistency increases from 95.4 to 99.2, reducing the remaining identity-consistency error by approximately 83% (3.8% absolute improvement). Reaction Quality and Visual Integrity also improve from 80.8 to 83.9 and from 82.1 to 83.3.

These results are consistent with the purpose of IAA: it strengthens the attention from the denoising revisit tokens to clean historical entity-reference tokens, reducing identity drift while still permitting the necessary changes in pose and state. IAA also indirectly improves reaction rendering quality, since rendering a coherent reaction requires generating new poses and subject–object interactions based on the subject’s appearance and structure. We will add these results and analysis to the final version.

$ $

Question1: Multi-turn evolution

Thank you for the question. As mentioned in the limitation section, we did not focus on multi-turn evolution in this paper. However, to evaluate this capability, we now add a multi-turn evolution evaluation on all SeerWorld-Bench cases:

subject → event → reaction → event evolves → next reaction evolution

Here is an example: In the game domain, the camera pans away from a marsh ranger and reveals a spectral knight materializing from the mist on the opposite bank. When the camera returns, the ranger has risen from her crouched pose and drawn her sword into a defensive stance. The camera then revisits the same spectral knight, which preserves its identity, raises its sword and shield, and advances in response. On the next return, the ranger retains her drawn-sword state, backs up beside a tree, and lowers into a defensive crouch rather than resetting to her initial pose (see details in paper Appendix Section J of the current version, page 21-22).

These multi-turn autoregressive videos contain 465 frames, or approximately 19.4 seconds at 24 fps. We use the same SeerWorld checkpoint without additional multi-turn training. Crucially, each later event or subject-reaction segment is conditioned on its corresponding previously generated state, enabling stateful multi-turn evolution rather than resetting to the initial observation. The main results are:

Multi-turn metricSeerWorld
Multi-turn subject revisit success95.8%
Subject identity consistency87.5%
Multit-turn causal plausibility81.7%
Multi-turn reaction quality75.8%

The results show that SeerWorld can recover the previously observed subject over repeated revisits while largely preserving its identity and producing a causally appropriate subsequent reaction. Specifically, the 87.5% identity consistency shows that the subject is usually preserved across the alternating rollout. The 81.7% multi-turn causal plausibility further demonstrates that the later behavior can remain consistent with the preceding event and state changes, rather than merely reproducing the initial reaction.

Meanwhile, the decrease of some metrics (e.g., reaction quality) reflects the expected accumulation of visual and motion errors over autoregressive turns. This is consistent with the scope stated in our limitations: SeerWorld was not designed for extended multi-turn evolution, so later turns show less quality than the initial reaction, although they retain a meaningful level of causal and visual consistency. Notably, despite operating in this more difficult multi-turn setting, SeerWorld’s multi-turn causal plausibility remains higher than the single-turn causal-plausibility results of all baseline methods reported in the main paper (e.g., 32.9% causal plausibility for Evoke). We emphasize that these results demonstrate the multi-turn capability of SeerWorld to a certain degree.

We will add these results and related discussions to the final version. We will also revise the limitations to clarify that repeated state updates are already feasible within the current framework, while improving robustness over substantially longer horizons and more turns is an important direction for future work. We believe SeerWorld can be extended to handle more complex scenarios through targeted designs, such as constructing high-quality data with richer multi-entity and multi-event interactions and explicitly modeling multi-turn evolution during training.

Thanks again for your valuable comments which help improve our paper!

AuthorsOpenReview ↗

Thanks for reviewing our paper!

Dear Reviewer Xxvp,

Thank you again for your time and effort in reviewing our manuscript. We greatly appreciate your assessment of our work.

As the reviewer-author discussion period is coming to an end, we would like to kindly ask if there are any remaining points that you would like us to clarify. We believe we have made a sincere effort to address your earlier comments, and we would be grateful for any additional feedback you may have.

Thank you again for your valuable comments which help improve our paper!

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.