acceptodds
Under review as a conference paper at ICLR 2027

PreShort: Efficient World Modeling with Pretrained Visual Representations

15 views

Abstract

Model-based reinforcement learning (MBRL) leverages learned world models for sample-efficient control, provided that their representations retain the information needed to predict the consequences of actions. General-purpose pretrained visual representations offer reusable features learned beyond an agent's online experience, motivating their use as a prediction space for world modeling, while turning these features into imagined trajectories requires learning action-conditioned transitions and accounting for uncertain future states. We introduce PreShort, an online world model that employs frozen visual features as both observation representations and continuous imagination targets. Its core component, Differential Shortcut Dynamics, encodes latent-action history with differential attention and conditions a stochastic shortcut predictor on the resulting context. The context is reused across a few refinement steps to generate each successor, and imagined trajectories support actor-critic learning. Freezing the visual interface also enables feature caching and confines online world model updates to downstream modules. Experiments show that PreShort achieves a mean human-normalized score of % on Atari K, a mean return of on the DeepMind Control Suite, and a return of on Crafter, exceeding strong world model baselines in mean aggregate performance across all three benchmarks. Further analyses assess rollout accuracy, compatibility with different pretrained visual backbones, and computational efficiency in terms of trainable parameters and training cost.

Primary area:
reinforcement learning
Keywords:
world model, reinforcement learning, shortcut model, pretrained visual encoder

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_wJcjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PreShort, a unified online model-based reinforcement learning framework that integrates frozen pretrained visual representations with efficient stochastic world modeling and imagination-based actor-critic learning. To make this integration practical, the proposed Differential Shortcut Dynamics combines differential attention for temporal context modeling with few-step shortcut generation of successor states. Experiments on Atari 100K, the DeepMind Control Suite, and Crafter demonstrate competitive performance, supported by component ablations, alternative visual backbones, rollout-accuracy evaluation, and efficiency analysis.

Critical strengths
  1. The paper is well written and easy to understand.
  2. Meaningful framework integration.PreShort combines frozen pretrained visual representations, efficient stochastic world modeling, and imagination-based actor-critic learning in a unified online framework, reducing the need to jointly learn perception and dynamics from limited interactions.
Critical weaknesses
  1. Limited methodological novelty. The framework mainly combines existing components, including frozen pretrained visual encoders, differential attention, shortcut generation, and standard imagination-based actor-critic learning. The contribution is therefore mainly system integration and with moderate methodological novelty.
  2. The ablations do not clearly isolate the contribution of each component. The “No Shortcut” variant simultaneously removes stochastic initialization, iterative refinement, token attention, and shortcut-consistency training. It is therefore unclear whether the improvement comes from stochastic modeling, the shortcut objective, additional computation, or the architecture itself.
Questions for authors
  1. Figure 3 and Appendix G.3 — attribution of the shortcut improvement. The “No Shortcut” variant replaces four-step stochastic generation with a single-pass deterministic MLP, simultaneously removing noise initialization, iterative refinement, token attention, and shortcut consistency. Can the authors provide a comparison against ordinary conditional flow matching with the same four refinement steps, temporal encoder, and comparable parameter budget? This would clarify whether the performance gap specifically supports shortcut consistency or primarily reflects the benefit of stochastic iterative generation.
  2. Figure 4(a) — evidence for reduced rollout error accumulation. Section 4.4.1 discusses prediction accuracy as the rollout horizon increases, but Figure 4(a) plots 16-step rollout PSNR against environment interactions. How is PSNR aggregated across the 16 prediction steps, and how are stochastic samples evaluated—for example, using one sample, an average over samples, or the best sample?
  3. Does the reported efficiency include the inference and feature-caching costs of the frozen visual encoder?
3 comments
AuthorsOpenReview ↗

Response to Reviewer wJcj (1/2)

We thank the reviewer for recognizing the empirical effectiveness of PreShort, and for the constructive feedback. We address each concern below and the corresponding revisions have been incorporated into the revised manuscript and highlighted in blue.

W1: Methodological and efficiency contributions of PreShort.

We would like to clarify that PreShort’s contribution extend beyond an online world-modeling architecture designed for frozen pretrained visual representations to the design of Differential Shortcut Dynamics (DSD) for accurate and efficient dynamics prediction. PreShort also contributes a latent-space diffusion design that improves efficiency by reusing compressed context to avoid excessive reliance on historical trajectories. DSD firstly introduces differential attention into online world modeling, which is used to learn a more informative compact representation of visual and action histories. Unlike diffusion-based methods that repeatedly process historical trajectories during denoising [1], DSD reuses the compact history across refinement steps, enabling efficient successor prediction in the continuous pretrained feature space. In terms of computational efficiency, PreShort achieved the highest performance in the shortest training time.

The comparisons in Appendix D.2 further motivate this design. Incorporating frozen Wan-VAE features into categorical world models does not improve control performance in the evaluated settings. Re-encoding continuous features into categorical latent variables (as in most world-model architectures) may weaken the preservation of fine-grained, control-relevant information (as shown in Figure 2). PreShort supports general pretrained visual models through a common flattened-feature interface (Appendix D1 and D3 for other visual backbones), with DSD directly modeling transitions in their continuous spaces. These results support the importance of the downstream DSD dynamics design in translating pretrained visual information into control performance.

[1] Alonso E, Jelley A, Micheli V, et al. Diffusion for world modeling: Visual details matter in atari[J]. Advances in Neural Information Processing Systems, 2024, 37: 58757-58791.

W2/Q1: Isolating the contribution of shortcut consistency.

We have added PreShort without Lsc\mathcal{L}_{\mathrm{sc}}, which removes the shortcut consistency loss while retaining the other framework (the full learning curves are provided in Appendix D.6 of the revised manuscript).

This variant is equivalent to parameter-matched conditional flow matching with xx-space prediction (same prediction space as PreShort). Removing Lsc\mathcal{L}_{\mathrm{sc}} leaves endpoint regression along the noise–data interpolation path as the dynamics prediction objective. And the equivalence follows because the endpoint regression corresponds to a time-weighted flow-matching objective, while the predictor architecture and parameter count remain unchanged. The results are summarized in the following table. Full PreShort performs better across all three benchmarks, supporting an additional benefit from shortcut consistency beyond these shared components.

BenchmarkPreShortPreShort w/o Lsc\mathcal{L}_{\mathrm{sc}}
Atari 100K — Mean HNS (%)146.7135.5
DMC — Mean return821.5737.8
Crafter — Return10.79.1

Q2: Definition and aggregation of rollout PSNR.

PSNR measures the pixel fidelity of frames decoded from predicted visual features relative to the corresponding ground-truth frames and higher values indicate closer agreement with the ground truth. Figure 4(a) plots this metric against environment interactions during training and the prediction horizon remains fixed at 16 steps throughout. Thus, the figure measures how rollout fidelity improves with training, rather than how prediction error varies with rollout horizon. We have revised the main text and figure caption to clarify the evaluation procedure and the interpretation of Figure 4(a).

All variants perform autoregressive prediction from the same observation histories and recorded action sequences, and their predicted features are decoded using the frozen Wan-VAE decoder. At each checkpoint, we evaluate 1,024 imagined trajectories using the same distribution of contexts and actions across variants. We compute PSNR separately for each predicted frame and its corresponding ground-truth frame, average over the 16 prediction steps and 1,024 trajectories, and then average the results across five random seeds. Every generated trajectory contributes to the metric; no best-sample selection is used. PreShort achieves the highest average rollout PSNR under this protocol (indicating more accurate imagined trajectories than those generated by the other methods).

Respond to Question 3 continued on the next comment.

AuthorsOpenReview ↗

Response to Reviewer wJcj (2/2)

Q3: Does the reported efficiency include visual encoding and feature caching?

Yes. The reported training time of 1.1 hours per Atari 100K game includes the total online training time, including Wan-VAE forward encoding of newly collected observations, feature-cache writes and reads, world model updates, imagined rollout generation, actor–critic updates, and environment interaction. PreShort uses the official Wan2.2_VAE.pth checkpoint released with Wan2.2-TI2V-5B, performs no additional visual fine-tuning, and keeps the pretrained visual encoder frozen throughout online RL.

Thank you again for the detailed and constructive feedback. We are happy to provide any further experiments or details as needed.

AuthorsOpenReview ↗

Follow-up on our response

Thank you again for your constructive feedback, which helped us clarify the methodological contribution of DSD, particularly how reusing the encoded history across refinement steps improves computational efficiency. We hope this explanation, together with our clarifications regarding the PSNR evaluation and efficiency measurements, has helped address your questions and concerns. If any points remain unclear or new questions arise, please let us know, and we would be very happy to provide further clarification and continue the discussion.

Reviewer_ghyrOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes PreShort, an MBRL method that performs self supervised training and accurate imagination in the continuous representation space of a frozen pretrained vision encoder. It summarizes history observations into compact representations, and condition the shortcut predictor on this context to integrate information for predicting future. Unlike Dreamer 4, this design allows reuse of the history summarization.

Critical strengths
  1. The investigation on how to preserve the information in the continuous features of pretrained visual encoder is valuable. The model design in this paper is closely aligned with the motivation. The insight that MBRL should focus more on preserving information for dynamics prediction and policy learning is interesting and leads to the idea of incorporating frozen visual encoder features, whose info is kept throughout rollouts through directly imaginations in this continuous space.

  2. The empirical results provide supporting evidence for the effectiveness of pretrained visual features, together with the model design which keeps information in that feature. Results on Pong demonstrate valuable information in pretrained encoders, while comparison against MBRL which preserves vanilla structure but substitutes visual features demonstrate the effectiveness of model design.

Critical weaknesses
  1. There lacks a comparison between pretrained vision encoders and jointly optimized encoders. This is a direct evidence for the effectiveness of pretrained vision encoders, despite the modifications on the model structure.
Questions for authors
  1. What's the comparison between vanilla DV3 and modified DV3 in D.2?

  2. Related to W1, what if substituting the vision encoder in PreShort with a randomly initialized one?

2 comments
AuthorsOpenReview ↗

Response to Reviewer ghyr

We thank the reviewer for recognizing the value of PreShort, and for the constructive feedback. Below, we address each concern and describe the corresponding revisions, which have been incorporated into the revised manuscript and highlighted in blue.

W1/Q2: Comparison with a jointly optimized visual encoder.

We have added CNN-PreShort in Appendix D.5 (revised manuscript), which replaces PreShort’s frozen pretrained visual encoder with a randomly initialized CNN encoder trained jointly with the world model (consistent with PreShort’s decoder-free architecture). We retain PreShort’s downstream DSD architecture and use the same experimental conditions.

BenchmarkPreShortCNN-PreShort
Atari 100K — Mean HNS (%)146.778.2
DMC — Mean return821.5606.3
Crafter — Return10.77.2

CNN-PreShort performs worse across all three benchmarks. These results support the benefit of using frozen pretrained visual features over learning a CNN encoder jointly from scratch within the PreShort framework. Related works on decoder-free world models [1, 2] also highlights the requirement for additional regularization losses to prevent representation collapse when a decoder is not used to supervise representation learning.

[1] Morihira N, Nahar A, Bharadwaj K, et al. R2-Dreamer: Redundancy-reduced world models without decoders or augmentation[J]. ICLR 2026.

[2] Bredis G, Balagansky N, Gavrilov D, et al. Next embedding prediction makes world models stronger[J]. arXiv preprint arXiv:2603.02765, 2026.

Q1: Comparison between vanilla and modified DreamerV3.

We have added the learning curves of vanilla DreamerV3 to Figure 6 in Appendix D.2 (purple curves in the revised manuscript). Vanilla DreamerV3 performs slightly better than the modified version incorporating frozen Wan-VAE features. This comparison indicates that solely introducing pretrained visual features does not guarantee improved control performance; their benefits depend on how the downstream world model uses them.

BenchmarkPreShortModified DreamerV3Modified STORMDreamerV3
Atari 100K — Mean HNS (%)146.7114.4108.6124.5
DMC — Mean return821.5591.4558.6784.9
Crafter — Return10.77.36.59.4

Although the pretrained features retain fine-grained visual information, they are subsequently encoded into categorical latent variables for dynamics modeling. This step make subtle but control-relevant distinctions harder to preserve (such as the ball’s position and its changes over time illustrated in Figure 2).

PreShort directly predicts successor states in the continuous pretrained feature space. Its Differential Shortcut Dynamics (DSD) uses differential attention to aggregate visual and action histories into a compact context for successor prediction, allowing the resulting continuous representations to support better multi-step imagination (as shown in Figure 4) and policy learning. The results support the value of pretrained visual features within PreShort and highlight the importance of a compatible downstream dynamics model.

We hope these clarifications address the reviewer’s concerns. Please let us know if there are any remaining questions or aspects that could benefit from further clarification.

AuthorsOpenReview ↗

Follow-up on our response

Thank you again for suggesting these comparisons, which motivated the additional experiments described in our response. We hope the results and accompanying explanations have helped address your questions and concerns, and would appreciate any further feedback you may have. If any points remain unclear or new questions arise, please let us know, and we would be very happy to provide further clarification.

Reviewer_FivBOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PreShort, an online model-based reinforcement learning framework that leverages frozen pretrained visual representations for efficient world modeling. Its core component, Differential Shortcut Dynamics (DSD), combines differential attention for temporal context modeling with shortcut-based stochastic latent prediction. The model enables efficient imagined rollouts for actor-critic learning and achieves competitive performance across Atari 100K, DMC, and Crafter.

Critical strengths
  • Well-designed architecture. DSD effectively integrates temporal context modeling and stochastic latent prediction, enabling efficient imagination-based policy learning.
  • High computational efficiency. PreShort substantially reduces online training time and the number of trainable parameters compared to existing world models.
Critical weaknesses
  • Potentially unfair comparisons. The main comparisons against methods trained from scratch do not fully account for the advantage of external visual pretraining.
  • Limited novelty. The proposed framework primarily integrates existing techniques, including pretrained visual encoders, differential attention, and shortcut models, with limited methodological innovation.
  • Missing relevant baseline. The main experiments lack a direct comparison with JEDI, a closely related online diffusion-based world model.
  • Limited backbone generalization evidence. Evaluating only three pretrained visual backbones provides insufficient evidence to support a broad backbone-agnostic claim.
  • Unclear pretraining data and out-of-distribution generalization. The paper does not specify the pretraining data of Wan-VAE or clarify potential overlap with the evaluation environments. Its robustness to unseen visual domains, such as DMC-GB2, remains unexplored.
  • Marginal performance improvements. Despite its additional architectural complexity, PreShort achieves only modest performance gains over the strongest baselines on several benchmarks.
Questions for authors

See strengths and weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer FivB (1/2)

We thank the reviewer for recognizing the architecture design and computational efficiency of PreShort. We address each concern below with the additional experiments and clarifications. The corresponding revisions have been incorporated into the revised manuscript and highlighted in blue.

W1: Further comparisons.

We assess the contribution of external visual pretraining through the following two complementary comparisons.

First, Appendix D.2 evaluates DreamerV3 and STORM with the same frozen Wan-VAE features used by PreShort, while retaining their categorical latent dynamics and associated learning procedures.

BenchmarkPreShortDreamerV3 + WanSTORM + Wan
Atari 100K — Mean HNS (%)146.7114.4108.6
DMC — Mean return821.5591.4558.6
Crafter — Return10.77.36.5

Access to the same pretrained features is insufficient to reproduce PreShort’s performance. This comparison supports the importance of a compatible downstream dynamics design for translating pretrained visual information into effective control.

Second, we further adds Appendix D.5 to the revised manuscript, which introduces CNN-PreShort, which replaces the frozen Wan-VAE with a randomly initialized CNN trained jointly with the world model (same decoder-free framework as PreShort). The results are shown in the following table and support the benefit of using frozen pretrained visual features over learning a CNN encoder jointly from scratch within the PreShort framework. Related works on decoder-free world models [1, 2] also highlight the need for additional regularization losses to prevent representation collapse when a decoder is not used to supervise representation learning.

BenchmarkPreShortCNN-PreShort
Atari 100K — Mean HNS (%)146.778.2
DMC — Mean return821.5606.3
Crafter — Return10.77.2

We use a CNN (with the same hyperparameters with STORM) as a practical training-from-scratch reference under the limited interaction and computational budgets of online visual RL. Jointly optimizing the much larger Wan encoder would substantially increase the online optimization burden (PreShort directly uses publicly available pretrained weights without any fine-tuning for the RL tasks). Together, these experiments support both the practical benefit of pretrained perception and the importance of the downstream DSD dynamics.

[1] Morihira N, Nahar A, Bharadwaj K, et al. R2-Dreamer: Redundancy-reduced world models without decoders or augmentation[J]. ICLR 2026.

[2] Bredis G, Balagansky N, Gavrilov D, et al. Next embedding prediction makes world models stronger[J]. arXiv preprint arXiv:2603.02765, 2026.

W2: Methodological and efficiency contributions.

PreShort addresses how to turn frozen visual features into accurate stochastic imagined trajectories at a cost suitable for online world modeling. DSD addresses this through a separation between temporal context encoding and within-transition stochastic generation. PreShort also contributes a latent-space diffusion design that improves efficiency by reusing compressed context to avoid excessive reliance on historical trajectories.

PreShort firstly introduce the causal differential-attention which summarizes the history into a compact context hth_t into online world modeling. The shortcut predictor then reuses this context across refinement steps, directly predicting the clean successor in the continuous feature space. History encoding is therefore performed once per transition, while refinement operates on the compact context and the evolving successor candidate. This context reuse avoids repeatedly processing the historical sequence within each refinement step compared with existing diffusion-based methods [3]. In terms of computational efficiency, PreShort achieved the highest performance under the least cost.

The component ablations support these design choices: replacing differential attention with standard self-attention or replacing the token-attention interface with concatenation reduces control performance (Section 4.3 and Appendix D). We have also added a controlled comparison in Appendix D.6 that removes shortcut consistency while retaining the other settings.

BenchmarkPreShortPreShort w/o Lsc\mathcal{L}_{\mathrm{sc}}
Atari 100K — Mean HNS (%)146.7135.5
DMC — Mean return821.5737.8
Crafter — Return10.79.1

This comparison supports an additional benefit from shortcut consistency under the same architecture and sampling budget.

[3] Alonso E, Jelley A, Micheli V, et al. Diffusion for world modeling: Visual details matter in atari[J]. Advances in Neural Information Processing Systems, 2024, 37: 58757-58791.

Responses to the remaining weaknesses continue in the next comment.

AuthorsOpenReview ↗

Response to Reviewer FivB (2/2)

W3: Comparison with JEDI.

JEDI is an excellent world-modeling approach that jointly learns visual representations and latent diffusion dynamics through online interaction. We have added the reported per-game results of JEDI’s flagship version to the revised Table 7. JEDI’s flagship version achieves a mean HNS of 145.0%, compared with 146.7% for PreShort. As discussed in W2 and Appendix A, a key distinction between PreShort and JEDI is that PreShort reuses a compressed representation of the trajectory history across prediction steps, avoiding the need to repeatedly process the raw history as JEDI does and thereby improving computational efficiency.

We would like to respectfully clarify that, as discussed in Appendix A, PreShort and JEDI address distinct but complementary questions in online world modeling. JEDI’s contribution lies in combining JEPA-style predictive representation learning with conditional diffusion, allowing the denoising objective to directly shape the visual latent space learned online. PreShort focuses on efficient context compression and stochastic dynamics modeling over pretrained representations. By incorporating pretrained visual representations, PreShort reduces the number of parameters that require training and achieves higher training efficiency than JEDI, requiring only 4.5 hours per Atari 100K task on a single A100 GPU.

W4: Evidence for compatibility across visual backbones.

We have expanded the evaluation from 3 to 6 pretrained visual encoders. The original experiments cover Wan-VAE, the continuous Cosmos Tokenizer, and DINOv3, representing variational reconstruction, non-variational reconstruction, and self-supervised visual learning. We have additionally evaluated MAE ViT-B/16, SigLIP2 ViT-B/16, and R3M ResNet-50, broadening the evaluated pretraining approaches. The results are shown below.

Frozen visual encoderAtari mean HNS (%)DMC mean returnCrafter return
Wan-VAE146.7821.510.7
SigLIP2 ViT-B/16132.5717.49.4
MAE ViT-B/16136.3782.18.7
R3M ResNet-50125.2748.69.3

These results further support PreShort’s compatibility across the evaluated encoders and pretraining approaches. Flattened continuous feature vectors provide a common interface through which DSD learns action-conditioned history representations and stochastic successor predictions. This interface accommodates the feature dimensions of different representation models, allowing the same dynamics modeling framework to be reused across these visual backbones.

W5: Pretraining provenance and visual generalization.

PreShort uses the official Wan2.2_VAE.pth checkpoint released with Wan2.2-TI2V-5B. We perform no additional visual pretraining or fine-tuning on Atari, DMC, or Crafter data, and keep the encoder frozen throughout online RL. To address visual generalization empirically, we have added DMC-GB2 experiments in Appendix D.4, covering Color Hard, Video Hard, and Color Video Hard. The comparison uses the same settings as in EAWM [4].

DMC-GB2 settingPreShortDreamerV3STORM
Color Hard610456430
Video Hard423343315
Color Video Hard423402362

PreShort achieves higher mean returns in all three settings. These results provide additional evidence of robustness to appearance and background changes.

[4] Zhao-Han P, Li S, Li Z, et al. From Observations to Events: Event-Aware World Models for Reinforcement Learning[C]//International Conference on Learning Representations. 2026, 2026: 155763-155805.

W6: Performance gains relative to architectural complexity.

PreShort’s architecture is designed to support training efficiency and performance (as we discussed in W2). As shown in the main text, PreShort requires only 1.1 hours to complete training on a single Atari 100K task on one RTX 4090, whereas STORM, DreamerV3-torch, and DIAMOND require 3.2 hours, 7.9 hours, and 3.09 days, respectively. At a lower training cost, PreShort achieves the highest mean and median HNS. These results demonstrate that PreShort achieves higher average returns and surpasses human-level performance on more tasks than the baseline methods, while reducing online training costs.

We appreciate the reviewer’s suggestions, which helped strengthen the comparisons and clarify the scope of our contribution. We hope the additional evidence addresses these concerns and provides a clearer basis for reassessing the paper.

AuthorsOpenReview ↗

Follow-up on our response

Thank you again for your detailed feedback, which motivated the additional comparisons, backbone evaluations, and DMC-GB2 experiments described in our response. We also clarified the methodological contribution of DSD, particularly its separation of temporal context encoding from within-transition stochastic generation and its reuse of the encoded history across refinement steps to improve computational efficiency. We hope these results and explanations have helped address your questions and concerns. If any points remain unclear or new questions arise, please let us know, and we would be very happy to provide further clarification and continue the discussion.

AuthorsOpenReview ↗

Further experiments

We sincerely thank you again for the constructive comments, which led us to test PreShort with a wider range of visual backbones. Since our previous response, we have evaluated three additional frozen encoders, I-JEPA ViT-H/14, iBOT Swin-T/7, and VIP ResNet-50, bringing the total from 6 to 9. The table reports results under the same interaction budgets, and the revised manuscript includes the corresponding learning curves.

Frozen visual backbone in PreShortAtari mean HNS (%)DMC mean returnCrafter return
PreShort (Wan-VAE)146.7821.510.7
DINOv3 ViT-S/16117.4681.89.3
Cosmos Tokenizer CI8×8105.6599.78.4
SigLIP2 ViT-B/16132.5717.49.4
MAE ViT-B/16136.3782.18.7
R3M ResNet-50125.2748.69.3
I-JEPA ViT-H/14132.0703.110.3
iBOT Swin-T/7134.4794.19.6
VIP ResNet-50123.6777.39.3

These results demonstrate that PreShort generalizes across the visual backbones evaluated here. With the same downstream dynamics architecture, Wan-VAE remains the strongest encoder overall, while the gains over the baselines extend across backbones with different architectures and pretraining objectives.

As the discussion draws to a close, we hope this expanded evaluation addresses your concern about the evidence for backbone generality. We appreciate your feedback and would be happy to clarify any remaining questions.

Comments on the paper

2 comments

We are grateful to the reviewers for their time and thoughtful feedback. The additional experiments and revisions have strengthened the empirical support for our method and improved the clarity of the paper. As the discussion period draws to a close, we would be grateful if you could reassess the paper in light of these additional results and clarifications. We would be happy to answer any remaining questions.

AuthorsOpenReview ↗

Final Rebuttal Summary

We sincerely thank the reviewers, AC, and SAC for the time, effort, and insightful comments. Across the reviews, we were encouraged by the recognition of PreShort’s approach to efficient imagination-based policy learning, which combines frozen pretrained visual features with stochastic dynamics in an online world model. We also appreciate the positive comments on its computational efficiency and the empirical support for its design. As the discussion period draws to a close, we would like to share a brief summary of our rebuttal and the revision in the manuscript for the consideration.

Methodological and efficiency contributions of PreShort.

  • A general dynamics architecture for pretrained visual encoders. PreShort learns stochastic transitions directly in the continuous feature space of the visual representation. This design supports imagination-based policy learning across different feature layouts and pretraining objectives, since the dynamics architecture does not assume a specific spatiotemporal layout (Appendices D.1 and D.3).
  • Differential attention for stochastic shortcut prediction. To our knowledge, PreShort firstly introduces differential attention into online world modeling to better preserve predictive information when compressing visual and action histories. DSD reuses this compact context across refinement steps, improving computational efficiency by avoiding the repeated processing of long trajectory histories during the denoising process of diffusion-based methods.
  • Computational efficiency. PreShort achieves the highest performance in the shortest training time.

Further experiments.

  • Compatibility across more visual backbones. We evaluated PreShort with additional frozen visual encoders, providing further evidence of its compatibility across backbones and pretraining approaches.
  • Further evaluation on DMC-GB2. We added experiments on DMC-GB2 to test PreShort’s out-of-distribution generalization under changes in appearance and background. PreShort achieved higher mean returns than baselines in all three settings.
  • Comparison with a jointly optimized visual encoder. The variant CNN-PreShort replaces the frozen pretrained visual encoder with a randomly initialized CNN trained jointly with the world model, while keeping the DSD architecture fixed. Full PreShort performed better on all three benchmarks, supporting the benefit of using frozen pretrained visual features over learning a CNN encoder jointly from scratch.
  • Isolating the contribution of shortcut consistency. PreShort w/o Lsc\mathcal{L}_{\mathrm{sc}} provides a conditional flow matching baseline with the same predictor architecture, parameter count, and four sampling steps. Full PreShort outperformed this baseline on all three benchmarks.
  • Additional baseline. We clarify that, a key distinction between PreShort and JEDI is that PreShort reuses a compressed representation of the trajectory history across denoised-based prediction steps, avoiding the need to repeatedly process the raw history as JEDI does and thereby improving computational efficiency.

Clarifications.

  • Rollout evaluation. Figure 4(a) tracks rollout accuracy during training at a fixed horizon (16 steps). We average PSNR over all prediction steps and sampled trajectories.
  • Pretraining provenance and training cost. PreShort uses the official Wan2.2_VAE.pth checkpoint released with Wan2.2-TI2V-5B without fine-tuning and keeps the encoder frozen throughout online RL. The reported training time covers the full online run, including visual encoding and feature-cache reads and writes.

Thank you for considering our revisions. We are grateful for the community's input and for the opportunity to present our work.

Kind regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.