acceptodds
Withdrawn from ICLR 2027

Loop LeWorldModel: Reliable Recursion for Depth Reuse and Temporal Rollout in Latent World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual world models based on joint embedding predictive architectures (JEPAs) support planning by predicting action consequences before execution. In real-world deployment, compact and reliable end-to-end world models are increasingly needed under limited computational and memory budgets. Recursive layer sharing offers a natural way to reduce model size. However, naively looping pretrained layers across depth erases depth-specific computations, and recursively feeding predicted states back often accumulates errors and destabilize imagined trajectories. To address these two issues, we propose Loop LeWorldModel, a loop-based adaptation of pretrained visual world models. Specifically, we utilize depth-specific low-rank adapters to preserve distinct transformations at each loop. Additionally, we design a training-free self-correction gate to suppress abnormal prediction steps using recent motion statistics. Extensive experiments on various visual-control tasks demonstrate that Loop LeWorldModel can significantly improve planning success with fewer parameters and less accumulated errors than the original model. All source codes will be released online.

Primary area:
foundation or frontier models, including LLMs
Keywords:
World Models, Looped Transformers, Post-Training, Joint-Embedding Predictive Architectures, Self-Correction

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_JkRCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper converts a pretrained LeWorldModel into a recurrent-depth JEPA while preserving its original effective encoder and predictor depths. It averages pretrained layers into shared bases, initializes depth-specific low-rank adapters from truncated-SVD residuals, and jointly uptrains the adapters and predictor AdaLN with prediction, SIGReg, and teacher-distillation losses. It also proposes a training-free gate that compares predicted latent displacements with a recent lower-median motion scale, then softly clamps and blends unusually large steps. On PushT, TwoRooms, Cube, and Reacher with CEM planning, the paper reports that the loop model uses 43–47% fewer stored parameters than the official model while modestly improving ungated mean success; the gate further improves planning for both looped and untied models.

Critical strengths
  1. The method shares weights across layers to reduce stored parameters, while using depth-specific low-rank adapters to approximate the differences between the original pretrained layers. Figure 3 and Table 2 support the importance of recovery training and retaining these adapters. Table 1 reports that one configuration reduces stored parameters by 43–47% while improving mean success by 1.8 points over Official, although this result alone does not establish a practical deployment benefit.

  2. The Self-Correction Gate is training-free and can be applied independently of depth sharing. With fixed gate settings, Table 1 reports a higher cross-task mean for all eight Loop schedules, while Table 4 shows gains on all four tasks for the untied Official model. Figure 2(b) also reports lower 95th-percentile latent step magnitudes during free rollouts.

Critical weaknesses
  1. The practical motivation and efficiency case are under-established. The paper reports 43–47% fewer stored parameters, but does not identify a concrete memory constraint this resolves or measure whether the conversion reduces inference FLOPs, peak memory, prediction latency, or end-to-end CEM planning time. Since effective encoder/predictor depth is preserved and depth-specific adapters plus the gate add computation, parameter reduction alone cannot establish faster or more deployable planning. The modest mean-success gain is promising, but its robustness and runtime trade-off are not quantified.

  2. The gate improves planning success, but its effect on accumulated prediction error remains unclear. In the main five-step planning setup, the gate can first act on the fifth and final prediction (Section 4.1). It can therefore change which action sequence the planner selects, but no later prediction within that sequence uses the corrected state. Figure 2(b) shows smaller latent steps in longer rollouts; smaller steps alone do not show that predictions are closer to the true trajectory. The evidence should therefore more clearly establish whether feeding corrected states into subsequent predictions improves accuracy, beyond the demonstrated planning benefit.

  3. Presentation could be more concise. The reference list includes URLs for nearly every entry, making it visually dense. The authors should check the venue’s bibliography style and remove links that are unnecessary under that style.

  4. Some elementary steps are over-formalized in the main text. For example, Equations (7) and (8) express the gate’s thresholding and displacement clamp, which could first be explained more simply in prose or pseudocode. Keeping the exact definitions for reproducibility while moving some implementation detail to an appendix would make the central idea easier to follow.

Questions for authors
  1. What concrete deployment constraint is the 43–47% reduction in stored parameters intended to address? Please report or clarify the corresponding inference FLOPs, peak memory, prediction latency, and end-to-end planning time for the compared models, including the overhead of adapters and the gate, and explain whether the reported success-rate gain persists at comparable runtime or resource budgets.

  2. Does feeding corrected states into subsequent predictions reduce prediction error over longer rollouts, compared with leaving those states uncorrected? The main five-step planning experiment cannot establish this because the gate can first act on the final prediction. Please clarify whether the controls in Appendix F.2 provide evidence of this benefit.

  3. Could you review the formal presentation throughout the paper and simplify equations where a concise explanation would be clearer?

Reviewer_PqDXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to convert a pretrained visual JEPA world model (LeWorldModel) into a looped architecture after training, and how to make latent planning more reliable. The method has two components. The first is a Loop Encoder/Predictor: layers assigned to the same slot share a frozen group-mean weight, and each effective depth is given a LoRA adapter initialized from the SVD of its residual. Only the adapters and the per-depth predictor AdaLN parameters are then trained, using the original LeWorldModel objective plus distillation from the original model. The second is a training-free Self-Correction Gate: when a predicted step is noticeably larger than the median of recent step sizes, it is shortened along its original direction. Experiments are conducted on PushT, TwoRooms, Cube, and Reacher. For the (4,3;2,3) configuration, the looped model has 10.3M parameters (vs. 18.0M for the original), reaches an average CEM success rate of 82.4% without the gate (vs. 80.6% for the original), and 87.2% with the gate. The gate also improves success rates when applied to the original model alone.

Critical strengths
  1. The method is simple and easy to implement. The Self-Correction Gate requires no training and, when added directly to the original model, yields clear gains on TwoRooms (81.0% → 91.5%) and Reacher (79.0% → 89.0%). This suggests that handling atypical rollout steps in latent planning may be a direction worth further attention.

  2. The experimental effort is fairly substantial: 8 sharing schedules, 3 training seeds, 4 CEM seeds, and several ablations on the conversion procedure. The appendix also transparently reports some unfavorable results and limitations, which helps readers assess the reliability of the conclusions.

Critical weaknesses
  1. The motivation for and practical benefit of the looped conversion could be further clarified. The paper starts from the observation that planning repeatedly applies the predictor over time, and uses this to motivate looping layers across depth inside the model. However, the connection between the two does not seem to be fully argued. The main benefit of the conversion is a reduction in stored parameters (18.0M → 10.3M); for a model of this size, the practical significance of this saving may need more explanation. Since the effective depth and the CEM settings are unchanged, inference and planning cost are not expected to decrease, and the paper does not report wall-clock time, FLOPs, or memory. In terms of planning performance, the average gain without the gate is +0.4 to +1.8 points, which is of a similar magnitude to the original model's variation across CEM seeds (standard deviation 3.8 to 6.2). In addition, the looped models receive 5 extra epochs of training with distillation, without a corresponding control for the original model, so it is difficult to tell how much of this gain comes from the looped structure itself. The conversion procedure also largely follows Relaxed Recursive Transformers (Bae et al., 2025). Taken together, these factors make the contribution of this part appear relatively limited.

  2. The mechanism behind the Self-Correction Gate remains unclear. The paper explains the gate as correcting accumulated autoregressive errors. However, the free-rollout diagnostics in Appendix F.1 show that, with the gate, the final-step error against encoded observations increases on all four tasks (e.g., from 0.0886 to 0.5777 for Loop on Cube), which does not seem fully consistent with this explanation. The reduction in step size in Fig. 2(b) is, to some extent, a direct consequence of the clamping rule. Moreover, with one initial observation and a planning horizon of 5, the gate only acts on the terminal prediction during planning, and the authors note in Appendix C.2 that it influences action selection through the terminal cost. It therefore seems that a convincing explanation of why the gate improves planning success is still missing. Since most of the paper's main improvement (+6.6 / +6.8) comes from the gate, this point weighs considerably in my overall assessment.

Questions for authors

The following are clarification questions:

  1. Could the authors report a comparison of planning time, FLOPs, and memory between the original and looped models? At this model scale, what practical problem does reducing stored parameters primarily address?

  2. Appendix F.1 shows that prediction error increases with the gate. How do the authors understand the gate's improvement in planning success? Beyond modifying the terminal cost, is there evidence that the gate actually corrects errors?

  3. How much of the one-to-two-point gain of the looped models can be attributed to the looped structure itself, and how much to the additional 5 epochs of training and distillation?

Reviewer_ksgKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work introduces a loop architecture into JEPA world models (LeWorldModel) which allows for reduction of the parameters of the model without losing accuracy. In addition, a Self-Correction Gate is proposed to correct large jumps when transitioning between states in latent space in order to reduce the temporal drift in sequences of transitions. Evaluation shows comparable performance with reduced number of parameters of the loop architecture and improvements from the gate architecture.

Critical strengths

The proposed gate is training-free and is empirically well demonstrated to generate improvements in multiple experiments.

Critical weaknesses
  • Motivation for the loop architecture - The paper describes the loop idea as established in language models as well as extended to world models. The described contribution is then bringing this established idea into an existing JEPA model (LeWorldModel). The described benefit is decreasing the number of parameter of the models while preserving comparable to the original model or slightly better performance. However, the significance and motivation for this contribution is not well established. The reduction of weight parameters does not translate to reduction of compute operations as the parameters are reused for the same depth. Furthermore, LeWorldModel is already established as a very light model of a few million parameters. Reducing its number of parameters for storage measures in just tens of megabytes. It is not clear why the number of parameters is a limitation of these already known to be light models in the first place and it is not clear what the technical challenge is of bringing the loop idea in JEPA world models as a similar architecture has been established in Loop World Models (for much larger models).
  • Design of Self-Correction Gate - While temporal drift is indeed an issue in JEPA-space predictors, the paper does not motivate well the design of their proposed Self-Correction Gate as a solution to that problem. The gate entirely operates on the magnitude of a step - modification of this magnitude is made based on historical magnitude. It is not established why the magnitude of a step is a telling sign for a drift, why a large magnitude means an incorrect transition or why smoothness of step magnitude is assumed in this unknown latent space manifold. For example, can a change of action throughout the sequence suddenly influence the magnitude? The paper presents experiments in which low magnitude is used as a metric for success - however it still remains unclear why and how a smaller magnitude translates to better performance. Evaluation establishes that the gate leads to empirical improvements of the success rate but it does not show the connection between these improvements and the design choice. Therefore, it is not clear if this specific design is necessary for the improvements or a simple technique like smoothing over the predicted and previous state is already enough.
Questions for authors
  • What is the motivation to bring the loop architecture to JEPA architectures? What is the motivation to reduce number of parameters for these models?
  • What is the challenge of bringing the loop architecture to LeWorldModel compared to Loop World Models or related literature?
  • What is the source of improvement of introducting the Self-Correction Gate? Can you show this experimentally? Can you establish that your component design is necessary for the improvement rather than using a simpler alternative?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.