acceptodds
Withdrawn from ICLR 2027

FlexiWorld: Learning and Planning via Flexible Action Chunks Across Multiple Time Scales

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Latent world models predict future states for goal-directed planning using action chunks spanning multiple primitive steps. Existing methods typically use fixed-length chunks and either omit goal-conditioned action generation or limit their supervision to short goal spans. We introduce FlexiWorld, a JEPA-based world model that combines mixed-span goal supervision with variable-length action chunks to improve long-horizon control. During training, we sample varying goal spans and randomly partition the actions into variable-length chunks. We jointly train the world model with a causal action encoder that embeds variable-length chunks and an autoregressive actor that generates primitive actions sequentially. Student Forcing reduces exposure bias by training on generated action prefixes. For planning, Actor-Residual Cross-Entropy Method (ARCEM) combines action-residual search with within-chunk autoregressive feedback and chunk-boundary latent prediction. Across four benchmarks and goal distances, FlexiWorld with ARCEM achieves 89.29% mean success, compared with 83.98% for the strongest baseline. PushT ablations show improved direct control from mixed-span supervision, variable-length chunks, and Student Forcing. Without retraining, FlexiWorld supports different planning chunk lengths: longer chunks accelerate ARCEM by approximately 1.3× on average while maintaining comparable average success.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
joint-embedding predictive architectures, multi-horizon learning, long-horizon control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_h4LjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose FlexiWorld, a JEPA-based world model that combines mixed-span goal supervision with variable-length action chunks. FlexiWorld jointly learns latent prediction and goal-conditioned action generation for action chunks of different lengths using a variable-length causal action encoder and an autoregressive actor. During planning, FlexiWorld can either perform Direct planning or refine actor-generated plans through action-residual search using ARCEM.

Critical strengths
  1. The authors examine specific limitations of existing JEPA-based world models and propose a method that combines mixed-span goal supervision with variable-length action chunks to address these limitations.
  2. The paper demonstrates that the proposed method achieves a higher average success rate than existing methods across four visual goal-reaching benchmarks. The authors also analyze the effectiveness of key components and the characteristics of the model through various experiments.
Critical weaknesses

Although the authors claim that the use of variable-length action chunks is one of the main contributions of the paper, a major weakness is that its effectiveness has not been clearly validated.

  1. Although FlexiWorld is trained with variable-length action chunks, the evaluation only uses fixed chunk lengths, such as k=5 or k=10. Since the authors do not evaluate scenarios where different chunk lengths are used together during planning, the benefits of variable-length training have not been sufficiently validated.
  2. Performance improvements can be observed when variable-length chunks are used in the single-span setting. However, the effectiveness of variable-length chunks has not been independently validated under mixed-span supervision. Furthermore, since applying mixed-span supervision alone to INTACT improves the success rate from 46.58% to 54.42%, the independent contribution of variable-length chunks to the performance improvement remains unclear. Additional experiments directly comparing fixed-length and variable-length chunks under mixed-span supervision, while maintaining the same action architecture and Student Forcing setting, would be necessary.
  3. FlexiWorld's performance improvement may result from including transitions longer than k=5 during training, rather than from the diversity of chunk lengths. The current ablation study only compares fixed-length chunks with variable-length chunks. An additional comparison with fixed-length chunks using larger values of k would help more clearly isolate the contribution of variable-length chunks.
Questions for authors
  1. Although the model is trained with variable-length chunks, the evaluation only uses fixed chunk lengths, such as k=5 or k=10. It would be interesting to see whether performance is maintained or improved when multiple chunk lengths are mixed or randomly varied within the same trajectory. This would help determine whether the benefits of variable-length training also extend to actual variable-length chunk schedules.
  2. It would be interesting to examine how sensitive the method is to the choice of goal spans S={35,55,75} used during training. It remains unclear whether the performance improvement comes from the diversity of using multiple goal spans or from the specific combination of temporal distances {35,55,75}. To investigate this, it would be useful to evaluate whether the performance improvement is maintained with different span combinations or when goal spans are randomly sampled.
3 comments

Dear Reviewer h4Lj,

Thank you again for your concrete experimental suggestions. In response, we added the fixed-five training control under matched architecture, mixed-span supervision, and Student Forcing; a variable-length training experiment capped at five actions; mixed-duration planning evaluations; and an alternative training-span setting, {40,60,80}. These results are presented in our individual response and the response to all reviewers.

We would appreciate your feedback on whether these additions address your concerns about variable-length training and sensitivity to the chosen goal spans. If any specific concerns remain, could you let us know soon so that we can address them before the discussion period ends?

Thank you for conducting the additional experiments. The new results demonstrate that variable-length training can outperform fixed-length training, which partially addresses my previous concerns.

However, it remains unclear which aspects of variable-length training actually contribute to the performance improvements. In particular, the nearly identical performance of variable-length training with ranges of 1–5 and 1–10 is an interesting observation, but the authors do not provide a sufficient interpretation of this result. Although the authors emphasize the benefits of jointly learning latent dynamics and goal-conditioned action generation across multiple temporal scales, the current experiments do not clearly establish which aspects of this training strategy are responsible for the observed improvements. Given that variable-length training is emphasized as one of the paper's main contributions, it would be helpful to further clarify its distinctive advantages over existing approaches and provide more insight into its technical significance beyond the modification of the training strategy.

Thank you for acknowledging the benefit of variable-length training.

The similar performance of 1–5 and 1–10 shows that the improvement does not require training transitions longer than five actions. Both configurations learn across multiple durations; increasing the maximum training duration need not improve fixed-five evaluation.

The matched fixed-five versus full-method comparison holds architecture, goal spans, Student Forcing, and chunk count constant, yielding a 7.42% gain. Different partitions expose the actor and predictor to different intermediate transitions toward the same goal.

Beyond training, FlexiWorld removes the fixed-duration constraint from action generation and latent prediction. Its variable-length encoder and autoregressive actor allow one trained model to change imagined transition resolution without retraining, as demonstrated by our planning-resolution experiments. This could support robotic deployment with finer transitions when control precision matters, or coarser transitions to reduce planning computation when efficiency is the priority.

Reviewer_h7etOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

FlexiWorld jointly learns a latent world model and a goal-conditioned autoregressive actor using mixed-span goal supervision and variable-length action chunks. After then, student Forcing exposes the actor to its own generated action prefixes during training. During inference, the actor can generate a plan directly or use ARCEM to refine its actions through residual search. These corrections affect subsequent action generation and predictions at chunk boundaries. Across four simulated benchmarks, FlexiWorld improves both direct control and search-based planning.

Critical strengths
  1. The gains are already visible without test-time search. ARCEM further raises success to 89.29%. Actor-free CEM performs similarly with FlexiWorld and INTACT, suggesting that the clearest benefit comes from learned action generation.

  2. The evaluation includes useful ablation studies. The paper examines architecture changes, goal spans, variable chunks, and Student Forcing.

Critical weaknesses

My main concern is the significance of the proposed temporal flexibility. The model supports externally specified chunk lengths, but does not learn which duration suits a given state or goal, or choose it based on expected return or another task-dependent criterion. Prior reinforcement learning work has studied how to learn these choices, including Option-Critic and more recent value-based adaptive action chunking as cited by the paper.

I also recognize that FlexiWorld changes the temporal resolution of imagined dynamics rather than directly adapting physical feedback timing. What is less clear is what general insight we gain from supporting several manually chosen resolutions. When are longer or shorter chunks useful?

The discussion of temporal abstraction also seems to miss relevant prior work, particularly TAP (Efficient Planning in a Compact Latent Action Space, ICLR 2023) and L-MAP (Scalable Decision-Making in Stochastic Environments through Learned Temporal Abstraction, ICLR 2025). Both study efficient planning through learned representations of temporally extended behavior.

Questions for authors

Please see weaknesses.

1 comment

Dear Reviewer h7et,

Thank you again for your thoughtful comments. We would appreciate your assessment of our clarification distinguishing variable-duration modeling from automatic duration selection, together with the new fixed-length and mixed-schedule evaluations. These results characterize the task-dependent trade-off between planning time and success without retraining. We also clarified the methodological differences from TAP and L-MAP in our individual response.

Could you let us know whether these explanations and results address your concerns about the practical significance of temporal flexibility? With the discussion period drawing to a close, we would appreciate hearing about any remaining concerns soon so that we have time to respond.

Reviewer_DrzEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The author trained a model based on JEPA to do long-horizon goal-conditioned planning using flexible action chunks across multiple time scales. It trains a goal-conditioned actor and latent dynamics model using different goal horizons and variable-length action chunks. It also introduces Actor-Residual CEM (ARCEM) to refine generated actions during planning. Experiments across four visual control benchmarks show improved goal-reaching performance over existing methods.

Critical strengths

The experimental performance is great. FlexiWorld consistently outperforms existing baselines across four visual control benchmarks, with notable improvements on PushT and Cube.

Mixed-span training improves goal-conditioned action generation, especially for distant goals, demonstrating better generalization across different goal horizons.

Critical weaknesses

The method combines several components, but the ablations do not sufficiently isolate which ones are responsible for the performance gains. I'm not clear if jointly training local and global goal-conditioned objectives is necessary and which contributes more to the final results.

I like the idea of combining local inverse dynamics and long-horizon goal-conditioned action prediction together, but they are different learning objectives and they are optimized using a shared actor. The paper does not investigate potential conflicts between these objectives or sensitivity to their relative weights.

There is a minor concern about the limited evaluation. Although the method supports variable-length action chunks, inference primarily evaluates fixed chunk lengths (5 and 10).

Questions for authors

See weaknesses

1 comment

Dear Reviewer DrzE,

Thank you again for your time. We would appreciate your feedback on whether our response addresses your concerns about component contributions and the local/global supervision design. We highlighted the original controlled ablations, added a matched fixed-length control under mixed-span supervision, and provided the supporting evidence from INTACT for the inherited supervision scheme. Our response to all reviewers also includes the expanded planning evaluations.

As the discussion period approaches its end, could you let us know which concerns, if any, remain unresolved? An early reply would help us provide any further clarification or targeted analysis while time remains.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.