acceptodds
Under review as a conference paper at ICLR 2027

FLORA: Dual-Latent Discrete World Models with One-step Continuous Flow Matching

Abstract

Discrete latent world models have demonstrated remarkable potential for sample-efficient visual reinforcement learning, enabling agents to learn policies from imagined rollouts in compact latent spaces. However, current methods face two key limitations. First, efficient one-step latent predictors typically use factorized sampling, predicting each token of the latent independently and ignoring inter-token dependencies; dependency-aware alternatives such as MaskGIT-based decoding improve coherence but require costly multi-step refinement during imagination. Second, reconstruction-based encoders preserve visual detail but may allocate capacity to task-irrelevant distractors, while reconstruction-free representations can lose fine-grained information needed for precise control. We propose **FLORA**, a discrete latent world model that addresses both limitations in a unified framework. FLORA introduces a *Denoising Latent Predictor (DP)* trained with continuous flow matching and shortcut forcing, enabling it to model inter-token dependencies in a single step with rollout cost comparable to factorized one-step predictors. It also introduces a *Dual-latent Representation (DR)* that pairs a stochastic reconstructive latent with an auxiliary feature-aligned latent, preserving visual detail while capturing task-relevant abstraction. Across Atari 100k, the DeepMind Control Suite, Meta-World, and Crafter, FLORA outperforms or matches strong model-based baselines. On Atari 100k, FLORA achieves a human-normalized mean score of 170% and a median score of 101%, making FLORA the first world model to surpass human-level median performance without decision-time planning.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, World Models, Model-based RL, Flow Matching

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_feegOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a novel world model architecture for model-based RL. It builds upon the TWISTER method and proposes two main modifications:

  • Denoising Latent Predictor (DP), training a denoising latent predictor head that applies continuous flow matching with shortcut forcing to discrete-latent world modeling, capturing inter-token dependencies in one forward pass.
  • Dual-latent Representation (DR), using two encoder networks for representations. With the default encoder trained using a reconstruction loss and the additional encoder learning representations using Barlow Twins objective.

The paper is evaluated on commonly used model-based RL environments (Atari 100k, DMC, Meta-World and Crafter), achieving superior performance compared to recent alternatives (DreamerV3, DIAMOND, TWISTER, STORM, EMERALD, Delta-IRIS, ...) It also experiment with DMC-GB2 and provide ablation studies on the proposed modifications using Atari-5 (Battle Zone, Double Dunk, Name this Game, Phoenix, and Qbert).

Critical strengths

S1: The paper tackles interesting problems in the field of model-based RL and proposes interesting solutions. The use of two encoders, each with its own representations, instead of fully replacing the representation learning objective is novel.

S2: FLORA demonstrates superior results compared to TWISTER, EMERALD and other model-based approaches on Atari 100k, DMC, Crafter and Meta-World.

Critical weaknesses

W1: EDELINE baseline is missing in experiments and IQM metric for Atari 100k.

The paper does not compare with EDELINE, which achieved a mean/median/IQM score of 186.6%/81.7%/0.940 on Atari 100k. While the Current paper achieves a mean/median of 170%/101%.

For a more informative comparison, I advise the authors to compute IQM scores to compare with EDELINE. FLORA achieves a high score on Gopher, which has a significantly positive impact on the mean score. I would be less concerned if the authors used the IQM metric for comparison to filter outlier tasks like Gopher.

EDELINE also achieves an average return of 11.5 on Crafter (1M frame budget), which is higher than FLORA 10.2 score.

W2: Unclear link between Barlow Twins objective and the encoding of task-relevant information.

The paper claims that the Barlow Twins objective emphasizes the encoder to acquire task-relevant information while the reconstruction objective force the latent to encode task-irrelevant information. However, It is unclear to me how the Barlow twins objective emphasizes task-relevant information. There is no mechanism that selects for task-relevance like a reward objective, a value objective or inverse dynamics objective would specifically requires to encode task relevant information.

W3: Ablations are restricted to only a subset of tasks.

The paper provides ablations for each design choice. However, these ablations are performed on a restricted subset of tasks. Ablations on all Atari 100k games (Table 4) and all DMC-GB2 tasks (Figure 4) instead of a subset would be preferred.

Questions for authors

Q1: What IQM score is achieved by FLORA on Atari 100k?

2 comments

We appreciate the reviewer’s valuable feedback and effort spent on the review and would like to respond to the reviewer’s questions as follows.


Q1. EDELINE comparisons and Atari 100k IQM

Q1.1. EDELINE baseline is missing in experiments and IQM metric for Atari 100k.
The paper does not compare with EDELINE, which achieved a mean/median/IQM score of 186.6%/81.7%/0.940 on Atari 100k. While the Current paper achieves a mean/median of 170%/101%.
For a more informative comparison, I advise the authors to compute IQM scores to compare with EDELINE. FLORA achieves a high score on Gopher, which has a significantly positive impact on the mean score. I would be less concerned if the authors used the IQM metric for comparison to filter outlier tasks like Gopher.
EDELINE also achieves an average return of 11.5 on Crafter (1M frame budget), which is higher than FLORA 10.2 score.
Q1.2. What IQM score is achieved by FLORA on Atari 100k?

Response. We thank the reviewer for the question and would like to clarify the distinction between EDELINE and FLORA. EDELINE [1] is a diffusion-based world model that generates future observations in pixel space through iterative denoising. In contrast, FLORA is a discrete-latent world model that performs imagination entirely in latent space, generating the complete latent representation in a single denoising step without reconstructing observations during imagined rollouts. This distinction enables substantially more efficient world-model-based policy learning.

To provide a direct comparison, Table R7 reports the mean, median, and interquartile mean (IQM) human-normalized scores on Atari 100k, together with the reported training times. We kindly note that the IQM of FLORA is 0.846, as reported in Fig. 8 of Appendix G with stratified bootstrap confidence intervals.

Table R7. Aggregate human-normalized scores (HNS) on Atari 100k and training times of EDELINE and FLORA.

MethodMean HNSMedian HNSIQMTraining Time
EDELINE [1]1.8660.8170.94069.6 h
FLORA1.7001.0100.8465.43 h

FLORA achieves a higher median HNS of 1.010, compared with 0.817 for EDELINE, surpassing human-level median performance without decision-time planning. Since the median is insensitive to extreme scores from individual games, this result demonstrates that FLORA's human-level median performance is not driven by high-scoring outliers such as Gopher. The additional IQM result provides a complementary aggregate evaluation that is less sensitive to extreme scores than the mean.

In terms of computational efficiency, FLORA requires only 5.43 hours for an Atari 100k training run on a single NVIDIA RTX 4090, compared with 69.6 hours reported by EDELINE on the same GPU model, corresponding to approximately 12.8× shorter training time. This efficiency follows from FLORA's compact categorical latent representation and one-step latent generation, avoiding repeated pixel-space denoising during imagination.

Together, these results highlight FLORA's favorable combination of median performance and computational efficiency, while providing the additional IQM comparison requested by the reviewer.

Reference

[1] Lee et al. EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling. NeurIPS, 2025.

Q2. Unclear link between Barlow Twins objective and the encoding of task-relevant information.
The paper claims that the Barlow Twins objective emphasizes the encoder to acquire task-relevant information while the reconstruction objective force the latent to encode task-irrelevant information. However, It is unclear to me how the Barlow twins objective emphasizes task-relevant information. There is no mechanism that selects for task-relevance like a reward objective, a value objective or inverse dynamics objective would specifically requires to encode task relevant information.

Response. We thank the reviewer for the question and would like to clarify that the task-relevant information in the auxiliary latent dtd_t is learned through the world-model prediction objectives, while the Barlow Twins objective serves as a complementary representation regularizer.

Specifically, dtd_t is incorporated into the world-model state st=[zt,dt,ht]s_t=[z_t,d_t,h_t], which is used for reward, continuation, and AC-CPC prediction. Gradients from these objectives propagate to the auxiliary encoder, encouraging dtd_t to retain information useful for predicting rewards, episode continuation, and future latent representations. In particular, the reward prediction objective provides a direct task-relevant learning signal. Furthermore, the dynamics loss Ldyn\mathcal{L}_{\mathrm{dyn}} encourages the latent representation to remain predictable from the action-conditioned history.

The Barlow Twins objective in Eq. (7) complements these prediction objectives by aligning the projected latent features gϕ(dt)g_\phi(d_t) with the image features sg(ut)\mathrm{sg}(u_t) through its diagonal terms, while reducing feature redundancy and discouraging representation collapse through its off-diagonal terms. Unlike the reconstructive latent ztz_t, the auxiliary latent dtd_t is not directly optimized for pixel reconstruction and is therefore not required to preserve task-irrelevant background details.

The representation probing results in Table 2 further support this distinction. On Walker Walk under DMC-GB2 visual distractions, dtd_t achieves a linear-probe MSE of 0.487 and Pearson correlation of 0.620, compared with 0.685 and 0.445 for the reconstruction-oriented latent ztz_t. These results demonstrate that control-relevant simulator-state information is more readily accessible from dtd_t.

Therefore, the world-model prediction objectives provide task-relevant learning signals, whereas Barlow Twins preserves informative, non-collapsed representations without requiring pixel reconstruction. Their roles are complementary rather than redundant.


Q3. Ablations are restricted to only a subset of tasks.
The paper provides ablations for each design choice. However, these ablations are performed on a restricted subset of tasks. Ablations on all Atari 100k games (Table 4) and all DMC-GB2 tasks (Figure 4) instead of a subset would be preferred.

Response. We thank the reviewer for the question. To further evaluate the contribution of DR across different environments, we extend the ablation in Fig. 4 to all six DMC-GB2 tasks, as summarized in Table R8. Specifically, TWISTER and TWISTER+DR are trained and evaluated under the same experimental protocol, with both retaining the AC-CPC objective.

Table R8. Returns of TWISTER and TWISTER+DR on all six DMC-GB2 tasks under the video-hard setting.

TaskTWISTERTWISTER+DR
Cartpole Swingup204.27231.35
Cheetah Run146.10176.33
Cup Catch86.2394.97
Finger Spin238.00461.80
Walker Stand610.17773.31
Walker Walk342.22486.46
Mean271.17370.70

TWISTER+DR outperforms TWISTER on all six tasks, improving the mean return from 271.17 to 370.70, corresponding to a 36.7% relative improvement. The largest improvements are observed on Finger Spin (238.00 to 461.80) and Walker Walk (342.22 to 486.46). These results provide additional evidence that the benefits of DR under visual distractions extend beyond the two representative tasks originally reported in Fig. 4.

Regarding Atari 100k, the component ablations in Table 4 follow the Atari-5 protocol [1], a reduced benchmark consisting of five representative Atari games. Atari-5 was introduced to efficiently estimate aggregate performance on the full 57-game Arcade Learning Environment, with median estimates generally within 10% of the corresponding full-benchmark results. This reduced evaluation protocol enables controlled comparisons across multiple model configurations under the same training budget. The additional full six-task DMC-GB2 ablation complements these results by evaluating DR across a broader range of continuous-control environments.

Reference

[1] Aitchison et al. Atari-5: Distilling the Arcade Learning Environment down to Five Games. ICML, 2023.

Reviewer_m2a2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FLORA is a discrete-latent world model built on TWISTER that targets two limitations of prior latent world models. First, it introduces a Dual-latent Representation (DR) that combines a reconstruction-trained stochastic latent with an auxiliary feature-aligned latent, aiming to retain both visual detail and task-relevant information. Second, it introduces a Denoising Latent Predictor (DP) that replaces the factorized categorical predictor for imagined rollouts with a continuous flow-matching predictor trained with shortcut forcing. The original factorized categorical prior is retained for the KL-based dynamics loss.

The method is evaluated on Atari 100k, DeepMind Control Suite, Meta-World, and Crafter. FLORA improves over or matches strong world-model baselines across these benchmarks. The paper further isolates DR and DP on Atari-5: TWISTER (mean/median HNS: 0.39/0.24) → adding DP 0.47/0.30 → full FLORA (0.51/0.37). The paper also compares factorized TWISTER, iterative MaskGIT, and DP, showing that DP attains prediction accuracy comparable to MaskGIT with substantially lower rollout time.

Critical strengths

DP provides a practical way to remove the factorized-sampling restriction during imagined rollouts. The controlled comparison with MaskGIT shows that DP achieves comparable per-token prediction accuracy at substantially lower rollout cost, which is practically important because latent generation is repeatedly invoked during imagined actor–critic rollouts.

Critical weaknesses

The motivation for DP is under-validated under the paper's own representation-learning objective. Eq. (8) retains a factorized categorical prior and includes an encoder-side KL term that encourages the posterior representation to remain predictable by this factorized prior. Consequently, the aggregated history-conditioned latent distribution qˉϕ(xt∣ht)=Eot∣ht[qϕ(xt∣ot)]\bar q_\phi(x_t\mid h_t)=\mathbb E_{o_t\mid h_t}[q_\phi(x_t\mid o_t)] is under pressure to become more easily represented by ∏ip(xt(i)∣ht)\prod_i p(x_t^{(i)}\mid h_t). This creates a tension with DP's central motivation that substantial conditional inter-token dependence remains to be modeled. The paper neither measures the remaining conditional dependence nor isolates whether DP's gains actually come from modeling it.

Questions for authors
  1. Eq. (8) retains a factorized categorical prior for representation regularization, while DP is trained separately and used for imagination. Does the encoder-side KL in Eq. (8) encourage the aggregated hth_t-conditioned latent distribution to become more easily represented by a factorized distribution?
  2. TWISTER vs. DP changes the training objectives, model forms, noise injection, shortcut forcing, and joint interaction, so the improvement cannot be solely attributed to capturing inter-token dependence. Could the authors compare the proposed joint DP against a capacity-matched factorized DP that retains the same denoising and shortcut objectives but prevents each token prediction from accessing other noisy tokens?
  3. Line 296-297: "cannot capture the inter-token dependencies of the encoder distribution qϕz(zt∣ot)q_\phi^z(z_t\mid o_t)" doesn't seem right. Is the encoder posterior itself standard independent categoricals?
  4. Sec. 4.1 describes TWISTER as "the strongest prior baseline" on Atari 100K. This characterization appears inaccurate. Prior model-based methods without look-ahead/decision-time planning have reported higher mean HNS, including EDELINE (mean/median HNS: 1.866/0.817) and EASimulus (1.818/0.773), compared with TWISTER (1.62/0.77). EDELINE therefore also exceeds TWISTER in median HNS. Why are these stronger and more recent baselines omitted from the Atari 100K comparison? Please clarify the criterion under which TWISTER is considered the strongest prior baseline.

[1] Lee, J. H., Lin, B. J., Sun, W. F., & Lee, C. Y. (2026). Edeline: Enhancing memory in diffusion-based world models via linear-time sequence modeling. Advances in Neural Information Processing Systems, 38, 105587-105626.

[2] Zhao-Han, P., Li, S., Li, Z., Ruan, S., Liu, Y., & He, Y. (2026, April). From Observations to Events: Event-Aware World Models for Reinforcement Learning. In International Conference on Learning Representations (Vol. 2026, pp. 155763-155805).

3 comments

We appreciate the reviewer’s valuable feedback and effort spent on the review and would like to respond to the reviewer’s questions as follows.


Q1. Encoder KL and conditional token dependence

Q1.1. The motivation for DP is under-validated under the paper's own representation-learning objective. Eq. (8) retains a factorized categorical prior and includes an encoder-side KL term that encourages the posterior representation to remain predictable by this factorized prior. Consequently, the aggregated history-conditioned latent distribution qˉϕ(xt∣ht)=Eot∣ht[qϕ(xt∣ot)]\bar{q}_{\phi}(x_t\mid h_t)=\mathbb{E}_{o_t\mid h_t}[q_{\phi}(x_t\mid o_t)] is under pressure to become more easily represented by ∏ip(xt(i)∣ht)\prod_i p(x_t^{(i)}\mid h_t). This creates a tension with DP's central motivation that substantial conditional inter-token dependence remains to be modeled. The paper neither measures the remaining conditional dependence nor isolates whether DP's gains actually come from modeling it.
Q1.2. Eq. (8) retains a factorized categorical prior for representation regularization, while DP is trained separately and used for imagination. Does the encoder-side KL in Eq. (8) encourage the aggregated hth_t-conditioned latent distribution to become more easily represented by a factorized distribution?

Response. We thank the reviewer for the careful examination of the dynamics regularization objective. We would like to clarify that the encoder-side KL term in Eq. (8) encourages latent predictability from the history, but does not enforce conditional independence among tokens in the history-conditioned latent distribution. To examine the remaining dependence, we additionally measure the negative log-likelihood (NLL) gap between factorized and autoregressive predictors on Atari Boxing.

Regarding the encoder-side KL, Eq. (8) regularizes the encoder posterior qϕ(xt∣ot)q_\phi(x_t\mid o_t) toward a factorized categorical prior, with a coefficient of βreg=0.1\beta_{\mathrm{reg}}=0.1. However, the history-conditioned latent distribution is obtained by marginalizing over observations:

\mathbb E_{o_t\mid h_t} \left[ \prod_i q_\phi^{(i)}(x_t^{(i)}\mid o_t) \right].$$ This distribution is a mixture of factorized posteriors and need not factorize across tokens, since the expectation of a product does not generally equal the product of expectations. Furthermore, the KL term is jointly optimized with reconstruction and other world-model objectives, which encourage the latents to retain information from the observations. Therefore, the factorized prior provides a predictability regularizer without imposing a factorization constraint on the aggregated distribution. To evaluate the remaining conditional dependence, we conduct an additional experiment on Atari Boxing. Specifically, we separately train a factorized categorical predictor and an autoregressive (AR) predictor to predict the same 64-token dual latent $x_t=[z_t,d_t]$ conditioned on $h_t$. The factorized predictor independently predicts each token given $h_t$, whereas the AR predictor additionally conditions on preceding tokens. For optimal predictors, the difference in expected NLL equals the conditional total correlation: $$L_F^*-L_{\mathrm{AR}}^*= \sum_i H(X_i\mid H)-H(X\mid H)= \mathrm{TC}(X\mid H).$$ Table R4 reports the NLL results using base-2 logarithms. **Table R4.** NLL of the 64-token latent representation conditioned on history on Atari Boxing (bits; lower is better). | Predictor | NLL (bits) | |---|---:| | Factorized predictor | 56.195 | | Autoregressive predictor | 55.914 | | NLL gap | 0.281 | The AR predictor achieves a lower NLL, with a gap of **0.281 bits per latent sample**. This observed gap provides empirical evidence consistent with residual conditional inter-token dependence in the learned representation, despite the factorized KL regularization. These results support the motivation for retaining the factorized categorical prior for dynamics regularization while employing a more expressive, dependency-aware predictor for imagined latent generation.

Q2. TWISTER vs. DP changes the training objectives, model forms, noise injection, shortcut forcing, and joint interaction, so the improvement cannot be solely attributed to capturing inter-token dependence. Could the authors compare the proposed joint DP against a capacity-matched factorized DP that retains the same denoising and shortcut objectives but prevents each token prediction from accessing other noisy tokens?

Response. To directly isolate the contribution of inter-token dependencies, we conduct an additional controlled experiment comparing the proposed joint DP against a capacity-matched factorized DP. Both predictors retain the same parameter count, noise injection, flow-matching objective, shortcut forcing in Eq. (10), and one-step inference procedure. The only difference lies in whether each token prediction can access other noisy tokens. Specifically, the joint DP predicts each token conditioned on the entire noisy latent, Dϕ(i)(x^tτ,ht,τ,δ)D_\phi^{(i)}(\hat{x}_t^\tau,h_t,\tau,\delta), whereas the factorized DP restricts each prediction to its corresponding noisy token, Dϕ,fact(i)(x^tτ,(i),ht,τ,δ)D_{\phi,\mathrm{fact}}^{(i)}(\hat{x}_t^{\tau,(i)},h_t,\tau,\delta), preventing cross-token conditioning.

We evaluate both variants on six Atari 100k games under the same experimental protocol, with the results summarized in Table R5.

Table R5. Performance comparison between joint DP and capacity-matched factorized DP on six Atari 100k games.

TaskJoint DP (FLORA)Factorized DP
Boxing94.3084.02
Gopher21,178.3019,917.30
James Bond656.80614.50
Kangaroo5,730.704,524.00
Krull6,597.305,811.15
Road Runner18,904.5010,120.50
Mean HNS4.8084.165

Under matched capacity and identical denoising objectives, joint DP outperforms factorized DP on all six games, improving the mean HNS from 4.165 to 4.808. Since both variants share the same flow-matching and shortcut objectives, noise injection, and one-step inference procedure, the performance improvement can be attributed to the ability to condition each token prediction on other noisy tokens, rather than differences in training objectives or model capacity. Furthermore, Fig. 5 of the original manuscript shows that dependency-aware predictors achieve higher per-token prediction accuracy than TWISTER's factorized predictor across the rollout horizon. Together, these results provide direct empirical support for the benefit of modeling inter-token dependencies in latent world-model prediction.


Q3. Line 296-297: "cannot capture the inter-token dependencies of the encoder distribution qϕz(zt∣ot)q_{\phi}^{z}(z_t\mid o_t)" doesn't seem right. Is the encoder posterior itself standard independent categoricals?

Response. We thank the reviewer for the question and would like to clarify that the inter-token dependencies of interest arise in the history-conditioned latent distribution, rather than the encoder posterior conditioned on a given observation. Specifically, the encoder posterior qϕz(zt∣ot)q_\phi^z(z_t\mid o_t) is factorized across categorical tokens given oto_t. However, the distribution predicted from hth_t is obtained by marginalizing the encoder posterior over observations, Eot∣ht[qϕz(zt∣ot)]\mathbb{E}_{o_t\mid h_t}[q_\phi^z(z_t\mid o_t)], which need not factorize. Consequently, TWISTER's factorized predictor cannot capture the inter-token dependencies that arise after this marginalization, whereas DP does not impose this restriction.

We have revised Section 3.3 to make this distinction explicit:

"Although the encoder posterior qϕz(zt∣ot)q_\phi^z(z_t\mid o_t) factorizes given oto_t, its history-conditioned aggregate Eot∣ht[qϕz(zt∣ot)]\mathbb{E}_{o_t\mid h_t}[q_\phi^z(z_t\mid o_t)] need not. Consequently, TWISTER's factorized predictor cannot capture inter-token dependencies arising from this marginalization."

Q4. Sec. 4.1 describes TWISTER as "the strongest prior baseline" on Atari 100K. This characterization appears inaccurate. Prior model-based methods without look-ahead/decision-time planning have reported higher mean HNS, including EDELINE (mean/median HNS: 1.866/0.817) and EASimulus (1.818/0.773), compared with TWISTER (1.62/0.77). EDELINE therefore also exceeds TWISTER in median HNS. Why are these stronger and more recent baselines omitted from the Atari 100K comparison? Please clarify the criterion under which TWISTER is considered the strongest prior baseline.

Response. We appreciate the reviewer's careful examination of the Atari 100k comparison. The phrase "the strongest prior baseline" in Sec. 4.1 referred to the methods included in Table 3, with TWISTER providing the most direct architectural comparison for FLORA. To make this distinction more precise, we have revised the wording to "a competitive prior baseline."

To further contextualize FLORA's performance against recent world-model methods, we additionally compare with EDELINE [1] and EASimulus [2], as shown in Table R6. EDELINE employs diffusion-based world modeling in pixel space with state-space sequence modeling, while EASimulus extends token-based world modeling with event-aware prediction. In contrast, FLORA operates on stochastic categorical latents and employs one-step continuous flow matching for dependency-aware latent generation, enabling efficient imagination-based policy learning.

Table R6. Mean and median human-normalized scores (HNS) on Atari 100k and reported training times.

MethodMean HNS ↑Median HNS ↑Training Time ↓
TWISTER1.6200.7705.05 h
EDELINE [1]1.8660.81769.6 h
EASimulus [2]1.8180.77319.2 h
FLORA1.7001.0105.43 h

FLORA achieves the highest median HNS among the methods in Table R6, reaching 1.010 compared with 0.817 for EDELINE and 0.773 for EASimulus, and surpasses human-level median performance without decision-time planning. In terms of computational efficiency, FLORA requires 5.43 hours of training on an NVIDIA RTX 4090, compared with 69.6 hours for EDELINE and 19.2 hours for EASimulus, corresponding to approximately 12.8× and 3.5× shorter training times, respectively.

These comparisons highlight FLORA's favorable combination of median performance and computational efficiency, while the comparison with TWISTER provides a direct architectural baseline for evaluating the proposed DR and DP components. We have revised Sec. 4.1 to more precisely characterize TWISTER and FLORA's performance relative to prior methods.

References

[1] Lee et al. EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling. NeurIPS, 2025.

[2] Peng et al. From Observations to Events: Event-Aware World Models for Reinforcement Learning. ICLR, 2026.

Reviewer_iZ9jOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses limitations of prior work, TWISTER, a discrete latent variable world model for agent behaviour learning in imagination. The authors observe that discrete latent variable models factorise (no token dependency) latent prediction and suffer from a reconstruction-control-relevant information retraining tradeoff. The authors propose 1) Dual Latent Representation (DR) to essentially retain both reconstruction and abstraction channels and 2) Dual Latent Predictor (DP) with flow matching that allows for parallel predictor modelling of inter-token dependencies. The proposed model is evaluated on both standard discrete control and continuous control RL environments and shows very strong results, including surpassing human-level median performance on Atari100k.

Critical strengths
  1. The paper identifies two limitations of prior discrete-latent world models: 1) reconstruction-abstraction tradeoff that matters for control and 2) lack of token dependencies in the factorised predictions. They offer solutions to both: 1) adding a representation loss inspired by SSL literature (Barlow Twins) via introducing an additional encoder and 2) replacing factorised sampling with a technique from few-step flow-matching (shortcut models) that predicts together in one step in parallel. I think this observation is clear, and the solution is reasonable. Both additions are well justified, with careful ablation and analysis showing they are effective on top of the base TWISTER model.

  2. I find the overall experimentation to be thorough, covering standard RL benchmarks from both discrete and continuous control benchmarks. Beyond the main results where performance is strong. The authors did representation probing, careful investigation with prior work such as MaskGIT for modeling inter-token dependencies and also reasonable ablations of added modules.

  3. The proposed model FLORA achieves (or matches) state-of-the-art performance against prior efficient model-based RL baselines.

Critical weaknesses
  1. My main challenge with the paper is the overwhelming complexity when reading it. I find it difficult to follow the material. Perhaps because the proposed model is indeed complicated. At its core, there are essentially two encoders: discrete VAE z_t and continuous latent d_t. It’s fed to a TSSM (which is a Transformer version of Dreamer’s RSSM) that models the memory state h_t. The predictor (perhaps the dynamics predictor?) predicts the latents in the future step using one-step flow matching while receiving h_t and both input latents. As in Twister, which identifies that TSSM lacks representation capabilities, there’s an AC-CPC loss which improves high-level temporal feature representation. That’s how I was able to read the paper. There are two representation problems following TWISTER. One, as identified in this paper, is the encoded latent itself (hence, the proposed dual latent), and the other issue is the TSSM model (hence, the AC-CPC objective in TWISTER). So, it seems to me that the method as proposed is tackling a problem that has also been discussed a lot in the simpler architecture JEPA world model [1][2][3], and was not discussed at all in this paper. The writing was quite confusing in multiple places, such as: 1) there’s a loss “l_dyn”, but it’s not clear what the dynamics function is; 2) violation of standard variable naming, such as x_t is generally, in prior diffusion works, considered a raw observation, but in this case it’s the combined latent. What’s the input/output to the L-dyn given that L-dp is the one now predicting latents, replacing the dynamics predictor? I think the paper could’ve been more readable if the encoders (VAE and SSL encoders) were separated from the world model (TSSM and its predictors).

  2. FLORA makes two changes to TWISTER; one of them concerns learned representation in the observation encoder. My understanding of TWISTER is that it’s also trying to learn representations, but inside the TSSM by modelling a multi-step predictive future for the world model “state”. So, from the representation learning perspective, it’d be interesting to know whether the encoder is the main limiting factor, by ablating the AC-CPC objective from FLORA.

  3. FLORA has two different objectives for two purposes serving representation learning. The observation encoder is trained with the Barlow Twins objective, and the TSSM that models world state is trained with contrastive AC-CPC. This makes the design complicated, as I mentioned already. Since the auxiliary encoder is a critical component, it leaves the reader with the question of whether a contrastive training objective could be more suitable and more unified in terms of design. The paper doesn’t justify the design or discuss other choices that could have been made for the encoder.

Questions for authors
  1. In Figure 3, it seems to me that the VAE latent z_t, which is responsible for reconstruction, is reconstructing worse than the latent d_t, whose job is motivated for abstraction. But, visually, it doesn’t seem so!

  2. Of course, with such a complex architecture design, one natural question is how scalable this model is. For example, if both TWISTER and FLORA scaled to a bigger size, e.g., doubled param size, on tasks such as Crafter/DMC, will the relative gap remain?

  3. Since the encoder has gotten better with dual latent, do we need AC-CPC anymore? Did the author try dropping AC-CPC component?

  4. This is not a question per se, but rather a suggestion on the presentation. At multiple places, the paper reads like a long list without substance. For example, L081-L084, the paper lists a set of baselines and names them one by one. What do those baselines represent thematically (such as discrete space, recurrent state-space models, transformers, etc.)? The same goes for benchmarks: what type of tasks do those benchmarks represent? I think there’s a lack of rationale / argumentative order in multiple places, which hurts the readability of the paper.

4 comments

We appreciate the reviewer’s valuable feedback and effort spent on the review and would like to respond to the reviewer’s questions as follows.


Q1. At its core, there are essentially two encoders: discrete VAE z_t and continuous latent d_t. It’s fed to a TSSM (which is a Transformer version of Dreamer’s RSSM) that models the memory state h_t. The predictor (perhaps the dynamics predictor?) predicts the latents in the future step using one-step flow matching while receiving h_t and both input latents. As in Twister, which identifies that TSSM lacks representation capabilities, there’s an AC-CPC loss which improves high-level temporal feature representation. That’s how I was able to read the paper. There are two representation problems following TWISTER. One, as identified in this paper, is the encoded latent itself (hence, the proposed dual latent), and the other issue is the TSSM model (hence, the AC-CPC objective in TWISTER). So, it seems to me that the method as proposed is tackling a problem that has also been discussed a lot in the simpler architecture JEPA world model [1][2][3], and was not discussed at all in this paper. The writing was quite confusing in multiple places, such as: 1) there’s a loss “l_dyn”, but it’s not clear what the dynamics function is; 2) violation of standard variable naming, such as x_t is generally, in prior diffusion works, considered a raw observation, but in this case it’s the combined latent. What’s the input/output to the L-dyn given that L-dp is the one now predicting latents, replacing the dynamics predictor? I think the paper could’ve been more readable if the encoders (VAE and SSL encoders) were separated from the world model (TSSM and its predictors).

Response. We thank the reviewer for the comments and would like to clarify four aspects of FLORA's design: (1) the architecture and the roles of its components, (2) the distinction between L_dyn and L_dp, (3) the notation used for flow matching, and (4) the relationship to JEPA-based world models.

First, FLORA extends TWISTER through two components: DR improves observation-level representation, while DP enables dependency-aware latent generation during imagination. The TSSM and AC-CPC objective are inherited from TWISTER. Both the reconstructive latent ztz_t and the feature-aligned latent dtd_t consist of discrete one-hot categorical tokens (Sec. 3.2). The TSSM summarizes past latents and actions into the history representation hth_t, while DP generates the next dual latent from hth_t and Gaussian noise in a single denoising step during imagination. Thus, the current observation and its encoded latents are not required as inputs during imagination. DR concerns the information encoded from the current observation, whereas AC-CPC encourages the world-model state to retain information predictive of future latent representations.

Second, L_dyn and L_dp serve distinct purposes. Following TWISTER, FLORA retains the factorized categorical prior pϕ(x^t∣ht)p_\phi(\hat{x}_t\mid h_t), which takes the history hth_t as input and outputs a categorical distribution for each token of xt=[zt,dt]x_t=[z_t,d_t].

As defined in Eq. (8), L_dyn computes the KL divergence between this prior and the encoder posterior qϕ(xt∣ot)q_\phi(x_t\mid o_t), encouraging both encoders to produce latents that are predictable from the history. We kindly note that this prior is used solely for dynamics regularization and is not sampled during imagination. Instead, DP is trained through L_dp in Eq. (10) and serves as the sole latent generator during imagined rollouts.

Third, xt=[zt,dt]x_t=[z_t,d_t] denotes the dual latent at environment timestep tt, while x^tτ\hat{x}_t^\tau denotes its noisy interpolation at denoising time τ\tau. Flow matching therefore operates on the dual latent rather than directly on raw observations, which are consistently denoted by oto_t throughout the paper.

Fourth, DINO-WM [1], V-JEPA 2 [2], and LeWorldModel [3] employ continuous visual embeddings and demonstrate control through latent-space planning. In contrast, FLORA learns stochastic categorical representations and trains an actor–critic policy through imagined rollouts. We have added a dedicated discussion of these distinctions in Appendix C.3.

To improve readability, we have further clarified the separation between observation encoding and temporal modeling, as well as the inputs and roles of the factorized prior and DP, in Sec. 3.

References

[1] Zhou et al. DINO-WM: World Models on Pre-trained Visual Features enable Zero-shot Planning. ICML, 2025.

[2] Assran et al. V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning. arXiv:2506.09985, 2025.

[3] Maes et al. LeWorldModel: Stable End-to-End Joint-Embedding Predictive Architecture from Pixels. arXiv:2603.19312, 2026.

Q2. AC-CPC ablation

Q2.1. FLORA makes two changes to TWISTER; one of them concerns learned representation in the observation encoder. My understanding of TWISTER is that it’s also trying to learn representations, but inside the TSSM by modelling a multi-step predictive future for the world model “state”. So, from the representation learning perspective, it’d be interesting to know whether the encoder is the main limiting factor, by ablating the AC-CPC objective from FLORA.
Q2.2. Since the encoder has gotten better with dual latent, do we need AC-CPC anymore? Did the author try dropping AC-CPC component?

Response. We thank the reviewer for the question. To directly examine whether AC-CPC remains necessary after introducing DR, we conduct an additional ablation on six Atari 100k games, removing AC-CPC from FLORA while retaining DR and DP. As shown in Table R1, removing AC-CPC reduces performance on all six games, decreasing the mean HNS from 4.808 to 2.539. The largest decreases occur on Gopher, Kangaroo, and Road Runner.

Table R1. Performance of FLORA with and without AC-CPC on six Atari 100k games.

GameFLORAFLORA w/o AC-CPC
Boxing94.3081.51
Gopher21,178.303,315.50
James Bond656.80510.00
Kangaroo5,730.701,767.00
Krull6,597.304,819.70
Road Runner18,904.5013,191.50
Mean HNS4.8082.539

The performance degradation reflects the distinct roles of DR and AC-CPC. DR complements the reconstructive latent ztz_t with the feature-aligned latent dtd_t, improving the information encoded from the current observation. In contrast, AC-CPC operates on the world-model state st=[xt,ht]s_t=[x_t,h_t] and encourages it to capture information predictive of future latent representations through action-conditioned multi-step contrastive prediction. Thus, improving the observation-level representation through DR does not replace the temporal predictive learning provided by AC-CPC.

These results demonstrate that AC-CPC remains an important component of FLORA and provides substantial benefits alongside the proposed dual-latent representation.

Q3. FLORA has two different objectives for two purposes serving representation learning. The observation encoder is trained with the Barlow Twins objective, and the TSSM that models world state is trained with contrastive AC-CPC. This makes the design complicated, as I mentioned already. Since the auxiliary encoder is a critical component, it leaves the reader with the question of whether a contrastive training objective could be more suitable and more unified in terms of design. The paper doesn’t justify the design or discuss other choices that could have been made for the encoder.

Response. We thank the reviewer for the question and would like to clarify that Barlow Twins and AC-CPC serve distinct purposes in FLORA. AC-CPC operates on the world-model state st=[xt,ht]s_t=[x_t,h_t] and employs action-conditioned contrastive prediction to encourage the state to retain information predictive of future latent representations. In contrast, the Barlow Twins objective Ldr\mathcal{L}_{\mathrm{dr}} operates on the auxiliary latent of the current observation.

Following R2-Dreamer [1], Barlow Twins is adopted to align the projected latent gϕ(dt)g_\phi(d_t) with the image features of the same observation, while reducing redundancy across feature dimensions and preventing representation collapse. Unlike contrastive objectives that rely on negative samples, Barlow Twins provides feature alignment and redundancy reduction without negative sampling or pixel reconstruction.

To examine the complementary roles of these objectives, we conduct two additional experiments. First, we extend the DR ablation in Fig. 4 to all six DMC-GB2 tasks, as shown in Table R2. Both TWISTER and TWISTER+DR retain AC-CPC, while TWISTER+DR additionally incorporates the proposed dual-latent representation.

Table R2. Returns of TWISTER and TWISTER+DR on all six DMC-GB2 tasks under the video-hard setting.

TaskTWISTERTWISTER+DR
Cartpole Swingup204.27231.35
Cheetah Run146.10176.33
Cup Catch86.2394.97
Finger Spin238.00461.80
Walker Stand610.17773.31
Walker Walk342.22486.46
Mean271.17370.70

TWISTER+DR outperforms TWISTER on all six tasks, increasing the mean return from 271.17 to 370.70, corresponding to a 36.7% relative improvement. Since TWISTER already incorporates AC-CPC, these results demonstrate that temporal predictive representation learning alone does not provide the observation-level representation benefits introduced by DR.

Second, the AC-CPC ablation in our response to Q2 (Table R1) shows that removing AC-CPC from FLORA reduces performance on all six evaluated Atari games, despite retaining DR. Together, these results demonstrate that Barlow Twins and AC-CPC address different aspects of representation learning and provide complementary benefits. The former preserves informative observation-level representations without pixel reconstruction, whereas the latter strengthens action-conditioned temporal prediction.

Reference

[1] Morihira et al. R2-Dreamer: Redundancy-Reduced World Models without Decoders or Augmentation. ICLR, 2026.


Q4. In Figure 3, it seems to me that the VAE latent z_t, which is responsible for reconstruction, is reconstructing worse than the latent d_t, whose job is motivated for abstraction. But, visually, it doesn’t seem so!

Response. We thank the reviewer for the question and would like to clarify that Fig. 3 illustrates the information retained by different latent representations, rather than comparing their overall pixel reconstruction accuracy. Specifically, the three decoded rows are obtained using post-hoc decoders trained on the corresponding frozen representations, while the first row shows the ground-truth observation.

Under the visual distractions of DMC-GB2, the reconstruction objective encourages ztz_t to preserve both task-relevant objects and background details. Consequently, the frames decoded from ztz_t retain the overall scene appearance, but the agent is less distinguishable from the background. In contrast, dtd_t is not directly trained for pixel reconstruction and places greater emphasis on control-relevant information, preserving the agent while retaining fewer background details. The combined representation xt=[zt,dt]x_t=[z_t,d_t] recovers both the scene and the agent, as illustrated in the fourth row of Fig. 3.

Table 2 further provides quantitative evidence of this distinction. On Walker Walk under DMC-GB2 visual distractions, the linear probe achieves an MSE of 0.487 and Pearson correlation of 0.620 using dtd_t, compared with 0.685 and 0.445 using ztz_t. The combined representation [zt,dt][z_t,d_t] achieves the best results, with an MSE of 0.483 and correlation of 0.622. These results support the complementary roles of the reconstructive and feature-aligned latents in DR.

Q5. Of course, with such a complex architecture design, one natural question is how scalable this model is. For example, if both TWISTER and FLORA scaled to a bigger size, e.g., doubled param size, on tasks such as Crafter/DMC, will the relative gap remain?

Response. We thank the reviewer for the question. To investigate the scalability of FLORA, we conduct an additional experiment on two visual DeepMind Control tasks, Quadruped Run and Quadruped Walk, scaling both FLORA and TWISTER to approximately twice their original parameter counts. Specifically, we increase the hidden dimension to 768, CNN channels to 48, Transformer depth to six blocks, and attention heads to 12. This increases FLORA from 57M to 115M parameters and TWISTER from 41M to 86M parameters.

Table R3. Performance of TWISTER and FLORA at their default and approximately doubled model sizes on two DeepMind Control Suite tasks.

ModelParametersQuadruped RunQuadruped Walk
TWISTER41M652.10904.90
TWISTER (scaled)86M649.30906.30
FLORA57M716.80867.00
FLORA (scaled)115M748.50860.95

As shown in Table R3, on Quadruped Run, scaling FLORA improves its return from 716.80 to 748.50, whereas TWISTER maintains a similar performance (652.10 vs. 649.30). Consequently, FLORA's advantage over TWISTER increases from 64.7 to 99.2 points, demonstrating that its benefit persists at the larger model size.

On Quadruped Walk, the performance of both methods changes by less than 1% after scaling, and their relative ordering remains unchanged. These results indicate that the performance differences observed between FLORA and TWISTER persist under the evaluated increase in model capacity, with FLORA's advantage on Quadruped Run becoming more pronounced.


Q6. This is not a question per se, but rather a suggestion on the presentation. At multiple places, the paper reads like a long list without substance. For example, L081-L084, the paper lists a set of baselines and names them one by one. What do those baselines represent thematically (such as discrete space, recurrent state-space models, transformers, etc.)? The same goes for benchmarks: what type of tasks do those benchmarks represent? I think there’s a lack of rationale / argumentative order in multiple places, which hurts the readability of the paper.

Response. We thank the reviewer for the comment. To clarify the rationale behind our experimental comparisons, we have revised Appendix D to organize the baselines by their world-modeling approaches and explain the distinct purposes of the benchmarks.

Specifically, the baselines represent different world-model designs: DreamerV3 employs recurrent latent dynamics, STORM and TWISTER are Transformer-based latent world models, TWM and IRIS use autoregressive token prediction, EMERALD employs iterative masked-token prediction, and SimPLe and DIAMOND operate in pixel space, with DIAMOND using diffusion-based generation. These comparisons cover different choices in representation space, temporal modeling, and dependency-aware latent prediction.

The benchmarks similarly evaluate complementary aspects of visual control. Atari 100k tests sample-efficient discrete control, the DeepMind Control Suite evaluates continuous control from pixels, Meta-World examines robotic manipulation, and Crafter evaluates long-horizon exploration under partial observability. DMC-GB2 further investigates robustness under visual distractions, directly relating to the fidelity–relevance trade-off addressed by DR.

These revisions make the rationale behind the baseline selection and benchmark coverage explicit in Appendix D, improving the organization and clarity of the experimental comparisons.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.