- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a novel world model architecture for model-based RL. It builds upon the TWISTER method and proposes two main modifications:
- Denoising Latent Predictor (DP), training a denoising latent predictor head that applies continuous flow matching with shortcut forcing to discrete-latent world modeling, capturing inter-token dependencies in one forward pass.
- Dual-latent Representation (DR), using two encoder networks for representations. With the default encoder trained using a reconstruction loss and the additional encoder learning representations using Barlow Twins objective.
The paper is evaluated on commonly used model-based RL environments (Atari 100k, DMC, Meta-World and Crafter), achieving superior performance compared to recent alternatives (DreamerV3, DIAMOND, TWISTER, STORM, EMERALD, Delta-IRIS, ...) It also experiment with DMC-GB2 and provide ablation studies on the proposed modifications using Atari-5 (Battle Zone, Double Dunk, Name this Game, Phoenix, and Qbert).
Critical strengths
S1: The paper tackles interesting problems in the field of model-based RL and proposes interesting solutions. The use of two encoders, each with its own representations, instead of fully replacing the representation learning objective is novel.
S2: FLORA demonstrates superior results compared to TWISTER, EMERALD and other model-based approaches on Atari 100k, DMC, Crafter and Meta-World.
Critical weaknesses
W1: EDELINE baseline is missing in experiments and IQM metric for Atari 100k.
The paper does not compare with EDELINE, which achieved a mean/median/IQM score of 186.6%/81.7%/0.940 on Atari 100k. While the Current paper achieves a mean/median of 170%/101%.
For a more informative comparison, I advise the authors to compute IQM scores to compare with EDELINE. FLORA achieves a high score on Gopher, which has a significantly positive impact on the mean score. I would be less concerned if the authors used the IQM metric for comparison to filter outlier tasks like Gopher.
EDELINE also achieves an average return of 11.5 on Crafter (1M frame budget), which is higher than FLORA 10.2 score.
W2: Unclear link between Barlow Twins objective and the encoding of task-relevant information.
The paper claims that the Barlow Twins objective emphasizes the encoder to acquire task-relevant information while the reconstruction objective force the latent to encode task-irrelevant information. However, It is unclear to me how the Barlow twins objective emphasizes task-relevant information. There is no mechanism that selects for task-relevance like a reward objective, a value objective or inverse dynamics objective would specifically requires to encode task relevant information.
W3: Ablations are restricted to only a subset of tasks.
The paper provides ablations for each design choice. However, these ablations are performed on a restricted subset of tasks. Ablations on all Atari 100k games (Table 4) and all DMC-GB2 tasks (Figure 4) instead of a subset would be preferred.
Questions for authors
Q1: What IQM score is achieved by FLORA on Atari 100k?
2 comments
Q2. Unclear link between Barlow Twins objective and the encoding of task-relevant information.
The paper claims that the Barlow Twins objective emphasizes the encoder to acquire task-relevant information while the reconstruction objective force the latent to encode task-irrelevant information. However, It is unclear to me how the Barlow twins objective emphasizes task-relevant information. There is no mechanism that selects for task-relevance like a reward objective, a value objective or inverse dynamics objective would specifically requires to encode task relevant information.
Response. We thank the reviewer for the question and would like to clarify that the task-relevant information in the auxiliary latent is learned through the world-model prediction objectives, while the Barlow Twins objective serves as a complementary representation regularizer.
Specifically, is incorporated into the world-model state , which is used for reward, continuation, and AC-CPC prediction. Gradients from these objectives propagate to the auxiliary encoder, encouraging to retain information useful for predicting rewards, episode continuation, and future latent representations. In particular, the reward prediction objective provides a direct task-relevant learning signal. Furthermore, the dynamics loss encourages the latent representation to remain predictable from the action-conditioned history.
The Barlow Twins objective in Eq. (7) complements these prediction objectives by aligning the projected latent features with the image features through its diagonal terms, while reducing feature redundancy and discouraging representation collapse through its off-diagonal terms. Unlike the reconstructive latent , the auxiliary latent is not directly optimized for pixel reconstruction and is therefore not required to preserve task-irrelevant background details.
The representation probing results in Table 2 further support this distinction. On Walker Walk under DMC-GB2 visual distractions, achieves a linear-probe MSE of 0.487 and Pearson correlation of 0.620, compared with 0.685 and 0.445 for the reconstruction-oriented latent . These results demonstrate that control-relevant simulator-state information is more readily accessible from .
Therefore, the world-model prediction objectives provide task-relevant learning signals, whereas Barlow Twins preserves informative, non-collapsed representations without requiring pixel reconstruction. Their roles are complementary rather than redundant.
Q3. Ablations are restricted to only a subset of tasks.
The paper provides ablations for each design choice. However, these ablations are performed on a restricted subset of tasks. Ablations on all Atari 100k games (Table 4) and all DMC-GB2 tasks (Figure 4) instead of a subset would be preferred.
Response. We thank the reviewer for the question. To further evaluate the contribution of DR across different environments, we extend the ablation in Fig. 4 to all six DMC-GB2 tasks, as summarized in Table R8. Specifically, TWISTER and TWISTER+DR are trained and evaluated under the same experimental protocol, with both retaining the AC-CPC objective.
Table R8. Returns of TWISTER and TWISTER+DR on all six DMC-GB2 tasks under the video-hard setting.
| Task | TWISTER | TWISTER+DR |
|---|---|---|
| Cartpole Swingup | 204.27 | 231.35 |
| Cheetah Run | 146.10 | 176.33 |
| Cup Catch | 86.23 | 94.97 |
| Finger Spin | 238.00 | 461.80 |
| Walker Stand | 610.17 | 773.31 |
| Walker Walk | 342.22 | 486.46 |
| Mean | 271.17 | 370.70 |
TWISTER+DR outperforms TWISTER on all six tasks, improving the mean return from 271.17 to 370.70, corresponding to a 36.7% relative improvement. The largest improvements are observed on Finger Spin (238.00 to 461.80) and Walker Walk (342.22 to 486.46). These results provide additional evidence that the benefits of DR under visual distractions extend beyond the two representative tasks originally reported in Fig. 4.
Regarding Atari 100k, the component ablations in Table 4 follow the Atari-5 protocol [1], a reduced benchmark consisting of five representative Atari games. Atari-5 was introduced to efficiently estimate aggregate performance on the full 57-game Arcade Learning Environment, with median estimates generally within 10% of the corresponding full-benchmark results. This reduced evaluation protocol enables controlled comparisons across multiple model configurations under the same training budget. The additional full six-task DMC-GB2 ablation complements these results by evaluating DR across a broader range of continuous-control environments.
Reference
[1] Aitchison et al. Atari-5: Distilling the Arcade Learning Environment down to Five Games. ICML, 2023.
We appreciate the reviewer’s valuable feedback and effort spent on the review and would like to respond to the reviewer’s questions as follows.
Q1. EDELINE comparisons and Atari 100k IQM
Response. We thank the reviewer for the question and would like to clarify the distinction between EDELINE and FLORA. EDELINE [1] is a diffusion-based world model that generates future observations in pixel space through iterative denoising. In contrast, FLORA is a discrete-latent world model that performs imagination entirely in latent space, generating the complete latent representation in a single denoising step without reconstructing observations during imagined rollouts. This distinction enables substantially more efficient world-model-based policy learning.
To provide a direct comparison, Table R7 reports the mean, median, and interquartile mean (IQM) human-normalized scores on Atari 100k, together with the reported training times. We kindly note that the IQM of FLORA is 0.846, as reported in Fig. 8 of Appendix G with stratified bootstrap confidence intervals.
Table R7. Aggregate human-normalized scores (HNS) on Atari 100k and training times of EDELINE and FLORA.
FLORA achieves a higher median HNS of 1.010, compared with 0.817 for EDELINE, surpassing human-level median performance without decision-time planning. Since the median is insensitive to extreme scores from individual games, this result demonstrates that FLORA's human-level median performance is not driven by high-scoring outliers such as Gopher. The additional IQM result provides a complementary aggregate evaluation that is less sensitive to extreme scores than the mean.
In terms of computational efficiency, FLORA requires only 5.43 hours for an Atari 100k training run on a single NVIDIA RTX 4090, compared with 69.6 hours reported by EDELINE on the same GPU model, corresponding to approximately 12.8× shorter training time. This efficiency follows from FLORA's compact categorical latent representation and one-step latent generation, avoiding repeated pixel-space denoising during imagination.
Together, these results highlight FLORA's favorable combination of median performance and computational efficiency, while providing the additional IQM comparison requested by the reviewer.
Reference
[1] Lee et al. EDELINE: Enhancing Memory in Diffusion-based World Models via Linear-Time Sequence Modeling. NeurIPS, 2025.