- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes LeDreamer, a decoder-free method for sample-efficient world models. The core observation of the paper is that, for discrete latents composed of multiple categorical distributions, directly applying SIGReg to flattened logits is affected by softmax invariance, thereby allowing low-cost zero-information collapse. Based on this, the authors propose per-distribution SIGReg and theoretically analyze the differing constraints on collapse imposed by the flattened and per-distribution regularization methods. Experiments further demonstrate that this method effectively mitigates marginal representation collapse, though stop-gradient is still required to achieve good control performance.
Critical strengths
- The paper offers a novel analysis of representation collapse in categorical latent variables. The authors leverage the block-wise shift invariance of the softmax function to explain why flattened SIGReg may fail to effectively penalize zero-information representations.
- The correspondence between theory and experiments is well established. The flattened/SG-Off models exhibit a collapse pattern where MI approaches zero while code perplexity approaches its maximum, whereas per-distribution SIGReg can avoid this phenomenon, providing strong support for the paper’s theoretical analysis.
Critical weaknesses
- The decline in final performance is too significant, and the efficiency gains do not demonstrate a convincing trade-off. On the full Atari 100K benchmark, LeDreamer’s IQM was 0.396, lower than DreamerV3’s 0.497 and STORM’s 0.636. Even for decoder-free methods, the gap is still noticeable. Although LeDreamer achieves approximately 2.12 times faster training compared to DreamerV3, it retains only about 79.7% of the IQM. For top-tier conferences, I believe the current improvements in computational efficiency are not yet sufficient to compensate for this level of performance loss.
Questions for authors
- Could the authors further explain why a training speedup of approximately 2.1 times is sufficient to compensate for this performance drop? Could they provide a more systematic analysis of the performance-compute trade-off, such as a comparison with DreamerV3 and STORM under the same wall-clock budget or the same compute budget? If this concern is addressed, I may raise the score.
Dear reviewer, thank you for acknowledging our theoretical contribution and highlighting this important concern. We agree that LeDreamer’s full-benchmark IQM gap remains significant. In response, we added a wall-clock comparison with STORM and DreamerV3 on Atari-5-100k (Section 5.1, Figure 7). We measured training times on the same RTX 5080, and both baselines used their default setups.
At 93 minutes, LeDreamer reaches 100k steps and achieves a mean human-normalized score (HNS) of 0.749, compared with 0.562 for STORM, our closest baseline, and 0.758 for DreamerV3. We use the baselines’ closest 50k checkpoints, as they reach approximately 51.2k and 48.8k steps, respectively.
At 181 minutes, LeDreamer reaches approximately 195k steps and scores 0.835, compared with 0.854 for STORM at 100k steps. DreamerV3 scores 0.917 at its final 100k checkpoint after 190 minutes. At these final evaluations, LeDreamer scores higher than STORM on Breakout and Kangaroo, similarly on Bank Heist, and lower on James Bond and Kung Fu Master. Against DreamerV3, it scores higher on Kangaroo and Kung Fu Master and lower on the other three games.
These results cover only five games and do not establish full-benchmark parity. They show that LeDreamer can reach competitive returns sooner, which may be useful when wall-clock time limits experimentation.