acceptodds
Under review as a conference paper at ICLR 2027

LeDreamer: Bridging the Understanding of Representation Collapse Between JEPA and Sample-Efficient World Models

Abstract

Joint Embedding Predictive Architectures (JEPAs) efficiently learn representations without pixel-level reconstruction, with recent methods seeking to replace stabilization heuristics such as stop-gradient with the minimization of well-defined objectives. Although the Sketched Isotropic Gaussian Regularizer (SIGReg) prevents representation collapse in continuous JEPAs, its direct application does not account for the factorized structure of discrete categorical latents used by sample-efficient architectures such as DreamerV3. In these latents, shifts applied to the logits of each categorical distribution do not change the resulting probabilities, but can still affect the regularization objective. We introduce LeDreamer, a decoder-free transformer-based world model that addresses this mismatch by applying SIGReg per distribution to pre-sampling categorical logits. We prove that applying SIGReg to the flattened representation, formed by concatenating the logits of all categorical distributions, admits zero-information solutions whose regularization cost decreases as with the number of categorical distributions , whereas applying SIGReg independently to each distribution assigns every such solution a positive cost independent of . Empirically, per-distribution SIGReg prevents marginal representation collapse where flattened regularization fails. We further find that preventing marginal collapse is insufficient for control: in this setting, retaining stop-gradient remains necessary to learn sufficiently action-sensitive dynamics and achieve competitive agent returns. To systematically evaluate these effects, we introduce Atari-5-100k, a statistically selected five-game ablation suite derived from results for 32 recent sample-efficient world models. Across the complete 26-game Atari 100k benchmark, LeDreamer recovers 79.7% of the interquartile mean score of DreamerV3 while training 2.12 times faster, performing on par with recent decoder-free alternatives while training between 2.0 and 3.8 times faster than evaluated decoder-enabled architectures.

Primary area:
reinforcement learning
Keywords:
World Models, Model-Based Reinforcement Learning, Representation Collapse, Joint Embedding Predictive Architecture (JEPA), Sample-Efficient World Models (SEWM), Self-Supervised Learning, Discrete Representations, Sample-Efficient Reinforcement Learning, Decoder-Free, Atari 100k

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Xcn2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes LeDreamer, a decoder-free method for sample-efficient world models. The core observation of the paper is that, for discrete latents composed of multiple categorical distributions, directly applying SIGReg to flattened logits is affected by softmax invariance, thereby allowing low-cost zero-information collapse. Based on this, the authors propose per-distribution SIGReg and theoretically analyze the differing constraints on collapse imposed by the flattened and per-distribution regularization methods. Experiments further demonstrate that this method effectively mitigates marginal representation collapse, though stop-gradient is still required to achieve good control performance.

Critical strengths
  1. The paper offers a novel analysis of representation collapse in categorical latent variables. The authors leverage the block-wise shift invariance of the softmax function to explain why flattened SIGReg may fail to effectively penalize zero-information representations.
  2. The correspondence between theory and experiments is well established. The flattened/SG-Off models exhibit a collapse pattern where MI approaches zero while code perplexity approaches its maximum, whereas per-distribution SIGReg can avoid this phenomenon, providing strong support for the paper’s theoretical analysis.
Critical weaknesses
  1. The decline in final performance is too significant, and the efficiency gains do not demonstrate a convincing trade-off. On the full Atari 100K benchmark, LeDreamer’s IQM was 0.396, lower than DreamerV3’s 0.497 and STORM’s 0.636. Even for decoder-free methods, the gap is still noticeable. Although LeDreamer achieves approximately 2.12 times faster training compared to DreamerV3, it retains only about 79.7% of the IQM. For top-tier conferences, I believe the current improvements in computational efficiency are not yet sufficient to compensate for this level of performance loss.
Questions for authors
  1. Could the authors further explain why a training speedup of approximately 2.1 times is sufficient to compensate for this performance drop? Could they provide a more systematic analysis of the performance-compute trade-off, such as a comparison with DreamerV3 and STORM under the same wall-clock budget or the same compute budget? If this concern is addressed, I may raise the score.
1 comment

Dear reviewer, thank you for acknowledging our theoretical contribution and highlighting this important concern. We agree that LeDreamer’s full-benchmark IQM gap remains significant. In response, we added a wall-clock comparison with STORM and DreamerV3 on Atari-5-100k (Section 5.1, Figure 7). We measured training times on the same RTX 5080, and both baselines used their default setups.

At 93 minutes, LeDreamer reaches 100k steps and achieves a mean human-normalized score (HNS) of 0.749, compared with 0.562 for STORM, our closest baseline, and 0.758 for DreamerV3. We use the baselines’ closest 50k checkpoints, as they reach approximately 51.2k and 48.8k steps, respectively.

At 181 minutes, LeDreamer reaches approximately 195k steps and scores 0.835, compared with 0.854 for STORM at 100k steps. DreamerV3 scores 0.917 at its final 100k checkpoint after 190 minutes. At these final evaluations, LeDreamer scores higher than STORM on Breakout and Kangaroo, similarly on Bank Heist, and lower on James Bond and Kung Fu Master. Against DreamerV3, it scores higher on Kangaroo and Kung Fu Master and lower on the other three games.

These results cover only five games and do not establish full-benchmark parity. They show that LeDreamer can reach competitive returns sooner, which may be useful when wall-clock time limits experimentation.

Reviewer_Rm5NOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies representation collapse when applying SIGReg to categorical latent world models. It identifies a key failure of SIGReg on flattened categorical logits due to the shift invariance of softmax, and provides a theoretical analysis of why per-distribution SIGReg can penalize representation collapse. Experiments further show that stop-gradient is important for effective action control, for which avoiding collapse alone is insufficient. The proposed method achieves competitive performance with better training efficiency on the Atari 100k benchmark.

Critical strengths
  1. The paper identifies a concrete failure mode when applying SIGReg to categorical world models. It explains how shift-invariant softmax logits can fool the flattened Gaussian regularization, and shows theoretical bounds for flattened and per-distribution SIGReg. This provides a useful understanding of representation collapse in categorical latent world models.

  2. The experiment results support the predicted collapse issue through mutual information, code perplexity, and SIGReg statistics. The observed separation between the critical regularization weights of flattened and per-distribution SIGReg is consistent with the theoretical analysis.

Critical weaknesses

The central motivation of the paper is that softmax allows the logits to shift without changing the probabilities. A natural alternative is to center the logits within each categorical distribution to remove these invariant directions, and then apply SIGReg with an appropriate Gaussian target. It remains unclear whether per-distribution SIGReg provides a particular advantage over this simpler shift-invariant formulation.

Questions for authors
  1. Have the authors considered removing these offsets by logit centering? How would this compare with per-distribution SIGReg?

  2. Could the authors provide a direct evaluation of action-conditioned prediction accuracy to show that stop-gradient improves the learned dynamics, in addition to action sensitivity?

1 comment

Dear reviewer, thank you for recognizing our contribution and suggesting these important controls.

Centered logits. We added Section 4.1 to study centering before applying SIGReg. Centering removes the gauge directions from the regularizer’s input. These directions can otherwise reduce the flattened regularizer’s penalty without changing the categorical probabilities. As shown in Figure 5, centering mitigates the observed collapse on average, although its effect on agent returns depends on the formulation. One hypothesis is that uncentered Per-Distribution SIGReg can use the gauge directions to fit its Gaussian target while allowing more deterministic posteriors and better returns. We will investigate this hypothesis in future work.

Figure 5 now reports the SG-Off sweep across all five Atari-5-100k games, with one run per game and weight at approximately 20k steps. Centered Flattened achieves higher mean normalized mutual information (NMI) than its uncentered counterpart at every tested weight. At the tested weights from 0.05 to 0.6, centering mitigates the near-zero mean NMI of Flattened/SG-Off. Centered Per-Distribution also achieves higher mean NMI than its uncentered counterpart at the tested weights from 0.05 to 0.4.

With SG-On at 100k steps, centered Flattened achieves a mean HNS of 0.702, compared with 0.577 without centering. Centered Per-Distribution achieves 0.649, compared with 0.806 for LeDreamer. Our current implementation uses the same standard Gaussian projection target for centered and uncentered logits. Investigating a target matched to the centered subspace is a natural direction for future work.

Action-conditioned prediction accuracy. We added an analysis of reward, termination, and latent predictions in Appendix C (Figure 10 and Table 2). All configurations are evaluated on the same recorded data, using one checkpoint per configuration and game and one rollout seed. Over longer imagined rollouts, the SG-On configurations predict nonzero rewards more accurately, while both SG-Off variants approach the zero-reward baseline.

For termination prediction, LeDreamer achieves 88.2% precision, compared with 79.6% for Per-Distribution/SG-Off, but has lower recall (33.9% versus 50.6%). For latent prediction, a low log MSE can be misleading: Flattened/SG-Off has near-zero error alongside near-zero NMI, indicating collapse. These results support the importance of stop-gradient for reward prediction and informative latents in this setting, while showing a trade-off in termination prediction.

Reviewer_Mb8bOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies decoder-free sample-efficient world models (SEWMs) with DreamerV3-style factorized categorical latents (KK distributions × CC codes) and asks how to transplant LeJEPA's SIGReg into this setting. Because SIGReg must act on pre-softmax logits while all task losses see the representation only through the softmax, each block has a gauge direction 1C\mathbf{1}_C that is free at zero task cost. The main contributions are:

  1. Theory (App. J). Lemma 1 shows that task losses depend only on the centered logits η\eta and that the zero-information manifold C0\mathcal{C}_0 (constant η\eta) has zero dynamics loss.

Theorem 3 shows that flattened SIGReg admits a collapsed law with penalty ≤2κˉρ/K\le 2\bar\kappa_\rho/K (the "fee"), while per-distribution SIGReg charges every law in C0\mathcal{C}_0 at least c0(ρ,C)>0c_0(\rho,C)>0, independent of KK (the "wall"). Proposition 5 shows that this power comes from aligning the test with the gauge direction, not from blocking per se.

  1. Method. LeDreamer is a STORM-like transformer SEWM with no decoder, an MSE-on-log-probabilities dynamics loss with stop-gradient (SG), and per-distribution SIGReg.
  2. Benchmark. Atari-5-100k is a 5-game ablation subset chosen by regression over published results of 32 SEWMs (LOOCV R2=0.922R^2=0.922).

Validation. Collapse diagnostics (normalized MI and code perplexity) are reported for the {Off, Flat, PD} × {SG on/off} grid. The λ\lambda-threshold sweep on Kung-Fu Master gives a measured critical-weight ratio of 12.5–24 against a predicted ≈11.5. Action-sensitivity analysis shows that SG is still needed. On full Atari-100k, LeDreamer reaches IQM 0.396, which is 79.7% of DreamerV3, while training 2.0–3.8× faster than decoder-enabled baselines.

Critical strengths
  1. A clean, correct, and transferable diagnosis of a real failure mode.

    • The gauge-invariance argument (Lemma 1) and the support-dimension law (Lemma 2) together explain why flattened SIGReg is fooled by a trace-matched rank-KK law. Every coordinate of that law is exactly N(0,1)\mathcal{N}(0,1), and its projections have variance 1±2/K1\pm\sqrt{2/K}.
    • The empirical signature matches the prediction: MI is 0 and perplexity sits at CC, while the SIGReg value stays low.
    • Proposition 5 (rotated degeneracies defeat block slicing) is the most valuable piece. It turns a fix into a design principle — apply isotropy tests along the architecture's invariances — that is relevant beyond Dreamer-style latents, for example to VQ/FSQ tokenizers and any softmax-bottlenecked JEPA.
  2. A useful efficiency operating point plus a reusable benchmark.

    • LeDreamer gives a credible decoder-free SEWM at about 1.5 RTX-5080 hours per run.
    • Runtimes are measured on the same hardware where possible.
    • Atari-5-100k, together with the released dataset of 32 methods, is a practical community resource for cheap and standardized ablations.
Critical weaknesses
  1. The theory explains a configuration the final method does not use, and the "wall" is quantitatively vacuous.

    • The fee-versus-wall separation concerns collapse without stop-gradient. LeDreamer, however, uses SG-On, and under SG-On neither variant collapses (Fig. 3).
    • The decisive empirical gap in Fig. 4 is between PD/SG-On and Flat/SG-On, and the theory says nothing about it. The authors acknowledge (Remark 9, App. B) that the conditional, action-dependent structure is outside the analysis. The headline claim "collapse prevented by construction, by a provable objective in place of a heuristic" is therefore overstated, since the heuristic is still load-bearing.
    • The wall constant itself is c0∼10−9c_0\sim10^{-9} at C=96C=96 (Remark 8). The paper also concedes that other zero-information laws can pay less than the realized cheat's ≈0.063. The 11× separation is therefore a Monte Carlo statement about one specific law, not a guarantee.
    • Remark 4's near-collapse bound inherits c0\sqrt{c_0}, so it gives essentially no robustness margin.
  2. The obvious alternative fixes are missing, so the "per-distribution" framing is not isolated.

    • Once the cause is identified as the 1C\mathbf{1}_C gauge, the minimal fix is to remove it: apply SIGReg (flattened or per-block) to the centered logits η=(I−ΠG)ℓ\eta = (I-\Pi_G)\ell, or use log-softmax logits. Explicitly penalizing Var(μk)\mathrm{Var}(\mu_k), or adding weight decay on the final projection (which breaks Assumption 3), are further options.
    • Without these baselines it is unclear whether the gains come from block alignment or simply from not regularizing a null direction. If centered-flattened SIGReg matches PD SIGReg, the design principle should be restated.
    • The flattened-on-raw-logits baseline is something of a strawman, because the softmax shift invariance is well known.
    • On the empirical side, absolute performance is modest:
      • IQM 0.396 against DreamerV3's 0.497 and STORM's 0.636.
      • The normalized mean is the lowest of all compared methods.
      • Ablations use 3 seeds and 20 episodes, and Fig. 5 uses one seed per point.
      • The "predicted order-KK" critical-weight ratio is only loosely tested: it is a single game, its interval [12.5, 24] does not contain 11.5, and it assumes comparable dynamics savings.
Questions for authors
  1. Centered-logit SIGReg. How does SIGReg applied to centered logits η\eta (flattened and per-block) compare with PD SIGReg, in both SG-Off and SG-On settings? This is the key missing control.
  2. Flat vs. PD under SG-On. With SG on, what mechanism explains PD > Flat? Do the Flat/SG-On logits still route a large fraction of their variance into the gauge coordinates? Please report ∑kVar(μk)/tr Cov(ℓ)\sum_k \mathrm{Var}(\mu_k)/\mathrm{tr}\,\mathrm{Cov}(\ell) for both variants.
  3. Tightness of the wall. Can the authors give a non-vacuous lower bound on SρpdS^{\mathrm{pd}}_\rho over C0\mathcal{C}_0, at least numerically, by optimizing over the law of μk\mu_k? Without it, how should the claim "constant, KK-independent penalty" be read in practice?
  4. Statistical support for the λ∗\lambda^* ratio. How many seeds and games support the critical-weight ratio? Does the ratio scale with KK when KK is varied (e.g., 4, 16, 64)? Such a scaling test is the sharpest check the theory admits.
  5. Atari-5-100k selection. Was the subset selection repeated under resampling of the 32 methods (bootstrap stability)? Many of these methods share lineage (STORM/Dreamer variants), which may inflate LOOCV R2R^2.

Suggestions (future work; not decision-relevant):

  • Extend the regularizer to the conditional law p(zt+1∣ht,at)p(z_{t+1}\mid h_t, a_t), as proposed, aiming to remove SG.
  • Test the alignment principle on FSQ/VQ tokenizers.
1 comment

Dear reviewer, thank you for recognizing our theoretical contribution and for the detailed questions that helped us improve the paper.

Centered logits. We added Section 4.1 to investigate centering, which removes the gauge directions from SIGReg’s input. Figure 5 now averages results over all five Atari-5-100k games at approximately 20k steps, using one run per game and weight. With SG-Off, centered Flattened achieves higher mean NMI than its uncentered counterpart at every tested weight. At the low weights where uncentered Flattened has near-zero mean NMI, centering mitigates this collapse. Centering also increases Per-Distribution’s mean NMI at the lowest tested weight, 0.05.

The effect on returns depends on the formulation. With SG-On at 100k steps, centered Flattened achieves a mean HNS of 0.702, compared with 0.577 without centering. Centered Per-Distribution achieves 0.649, compared with 0.806 for LeDreamer. We hypothesize that, during training, uncentered Per-Distribution can use gauge directions to fit its Gaussian target while allowing more deterministic posteriors and better returns. We will investigate this hypothesis and a Gaussian target matched to the centered subspace in future work.

SG-On gauge variance. As requested, we measured the fraction of logit variance in the gauge subspace. Averaged across five games and three runs per game, it is 0.24% for Flattened/SG-On and 0.75% for Per-Distribution/SG-On (Appendix A). Both fractions are small, and Flattened’s is lower, so excess gauge variance in Flattened is not supported as an explanation for the performance gap. Its underlying mechanism remains open.

Wall tightness and critical-weight ratio. We agree that the proven wall is quantitatively limited. The positive, KK-independent lower bound is approximately 10−910^{-9} at C=96C=96, so it does not establish a useful practical robustness margin. We have not obtained a tighter numerical bound over collapsed laws. We also revised claims suggesting that our final method replaces heuristics and now emphasize the role of stop-gradient.

The revised sweep uses K=16K=16 and one run per game and weight. Due to compute limits, we did not measure uncertainty in the critical-weight ratio or test its scaling with KK. We no longer claim that the sweep confirms a particular ratio.

Atari-5-100k selection. We did not repeat the full subset search under bootstrap resampling. The same results were used for selection and leave-one-out evaluation, and several methods share a common lineage. We now acknowledge that the reported R2R^2 may be optimistic and that prediction for new method families remains uncertain. We present Atari-5-100k as a fixed, accessible suite for inexpensive ablations that does not replace evaluation on the full benchmark.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.