acceptodds
Withdrawn from ICLR 2027

Innovation SIGReg: Regularizing Transition Residuals for JEPA World Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Gaussian regularization helps joint-embedding predictive architectures (JEPAs) resist collapse, or loss of distinctions between inputs. Yet the marginal embedding distribution pooled over trajectories need not be Gaussian, so enforcing this target can compete with linear state decoding. Innovation SIGReg applies Sketched Isotropic Gaussian Regularization to persistence-adjusted transition residuals, with a weighted raw-embedding anchor to constrain location and scale. Linear–Gaussian dynamics motivate the target: true transition noise can remain Gaussian while state distributions change, although learned residuals need not recover that noise. On Reacher, a robot-arm benchmark, mean linear angle-decoding at temporal stride 64 rises from to across ten runs per method, and the weakest-coordinate score rises from to . Normalization alone and rescaled temporal differences do not reproduce these gains at the tested settings. On Two-Room navigation, unguarded residual regularization produces both strong and poor planners. An anchor guard improves mean success over marginal SIGReg by percentage points in a prespecified first-cohort comparison and by points in an exploratory second-cohort comparison; fixed and scheduled anchor increases perform comparably in the first. Detaching current-batch statistics yields success versus with differentiable statistics, although three runs per configuration leave reliability unresolved. A separate Reacher ablation reverses this comparison: detachment reduces from to , while detached running averages perform best. These findings show that both the regularization target and the handling of its statistics matter for downstream decoding and planning.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
joint-embedding predictive architectures (JEPA), world models, self-supervised representation learning, transition innovations, Gaussian regularization, temporal representation learning, representation learning for planning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_osabOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Innovation SIGReg, which applies Gaussian regularization to persistence-adjusted transition residuals instead of marginal embeddings in JEPA world models. It combines estimated coordinatewise temporal persistence with a raw-embedding anchor to prevent collapse and investigates different gradient-handling strategies for estimated statistics. Experiments on Reacher and Two-Room evaluate representation quality and downstream planning performance.

Critical strengths

The paper distinguishes statistics updates from gradient propagation and reveals substantially different effects across Reacher and Two-Room. This analysis provides useful insights into optimization behavior beyond simply introducing another regularization objective.

Critical weaknesses
  1. The best batch-detached variant achieves 97.5% planning success on Two-Room but is not evaluated on the official Two-Room dataset or PushT. Meanwhile, the earlier unguarded variant underperforms marginal SIGReg on both settings. This raises concerns about whether the reported planning improvements generalize beyond the selected experimental setting.
  2. Reacher's normalization and differencing baselines are not independently tuned, and Two-Room lacks a temporal-differencing baseline with matched statistics detachment. Consequently, it remains unclear whether the improvements arise from the proposed persistence-adjusted residual or simpler stabilization and optimization choices.
Questions for authors
  1. Since the strongest batch-detached configuration is only evaluated on the regenerated Two-Room dataset, can you provide its performance on the official Two-Room dataset and PushT to demonstrate whether the observed improvements generalize?
  2. Given that the adaptive guard shows no statistically significant advantage over simpler fixed or scheduled anchors, nor over the marginal Sub-JEPA baseline, what evidence demonstrates that regularizing transition residuals or using participation ratio triggering provides a distinct benefit for planning beyond simply tuning the effective marginal regularization weight?
Reviewer_Qpi3OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a new regularization method for world models . Innovation SIGReg works via regularizing the transition residuals for sketched gaussianity. The target residuals are standardized via a diagonal first oder model, which is a source of complexity for the approach: estimating these mean/vars on the batch or via some exponential running average is not clearly settled and the preferred solution changes between the datasets. The approach is evaluated on two major benchmarks: reacher, a robotics-arm benchmark and two room navigation.

Critical strengths

The problem itself is important and progress, even if mainly empirically driven, is valuable. World models are a promising research direction and improving the performance of them is of major interest to the research community.

A major strength of the approach is it's plug-and-play nature of the regularization, which can be readily combined with architectures, problems etc.

Critical weaknesses

A first major weakness is its motivation: it is unclear to me why the gaussianity regularization is really needed/warranted. Especially section 3.1 (WHY REGULARIZE TRANSITION INNOVATIONS?) is lacking. It argues over the gaussianity of the innovations impacting the gaussianity of the state distribution but this is an arbitrary argument: I can replace the gaussian distribution with any other distribution and arrive at the same argument. There is neither an empirical or theoretical argument presented here why this is a direction to pursue, the approach therefore seems arbitrary.

The lack of justification would not be particularly relevant if the empirical results would speak for themselves, but the empirical results seem disconnected and not particularly convincing. The need to estimate the standardization parameters results in the issues of balancing stability (with the exponential moving average) with the ability to adapt (only estimate on the batch). Whether to differentiate through these statistics is another dimension introduced in the method. Neither question is accurately settled, which starts by a lack of a discussion of the impact of the these choices on loss, gradients and stability. E.g. do we even want to differentiate through these statistics? It is not clear. Also empirically, these choices are not settled. This issue connects to the general approach of the experimental section, which seems entirely descriptive of running the two experiments instead of settling research questions. E.g. in section 4.4 (THE EFFECT OF STATISTICS HANDLING DEPENDS ON THE BENCHMARK), a takeway is not formalized. What could be the reason for the phenomena of the EMA/batch estimate being better on one problem or the other? It does not inspire confidence that the method would transfer to a new problem. One does not learn a lot from the experiments, despite maybe that different hyperparamters are needed for different problems, as the section rather reduces to just iterating over the results and scores, verbalizing results. A deeper analysis/investigation is missing.

Background section 2.1 would benefit from a better explanation of the different methods, including their mathematical formulations.

Section 3.2 is very vague, despite describing the method. A whole algorithmic definition of the regularization, including all the intermediate computations, is essential. Algorithm 1 in the appendix again calls SIGReg, without it being properly defined.

Questions for authors

A few points would benefit from being clarified:

  • in equation (1), is the product-symbol is the Hadamard-product?
  • what is wmw_m in equation 4? is this a hyperparameter?
  • in equation (21), what is qlq_l?
  • on page 8, right above 4.6 (line 415/416): what is meant with guard?
  • why is EMA missing from figure 3, or is it included in one of the points?
Reviewer_t1ySOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes Innovation SIGReg, a novel regularizer for latent space world models. In contrast to standard SIGReg that applies the regularization to the raw embeddings themselves, Innovation SIGReg first normalizes the embeddings, estimates persistence and uses it to recover the residual innovation term which it applies the regularization to. The procedure is motivated my the fact that standard SIGReg may not only push embeddings from different trajectories towards Gaussian distribution, but also embeddings from the same trajectory that don't have to be Gaussian in marginal, e.g. in the non-stationary case. The Innovation SIGReg is anchored with a standard SIGReg on raw embeddings.

Critical strengths
  1. The idea is clear and well-motivated. Lemma 1 gives good intuition on how every state can be Gaussian in marginal, yet the pooled state over a trajectory does not have to be Gaussian.
  2. The motivation seems to be supported by the results on Reacher in the main paper, where Innovation SIGReg yields better linearly decodable embeddings.
Critical weaknesses
  1. The main weakness of the paper is that it is very difficult to read. While the main idea and the method appear to be simple, the findings and conclusions are difficult to recover from the experiments section even with careful reading. Fixing the story would most likely mean substantial revision.
  2. The evidence is rather weak with some important experimental results in the appendix. E.g. it is not clear how the parameters of the Reacher evaluation (such as the stride and λ\lambda) were selected, table 9 lists all different development runs and appears to select the configuration where the proposed method wins over the baseline, while other configurations report mixed results.
Questions for authors
  1. Why was EMA-detached variant not evaluated in Figure 2? As it seems to be the best configuration for Reacher.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.