acceptodds
Under review as a conference paper at ICLR 2027

GCSL from Pixels: Geometric Regularization for Navigable Visual Goal Spaces

Abstract

Goal-Conditioned Supervised Learning (GCSL) learns goal-reaching behavior through hindsight relabeling and self-imitation, without a reward function, value function, or expert demonstrations. We propose that its success in physical state spaces may depend on an assumption it never states, that the state/goal space carries the environment's factors in linearly decodable form. While low-dimensional states provide this for free, we find that a naively trained pixel encoder often fails to obtain linear decodability, and that this gap is fixable. Pretraining the image encoder with a reconstruction loss plus a geometric regularizer combining variance-covariance (VICReg) and an eigenvalue-based collapse penalty (EigenReg) restores linear decodability and reliable goal-reaching across four control environments. GCSL's relabeling objective stays unchanged, though reliable pixel performance also needs frequency-balanced hindsight sampling and horizon weighting in the actor loop. Two controlled comparisons isolate where the improvement comes from. Against end-to-end pixel GCSL, which fails outright with 0% success across all four environments despite working reliably from physical state, our method reaches 94 to 100% success. Against GCBC, a one-shot cloning variant without iterative self-improvement, we isolate the contribution of iteration alone. A further comparison against vanilla-VAE, Beta-VAE, VICReg-only, and EigenReg-only isolates the regularizer's own contribution. Finally, comparing linear against nonlinear decodability of the same representations tests whether linear recoverability is associated with goal-reaching in these environments, or recoverability of any kind is. These results give an empirical characterization of when GCSL's reward-free formulation extends from physical state spaces to pixels.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
goal-conditioned supervised learning, visual goal-reaching, representation learning, geometric regularization, hindsight relabeling, goal-conditioned imitation learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_fPjyOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies whether visual representations with linearly decodable physical factors enable goal-conditioned supervised learning (GCSL) from images. It pretrains an encoder using motion-weighted reconstruction and a scheduled combination of covariance and eigenvalue regularization. The encoder is then frozen while a goal-conditioned policy is trained through iterative hindsight relabeling, with frequency-balanced and horizon-weighted sampling. The authors report representation probes and goal-reaching results on four simulated control environments.

Critical strengths
  • The paper examines representation quality directly. Its linear probes, encoder comparisons, and closed-loop evaluations provide more insight than policy success rates alone. The reported improvements suggest that the encoder objective can matter for GCSL in the tested settings.

  • The proposed pipeline performs well on its selected tasks. The paper also includes useful initial ablations of motion-weighted reconstruction and actor sampling. These results motivate a more rigorous evaluation of the approach.

Critical weaknesses
  • The visual representation claim is tested in a setting too narrow to establish robustness or generalization. The motion mask is obtained by thresholding normalized frame differences and is treated as a proxy for task-relevant motion. This proxy is not validated: changing background regions could receive high reconstruction weight or dominate the normalization, while frame differences need not accurately cover the moving object even with a static camera. Table 8 shows a benefit over unweighted reconstruction within the same simple environments, but the paper provides neither mask-quality analysis nor tests with moving distractors, camera or lighting changes, or unseen visual appearances. All four environments have simple static scenes, and each is evaluated against only one fixed goal. The results therefore do not establish that the learned representation supports generalizable visual goal reaching.

  • The reported success depends on privileged training information, while the mechanism and comparison to prior work remain insufficiently isolated. Encoder pretraining uses simulator states to generate broad state coverage; actor training uses ground-truth states for frequency-balanced sampling. Removing that sampling reduces Acrobot success from 98.4% to 17.5% (Table 13), yet no state-free replacement is evaluated. The regularizer comparison also changes both loss weights and their schedule, so it does not isolate the benefit of combining the two terms. Evidence that linear decodability is specifically necessary relies heavily on one constructed representation. Finally, the paper omits relevant pixel-based GCSL evidence in GLAMOR and visual GCSL comparisons in DWSL, while offering no external comparison on a shared visual goal-reaching benchmark. Some reported success percentages also require clarification under the stated evaluation protocol.

Questions for authors
  • How do the motion masks behave with moving background objects, camera motion, or lighting changes? Please provide mask visualizations and controlled tests of whether motion weighting still improves decodability and goal-reaching performance under these changes.

  • What performance does the full pipeline achieve without simulator-state access for encoder data generation and pair-sampling weights—for example, using collected image trajectories and latent-based frequency bins? How does it perform across multiple evaluation goals and unseen visual appearances?

  • Under matched data and regularization scales, how do covariance-only, eigenvalue-only, fixed combined weighting, and the proposed schedule compare? Can the method also be evaluated against relevant pixel-based GCSL or supervised goal-reaching baselines on a shared task?

  • How were the success-rate means calculated? Under the natural reading of three seeds and 100 binary evaluation episodes per seed, percentages such as 94.4%, 78.4%, and 12.2% do not follow from the stated counts.

2 comments

Thank you for the detailed review. We will narrow our claims where the evidence is limited.

Q1: Motion masks under visual changes.

Our main contribution is learning navigable latent representations that let GCSL use images and other high-dimensional inputs. Motion weighting only improves autoencoder reconstruction; it is not a general foreground-segmentation method.

We used the simulated environments as provided, without changing backgrounds, cameras, or lighting. We have not tested such changes, so we do not claim robustness to them. We will state this limitation and add mask visualizations and controlled tests in the revision.

For real-world settings, we plan to test stronger masking methods, such as Robust PCA (Candès et al., 2011) and SAM 2 (Ravi et al., 2025). Neither is evaluated here.

Q2: State-free training and generalization.

We have not tested the full pipeline without simulator states. States currently guide pretraining-data selection and pair-sampling weights, but are not inputs to the encoder or policy. Our new experiment tests only pixel-based pretraining-data selection. We have not tested latent-based pair sampling or retrained the full model on pixel-selected data.

Pair sampling matters: paper Table 13 shows that uniform sampling lowers Acrobot success from 98.4% to 17.5% with the same encoder. This does not show whether state-free sampling preserves success.

Pixel-only pretraining-data selection. We used pixels because encoder embeddings are unavailable before pretraining. For each environment, we collected 100k random-action frames and selected equal-sized subsets using state bins, pixels, or random sampling. Pixel selection used no states; states were used only for later evaluation.

For continuous tasks, we applied PCA to frame stacks and favoured sparse regions using pi∝rk(zi)p_i\propto r_k(z_i), where rkr_k is the distance to the kkth nearest neighbour. For FrozenLake, we merged 512 pixel clusters into bins with Ward linkage and sampled equally from each bin.

For FrozenLake, DD is total variation over tiles; elsewhere, it is sliced Wasserstein-1 distance to uniform coverage. Recovery is

{D(\mathrm{random})-D(\mathrm{state})}.$$ A score of 0 matches random selection, and 1 matches state-based selection. **Response Table R1.** Results from the same 100k-frame pool per environment, using three seeds. Pixel entries are ranges over settings chosen on that pool. Coverage is tile count for FrozenLake and a fraction elsewhere. | Environment | State coverage | Random coverage | Pixel recovered | Pixel coverage | |:---|---:|---:|---:|---:| | FrozenLake | 47 tiles | 24 tiles | 0.74–0.85 | 43–46 tiles | | Acrobot | 0.84 | 0.57 | 0.86–0.94 | 0.75–0.80 | | MountainCar | 1.00 | 1.00 | 0.85–0.95 | 1.00 | | CartPole | 0.45 | 0.35 | 0.89–0.93 | 0.42–0.43 | Pixel selection recovers much of the state-based gain. However, this test measures data coverage, not task success. We also chose settings on the same pool and did not retrain the full model. We therefore do not claim that closed-loop success is preserved. We will report these limits in the revision. We also used one fixed goal and static scenes. Multiple goals and unseen appearances remain untested, and we will avoid claims of general visual goal-reaching. **Q3: Matched regularizers and external baselines.** Paper Table 3 already compares covariance-only and eigenvalue-only regularization. We will add fixed combined weighting and our proposed schedule using the same data and regularization scales. We will report whether the schedule still helps under these conditions. Paper Table 2 compares our method with end-to-end pixel GCSL under the same evaluation protocol. We have not run shared-task comparisons with GLAMOR or DWSL. These methods change other parts of the learning process, so they were not direct controls for our representation objective. Still, they are useful full-system baselines. We will add them to related work and state the missing comparison as a limitation. A controlled comparison remains future work. **Q4: Success-rate calculation.** Thank you for catching this. The calculation is correct for exactly three seeds, but some experiments used more. “Three seeds” was meant as a minimum, and we should have given exact counts. Each seed used 100 evaluation episodes. We computed one success rate per seed and then averaged over seeds. The three cited values each used five seeds, giving 500 episodes. One successful episode therefore changes the mean by 0.2 percentage points. The raw logs give 472, 392, and 61 successes, which yield 94.4%, 78.4%, and 12.2%, respectively. We will report exact seed and episode counts, define the mean and standard error, and include per-seed counts in the appendix.

Thank you for the detailed response. I appreciate the clarifications, but the current evidence on training without simulator states and generalization to new visual conditions is still insufficient for me to recommend acceptance at ICLR. I will keep my original score and hope the feedback is helpful as you continue to develop the work.

Reviewer_ewpUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper extends GCSL to pixel-based goal achievement tasks by combining a pre-trained visual encoder with VICReg and EigenReg regularizations. The proposed framework integrates geometric representation learning and iterative GCSL training. Experiments and ablation studies evaluate its effectiveness and examine the relationship between linear decodeability and goal achievement performance.

Critical strengths

This paper provides valuable empirical evidence on how visual representation quality affects GCSL performance, with a particular focus on the relationship between linear decodeability and goal achievement success. Another advantage is the inclusion of control and ablation experiments, which helps differentiate the contributions of representation regularization and iterative policy training.

Critical weaknesses

The main weakness of this method is its limited experimental scope, with evaluation confined to simple simulated environments with fixed targets. Therefore, its generalization ability to more realistic visual control tasks remains unclear. More importantly, the method relies on frequency-balanced sampling from real-world states during training, which weakens its ability to learn target-oriented behavior directly from pixels. These limitations significantly reduce the method's practical application value and broader impact.

Questions for authors

The authors should clarify how much of the performance improvement comes from geometric regularization rather than ground-truth-state-based sampling, and whether the method remains effective when sampling relies entirely on learned visual representations. The evaluation is also limited to a single fixed goal per environment, leaving the method's generalization to unseen goals unclear.

2 comments

Thank you for these comments. We ran a new experiment on state-free frame selection and will clarify the limits of our evaluation.

Q1: Regularization and state-free sampling.

Our ablations show that both sampling and regularization matter. With the same regularized encoder, uniform pair sampling lowers success to about 18%. This shows that sampling matters, so we do not attribute the gain to regularization alone. In the original method, b(⋅)b(\cdot) uses simulator state only to compute sampling weights. Neither the encoder nor the policy receives state as input.

We then tested whether pixels alone can select frames with similar state-space coverage. For each environment, we collected 100k random-action frames and selected equal-sized subsets using state bins, pixel features, or random sampling. Pixel selection used no state information. We used true states only after selection to evaluate the subsets.

For the three continuous environments, we applied PCA to frame stacks and sampled without replacement using

pi∝rk(zi),p_i \propto r_k(z_i),

where rk(zi)r_k(z_i) is the distance from feature vector ziz_i to its kk-th nearest neighbour. This gives a higher sampling probability to frames in sparse regions. For FrozenLake, we formed 512 pixel micro-clusters, merged them into kk bins using Ward linkage, and selected the same number of frames from each bin.

We measured the selected subset's distance from broad state-space coverage using DD. For FrozenLake, DD is total variation distance over tiles. For the continuous environments, it is sliced Wasserstein-1 distance to a uniform reference. We report the recovered improvement:

\frac{D(\mathrm{random})-D(m)} {D(\mathrm{random})-D(\mathrm{state})}.$$ A score of 0 matches random selection, and 1 matches state-based selection. **Response Table R1. Frame selection from the same 100k-frame pool in each environment.** All subsets have equal size. Coverage is the number of covered tiles for FrozenLake and the covered fraction for the other environments. Higher values are better for both metrics. | Environment | Selection method | Recovered | Coverage | |:---|:---|---:|---:| | FrozenLake | State bins (reference) | 1.00 | 47 tiles | | | Random | 0.00 | 24 tiles | | | Pixel Ward ($k=64$, 128 PCs) | 0.83 | 45 tiles | | | Pixel Ward ($k=96$, 256 PCs) | 0.85 | 46 tiles | | Acrobot | State grid (reference) | 1.00 | 0.84 | | | Random | 0.00 | 0.57 | | | Pixel kNN (256 PCs, whitened, $k=5$) | 0.86 | 0.75 | | | Pixel kNN (512 PCs, whitened, $k=5$) | 0.94 | 0.80 | | MountainCar | State grid (reference) | 1.00 | 1.00 | | | Random | 0.00 | 1.00 | | | Pixel kNN (32 PCs, $k=5$) | 0.93 | 1.00 | | | Pixel kNN (16–128 PCs, $k=5$) | 0.95 | 1.00 | | CartPole | State grid (reference) | 1.00 | 0.45 | | | Random | 0.00 | 0.35 | | | Pixel kNN (32 PCs, $k=5$) | 0.93 | 0.43 | | | Pixel kNN (16–128 PCs, $k=5$) | 0.93 | 0.43 | Pixel selection recovers much of the state-based improvement in all four environments. However, the best settings differ across environments, so we do not claim that one setting works everywhere. These experiments compare only the selected data, not task performance. Since the pixel-selected and state-selected subsets have similar distributions, we believe pixel selection is also likely to work well when training the encoder and policy. However, we could not run these closed-loop tests during the response period because of time limits. We therefore do not claim that closed-loop success is preserved. We will add this experiment to the ablation section and state this limitation clearly. **Q2: Fixed-goal evaluation.** We agree. Testing one fixed goal per environment does not show whether the method generalizes to unseen goals. Our focus was image-based GCSL under each environment's standard task and success criterion, not generalization across goals. We will make this scope clear and list fixed-goal evaluation as a limitation. Future work will test multiple and held-out goals to examine whether linear decodability improves generalization.

The additional experiments provide useful evidence that pixel-based frame selection can approximate state-based sampling in terms of state-space coverage. The lack of closed-loop policy evaluation still leaves its impact on downstream performance unclear. The authors also acknowledge that the fixed-goal evaluation does not establish generalization to unseen goals. These concerns are only partially addressed, so I maintain my initial score.

Reviewer_6y9SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how the linear decodability of latent representations affects task success when applying GCSL to image inputs. Building upon the original GCSL framework, the authors introduce three additional loss terms: motion-weighted reconstruction, VICReg, and EigenReg. They find that these losses simultaneously improve the linear decodability of latent representations and task success rates. Besides, a controlled nonlinear decodable representation fails to improve task performance. Based on these results, the paper claims that GCSL relies on linearly decodable representations.

Critical strengths

The problem formulation is clear and focused on the linear decodability of GCSL latent representations. Targeted regularization terms and quantitative metrics for linear decodability are designed accordingly.

Critical weaknesses

The experimental evidence is insufficient to fully support the claims:

  1. Compared to the original end-to-end GCSL baseline, the proposed model differs not only in the two linear decodability regularizers (VICReg and EigenReg) but also includes a motion-weighted reconstruction term and a separate pretraining phase. Therefore, it is difficult to attribute the performance gains solely to improved linear decodability rather than other modifications.
  2. The paper uses a controlled sin/cos encoding scheme to represent nonlinear decodability. However, this encoding also disrupts topological structure and thus cannot represent all possible forms of nonlinear decodable features.
Questions for authors
  1. Is linear decodability a primary cause of high task success, or merely a correlated phenomenon?
  2. Could the performance improvements shown in Tables 1-2 be attributed to the motion-weighted reconstruction loss rather than linear decodability?
  3. The original end-to-end GCSL model must simultaneously learn representations and adapt to the policy distribution, making optimization inherently more difficult than the proposed pretraining-plus-freezing approach. Moreover, the training data distributions differ substantially between the two setups. To what extent do the performance advantages of the proposed model stem from these training strategy differences?
  4. In Table 4, the controlled sine/cosine encoding is used to represent nonlinear decodability, but this encoding also destroys topological continuity. Could the poor performance result from structural disruption rather than nonlinearity per se? Have other smooth nonlinear decodable representations been compared?
  5. In Table 4, the nonlinear decodable representation still achieves high success rates on FrozenLake. Does this indicate that linear decodability is not a necessary condition for GCSL performance?
  6. Could the poor performance under non-linear decodable representations be due to an overly simple policy network rather than the uselessness of nonlinearity itself?
2 comments

Thank you for raising these points. They show where our claims were too strong. We will revise the paper to better match our claims to the evidence.

Q1: Does linear decodability cause success?

Our results show that better linear decodability is linked to higher task success. They do not show that it causes the improvement. A causal test would need to change decodability while keeping other factors fixed. We have not done this. We will revise our wording and leave causal tests and theoretical analysis to future work.

Q2: Could motion weighting explain the gain?

Paper Table 8 shows that motion weighting helps. Removing it lowers static R2R^2 from 93.3 to 81.8 and dynamic R2R^2 from 89.3 to 68.4. However, paper Table 3 gives a matched comparison. All pretrained models there use the same motion-weighted reconstruction loss and differ only in regularization. The vanilla VAE uses motion weighting without our regularizers. It reaches 74.4/63.9 static/dynamic R2R^2 and 67.7% success, while our encoder reaches 93.3/89.3 and 98.3%. Thus, motion weighting alone does not explain the gain.

Both parts improve decodability. Adding our regularizers to the motion-weighted loss raises dynamic R2R^2 from 63.9 to 89.3. Adding motion weighting to the regularized encoder raises it from 68.4 to 89.3. Paper Table 2 is not matched on this point because its end-to-end baseline uses no reconstruction loss. We will clarify this comparison and add standard errors for paper Table 8.

Q3: What about pretraining and freezing?

The vanilla VAE in paper Table 3 is also pretrained and frozen, using the same data and reconstruction loss. Its lower success (67.7% versus 98.3%) shows that pretraining and freezing alone do not explain the gap.

Still, paper Table 2's end-to-end baseline is not a matched comparison. It learns representations while the policy changes, and some training data comes from its own environment interactions. The setups therefore differ in both optimization and data distribution. We cannot assign an exact share of the total gain to each factor. We will state this limitation clearly.

Q4: Does the sine/cosine encoding change topology?

Yes, it can. Although the mapping is smooth, it is also periodic. At span 1.0, the two ends of a coordinate map to the same point. This creates a circular structure, so distant states may become close neighbours. Smaller spans reduce this wrap-around. Smoothness alone therefore does not mean that the original neighbourhoods are preserved.

We tested local neighbourhood overlap in all four environments. For each point, we found its five nearest neighbours in state space. Ov@5 is the average fraction of these neighbours that remain among its five nearest neighbours in the tested representation. We compared the latent and kernelled latent representations using the same state-space reference. The kernel uses K=2K=2. Results are means over five seeds; FrozenLake uses visited tiles.

Response Table R1: Local neighbourhood overlap (Ov@5).

EnvironmentLatentKernel, span 1.0Kernel, span 0.95
FrozenLake0.900.790.91
CartPole0.530.630.63
Acrobot0.660.750.75
MountainCar0.330.430.43

The kernel gives higher overlap in the continuous environments. On FrozenLake, the result depends on the span: span 1.0 lowers overlap, while span 0.95 gives about the same overlap as the latent space. This test covers only local neighbourhoods. It does not prove that the full topology is preserved and does not explain policy performance. We will limit our claims to the tested encoding. Tests with other smooth nonlinear representations remain future work.

Q5: Does FrozenLake rule out necessity?

Yes. FrozenLake shows that linear decodability is not needed for success in every setting. One possible reason is its small state space: with 64 states, simple exploration may already give good coverage. Geometric regularization may be less useful there than in larger environments, where coverage and navigation are harder. We have not tested this explanation, so we present it only as a hypothesis. We will remove claims of universal necessity and leave tests of state-space size to future work.

Q6: Could policy capacity explain the result?

Yes, this is possible. We did not change the policy network or test larger policies. The poor result may come from the representation, the policy, or how they interact. Our results therefore apply only to the tested policy class. They do not show that nonlinear representations cannot work with other policies. We will state this limitation and leave tests with higher-capacity policies to future work.

Thank you for your response, which rules out the influence of the motion-weighted reconstruction loss. However, several critical issues remain inadequately addressed, including the differences in training data, the lack of smooth non-linear representations, and the role of policy capacity. These issues are closely tied to the paper’s central claims. Therefore, I maintain my initial score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.