- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper studies whether visual representations with linearly decodable physical factors enable goal-conditioned supervised learning (GCSL) from images. It pretrains an encoder using motion-weighted reconstruction and a scheduled combination of covariance and eigenvalue regularization. The encoder is then frozen while a goal-conditioned policy is trained through iterative hindsight relabeling, with frequency-balanced and horizon-weighted sampling. The authors report representation probes and goal-reaching results on four simulated control environments.
Critical strengths
-
The paper examines representation quality directly. Its linear probes, encoder comparisons, and closed-loop evaluations provide more insight than policy success rates alone. The reported improvements suggest that the encoder objective can matter for GCSL in the tested settings.
-
The proposed pipeline performs well on its selected tasks. The paper also includes useful initial ablations of motion-weighted reconstruction and actor sampling. These results motivate a more rigorous evaluation of the approach.
Critical weaknesses
-
The visual representation claim is tested in a setting too narrow to establish robustness or generalization. The motion mask is obtained by thresholding normalized frame differences and is treated as a proxy for task-relevant motion. This proxy is not validated: changing background regions could receive high reconstruction weight or dominate the normalization, while frame differences need not accurately cover the moving object even with a static camera. Table 8 shows a benefit over unweighted reconstruction within the same simple environments, but the paper provides neither mask-quality analysis nor tests with moving distractors, camera or lighting changes, or unseen visual appearances. All four environments have simple static scenes, and each is evaluated against only one fixed goal. The results therefore do not establish that the learned representation supports generalizable visual goal reaching.
-
The reported success depends on privileged training information, while the mechanism and comparison to prior work remain insufficiently isolated. Encoder pretraining uses simulator states to generate broad state coverage; actor training uses ground-truth states for frequency-balanced sampling. Removing that sampling reduces Acrobot success from 98.4% to 17.5% (Table 13), yet no state-free replacement is evaluated. The regularizer comparison also changes both loss weights and their schedule, so it does not isolate the benefit of combining the two terms. Evidence that linear decodability is specifically necessary relies heavily on one constructed representation. Finally, the paper omits relevant pixel-based GCSL evidence in GLAMOR and visual GCSL comparisons in DWSL, while offering no external comparison on a shared visual goal-reaching benchmark. Some reported success percentages also require clarification under the stated evaluation protocol.
Questions for authors
-
How do the motion masks behave with moving background objects, camera motion, or lighting changes? Please provide mask visualizations and controlled tests of whether motion weighting still improves decodability and goal-reaching performance under these changes.
-
What performance does the full pipeline achieve without simulator-state access for encoder data generation and pair-sampling weights—for example, using collected image trajectories and latent-based frequency bins? How does it perform across multiple evaluation goals and unseen visual appearances?
-
Under matched data and regularization scales, how do covariance-only, eigenvalue-only, fixed combined weighting, and the proposed schedule compare? Can the method also be evaluated against relevant pixel-based GCSL or supervised goal-reaching baselines on a shared task?
-
How were the success-rate means calculated? Under the natural reading of three seeds and 100 binary evaluation episodes per seed, percentages such as 94.4%, 78.4%, and 12.2% do not follow from the stated counts.
2 comments
Thank you for the detailed response. I appreciate the clarifications, but the current evidence on training without simulator states and generalization to new visual conditions is still insufficient for me to recommend acceptance at ICLR. I will keep my original score and hope the feedback is helpful as you continue to develop the work.
Thank you for the detailed review. We will narrow our claims where the evidence is limited.
Q1: Motion masks under visual changes.
Our main contribution is learning navigable latent representations that let GCSL use images and other high-dimensional inputs. Motion weighting only improves autoencoder reconstruction; it is not a general foreground-segmentation method.
We used the simulated environments as provided, without changing backgrounds, cameras, or lighting. We have not tested such changes, so we do not claim robustness to them. We will state this limitation and add mask visualizations and controlled tests in the revision.
For real-world settings, we plan to test stronger masking methods, such as Robust PCA (Candès et al., 2011) and SAM 2 (Ravi et al., 2025). Neither is evaluated here.
Q2: State-free training and generalization.
We have not tested the full pipeline without simulator states. States currently guide pretraining-data selection and pair-sampling weights, but are not inputs to the encoder or policy. Our new experiment tests only pixel-based pretraining-data selection. We have not tested latent-based pair sampling or retrained the full model on pixel-selected data.
Pair sampling matters: paper Table 13 shows that uniform sampling lowers Acrobot success from 98.4% to 17.5% with the same encoder. This does not show whether state-free sampling preserves success.
Pixel-only pretraining-data selection. We used pixels because encoder embeddings are unavailable before pretraining. For each environment, we collected 100k random-action frames and selected equal-sized subsets using state bins, pixels, or random sampling. Pixel selection used no states; states were used only for later evaluation.
For continuous tasks, we applied PCA to frame stacks and favoured sparse regions using pi∝rk(zi), where rk is the distance to the kth nearest neighbour. For FrozenLake, we merged 512 pixel clusters into bins with Ward linkage and sampled equally from each bin.
For FrozenLake, D is total variation over tiles; elsewhere, it is sliced Wasserstein-1 distance to uniform coverage. Recovery is
{D(\mathrm{random})-D(\mathrm{state})}.$$ A score of 0 matches random selection, and 1 matches state-based selection. **Response Table R1.** Results from the same 100k-frame pool per environment, using three seeds. Pixel entries are ranges over settings chosen on that pool. Coverage is tile count for FrozenLake and a fraction elsewhere. | Environment | State coverage | Random coverage | Pixel recovered | Pixel coverage | |:---|---:|---:|---:|---:| | FrozenLake | 47 tiles | 24 tiles | 0.74–0.85 | 43–46 tiles | | Acrobot | 0.84 | 0.57 | 0.86–0.94 | 0.75–0.80 | | MountainCar | 1.00 | 1.00 | 0.85–0.95 | 1.00 | | CartPole | 0.45 | 0.35 | 0.89–0.93 | 0.42–0.43 | Pixel selection recovers much of the state-based gain. However, this test measures data coverage, not task success. We also chose settings on the same pool and did not retrain the full model. We therefore do not claim that closed-loop success is preserved. We will report these limits in the revision. We also used one fixed goal and static scenes. Multiple goals and unseen appearances remain untested, and we will avoid claims of general visual goal-reaching. **Q3: Matched regularizers and external baselines.** Paper Table 3 already compares covariance-only and eigenvalue-only regularization. We will add fixed combined weighting and our proposed schedule using the same data and regularization scales. We will report whether the schedule still helps under these conditions. Paper Table 2 compares our method with end-to-end pixel GCSL under the same evaluation protocol. We have not run shared-task comparisons with GLAMOR or DWSL. These methods change other parts of the learning process, so they were not direct controls for our representation objective. Still, they are useful full-system baselines. We will add them to related work and state the missing comparison as a limitation. A controlled comparison remains future work. **Q4: Success-rate calculation.** Thank you for catching this. The calculation is correct for exactly three seeds, but some experiments used more. “Three seeds” was meant as a minimum, and we should have given exact counts. Each seed used 100 evaluation episodes. We computed one success rate per seed and then averaged over seeds. The three cited values each used five seeds, giving 500 episodes. One successful episode therefore changes the mean by 0.2 percentage points. The raw logs give 472, 392, and 61 successes, which yield 94.4%, 78.4%, and 12.2%, respectively. We will report exact seed and episode counts, define the mean and standard error, and include per-seed counts in the appendix.