- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates visual generalization in reinforcement learning from the perspective of latent representation geometry. The authors observe that visual distribution shifts tend to alter latent representations primarily along the dominant principal components of the training distribution, rather than introducing entirely new directions. Motivated by this finding, they propose Latent Space Perturbation (LSP), which injects variance-scaled Gaussian noise along these principal directions during training to improve policy robustness to visual shifts. Experiments on LSP-Env, five Procgen environments, and two RL-ViGen tasks show consistent improvements in OOD performance. Additional analyses with different encoders and stronger visual shifts further examine when the proposed latent-space assumption holds and where the method begins to lose effectiveness.
Critical strengths
- The paper establishes a clear link between its latent-space analysis and the proposed method. The authors show that visual OOD shifts are largely concentrated in the dominant training subspace, and use this observation to motivate PCA-aligned perturbations rather than generic latent noise.
- LSP is simple, lightweight, and easy to integrate without modifying the policy architecture. Its gains on both Procgen and RL-ViGen suggest that the method extends beyond the controlled LSP-Env and has practical value across different visual RL settings.
Critical weaknesses
-
The latent-shift analysis needs a cleaner state-matched control. The training and evaluation latents appear to use different state distributions, so the measured mean and variance shifts may reflect both state changes and visual changes. Since this analysis is the main motivation for LSP, the authors should compare the same underlying states under training and OOD appearances and recompute the key shift statistics.
-
The experimental setup contains several inconsistencies. For RL-ViGen Door, Table 2 reports total training steps, while the selected LSP schedule starts at steps. The LSP-Env experiment is also described as PPO in some places and SAC in others. The authors should clarify the exact configuration used for each reported result and make the settings consistent throughout the paper.
Questions for authors
-
Can the authors clarify how task states were sampled for the training and evaluation latent sets in Section 5.2? As written, the two sets appear to use different state distributions, which may confound state variation with appearance variation. If so, could the authors recompute the key mean- and variance-shift statistics using the same underlying states rendered under different appearances? This control seems important for validating the main geometric motivation behind LSP.
-
Can the authors clarify the exact configurations used for Figures 9 and 10? For the Door experiment, Table 2 reports a total horizon of steps, while the selected LSP schedule activates at steps. Please clarify the activation time and its unit used in the actual run. Similarly, please confirm whether Figure 9 uses SAC or PPO and correct the inconsistent labels across the main text, figure, and appendix.
3 comments
Dear Reviewer,
We hope the new evidence addresses your original concerns. If any questions or issues remain, we would be happy to discuss them and provide further clarification.
Thank you for the detailed response and additional experiments. The state-matched analysis substantially addresses my first concern, providing much stronger evidence that visual shifts are concentrated in the dominant latent directions independently of task-state variation.
The clarification of the SAC/PPO configurations and the actual Door training horizon and perturbation onset also addresses my second concern. However, please ensure that these corrections are consistently reflected in the main text and tables, and that the Door results from different schedules are clearly distinguished rather than presented as independent replications.
Overall, my original concerns have been largely addressed, and I appreciate the additional effort.
Dear Reviewer,
Thank you for your careful evaluation and constructive feedback.
We have addressed your concerns through additional experiments and a thorough verification of our experimental configurations.
We kindly ask you to download the updated PDF of our submission, where we have included a dedicated Rebuttal section at the end of the Appendix, containing a detailed response to your comments and the new experimental results.
We believe these additions strengthen the paper and would greatly appreciate it if you could reconsider your assessment based on the updated evidence.
Thank you again for your time and consideration.