- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This study empirically investigates the spatial generalization of visuomotor policies that combine images and proprioceptive states. The authors find that State-free policies consistently outperform the State-based policies in both real-world and simulation environments. They suggest that such benefits are derived from relative end-effector(EEF) action representation and visual coverage.
Critical strengths
-
The paper studies an important and practical question of visuomotor policies: whether proprioceptive state helps or hurts spatial generalization of visuomotor policies. The height and horizontal shift experiments are controlled and easy to interpret, and they isolate a real failure mode of absolute state under spatial distribution shift.
-
The evaluation spans real-world tasks, simulations, ablations, and several policy architectures, with a substantial number of real-world trials. Comparisons with prior methods are also well-constructed.
Critical weaknesses
-
The title overclaims the scope of the work. Even though the paper narrows the claim to spatial generalization, it does not remove the risk of misunderstanding. Proprioception can still be useful when vision changes substantially, for example under a large background-color shift.
-
Figure 7 shows that at a data scale of n=300 the effect of state on policy performance is no longer clear, and the gap between state-free and state-based policies essentially disappears. The claim of spatial-generalization is therefore supported mainly in the small-data regime. Given the background-change case above, removing state at large scale may limit robustness rather than improve it.
Questions for authors
- For tasks with partial occlusion or background-color shift, will the generalization of State-free policies remain strong, since no visual coverage could be reliably guaranteed? And how about State-based policies?
3 comments
The authors' response has addressed my concerns regarding overclaim and visual disturbance. I will therefore raise my score to Weak acceptance.
Thank Reviewer MH42 for raising the rating
Thank you for your positive feedback and for raising your rating. We appreciate your thoughtful comments and are glad that our clarifications and additional experiments have addressed your concerns.
Response to Reviewer MH42
Thank you for recognizing the practical importance of our question, the controlled experimental design, and the breadth of our evaluations and baseline comparisons. Regarding your concerns about the title scope, data scale, and visual robustness, we respond point by point below.
Weakness 1: The title may overclaim. We clarified the intended scope of our conclusions and highlighted supporting evidence from existing experiments.
Thank you for your thoughtful consideration of the central question of our study. We agree that proprioception can be useful when visual conditions change within an in-domain spatial layout. We appreciate the opportunity to clarify our intended scope. Our title asks whether proprioceptive inputs are needed for spatial generalization, and our answer is conditional. The relevant conditions are relative EEF actions and sufficient task-relevant visual information, as examined in Sections 3.2-3.3 and Tables 4-5. Supporting results are reported in Figure 5, with further comparisons against alternative state representations and training strategies in Table 3.
We understand that your concern about the title may be related to the points raised in Weakness 2 and Question 1, which we address in detail below. Figure 7 reports in-domain rather than spatial-generalization performance, and State-based Policies still failed when occlusion was combined with unseen layouts. These points support our conditional answer.
Weakness 2: Concern of Figure 7. We clarified the interpretation of Figure 7 and explained its relationship to the existing spatial generalization results.
We clarify that Figure 7 reports the In-Domain performance, rather than generalization performance, as the number of demonstrations increases to 300 episodes. It shows that State-based Policies need more data to avoid overfitting even in the in-domain setting, as discussed in Appendix G. It also identifies 300 episodes as the scale at which both policies achieve 100% in-domain success.
Our main experiments use this same data scale of 300 episodes in Pick & Place tasks, as specified in Appendix A, yet show a substantial spatial generalization gap in Figure 5. Thus, the spatial-generalization advantage persists even when both policies achieve equally perfect in-domain performance.
Question 1: What happens under occlusion or background-color shifts? We provided further analyses of real-robot observations and conducted additional simulation experiments on visual robustness.
Occlusion. In our real-robot observations, occlusion adversely affected State-free Policies in both in-domain and unseen spatial layouts, with qualitatively similar effects. This is consistent with the need for task-relevant visual information discussed in Section 3.3 and supported by Table 5: both occlusion and reduced camera coverage can limit access to this information. For State-based Policies, however, the effect depended on the spatial layout. Under occlusion in in-domain layouts, they performed better than State-free Policies, with behavior consistent with following trajectories learned during training. However, they also failed when occlusion was combined with unseen layouts. The robot's own state does not directly reveal the target's new location. Thus, retaining state can support execution under occlusion in familiar layouts, but this benefit did not extend to the spatial-generalization settings we tested.
Background color. The moderate lighting changes we tested had only a limited impact on both State-free and State-based Policies. These observations suggest that the tested lighting variations were less disruptive than occlusions that obscured task-relevant visual information.
At the same time, during this discussion period, we added LIBERO-10 simulation experiments to quantitatively evaluate the effects of occlusion and background-color changes. We have included these results in Section R1.1 on page 10 of the revised manuscript. The results are similar to our real-robot observations described above.
Thank you again for your feedback. We hope these clarifications address your concerns and encourage you to reconsider the overall contribution of this paper. We will update the manuscript again after the discussion, incorporating all of the feedback received.