- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces DFPG (Dual-Flow Generation Policy), that implicitly learns viewpoint invariant visual representations as a way to guide accurate action generation under arbitrary viewpoints. It consists of two pathways: image mapping, that infers canonical views using flow matching, and action generation, that predicts robot actions condwitioned on the shared canonical visual features. The authors also introduce a bidirectional cross-attention module designed to fuse multi-source features for stronger cross-view action generation. Experiments on two simulation benchmarks and real-world robots show statistically significant improvements.
Critical strengths
The paper addresses what to me is a clear limitation of current policy methods: sensitivity to camera viewpoint. Most robotics datasets do not contain accurate camera information, which (1) difficults the study of the impacts that introducing such information could have in policy learning, and (2) limits the introduction of benchmarks that leverage such information and accurately measure the degradation of policy performance under different camera placements.
Critical weaknesses
Though interesting and appealing, I wonder if the choice of addressing the problem of image and action generation jointly is reasonable: in these days there are foundation models capable of generating spatio-temporally consistent images and 3D reconstructions from a single view. Such method should be able to shortcut the generation of canonical views, so a traditional policy can be trained without any modification. Even so, I wonder what are the drawbacks of foregoing viewpoint robustness in favor of abstracting it away, so all the heavy lifting is being done by the generative model. A proper study showing improvements in policy performance as image generation improves would be highly relevant here, as well as an analysis of what we lose or gain by relying on canonical images instead of free views (e.g., Are we losing relevant information by observing the scene from a different viewpoint? How far apart can free images be from the canonical view before results degrade? How much impact does hallucination have, since unobserved areas will be inpainted? How should one choose where to place the canonical view?).
Questions for authors
Minor comments:
- In line 55 it is stated that the main cause for policies failing to generalize to other platforms is camera mounting variation. I understand that this is indeed an issue, but is this really the main cause?
- The references are missing parentheses, and so author names are mixed with the text, making it hard to read citation-heavy sentences.
- In Table 3, a PSNR of 80+ is astoundingly high, can you explain how you can generate predictions that close to ground-truth images?
- What is the reasoning behind the selection for xyz perturbations in Section 4.1?
We sincerely thank you for your careful review. Below, we provide a point-by-point response to each of your comments. All revisions in the updated manuscript are highlighted in red for clarity and ease of reference.
W1&Q1: Questions about motivation and drawbacks.
A1: We agree that foundation models have advanced rapidly and can produce temporally consistent images and 3D reconstructions from a single view. However, robotic manipulation still requires a geometric transformation between the robot base coordinate system and the camera coordinate system, namely camera extrinsic calibration. Even a high-fidelity point cloud reconstruction is insufficient for action execution unless it is accurately registered to the robot’s coordinate system. Prior generative visuomotor policies typically assume a fixed camera viewpoint and implicitly absorb the static camera–robot transformation into their learned representations during training. When the camera viewpoint shifts—even slightly—their internal spatial alignment degrades, leading to policy failure. This is one of the main causes for poor cross-platform generalization of visuomotor policies (we have revised this phrasing for rigor at Line 55, other factors include deviations in robot kinematic modeling and differences in low-level control logic).
Compared with a single fixed viewpoint, a free-viewpoint setting inevitably leads to a lower success rate in manipulation when the same amount of training data is used. Fairly speaking, our work trades a certain level of manipulation accuracy for viewpoint flexibility during practice. Following your suggestion, we have added an evaluation of the manipulation success rate under out-of-range viewpoints in Section 4.6, and discussed the relationship between image generation quality and policy success rate. As can be clearly observed in Appendix A.5 and Figure 10, the misalignment (hallucination) in the generated images becomes increasingly severe as the viewpoint offset increases.
Q2: References are missing parentheses
A2: We thank the reviewer for this suggestion. We have reworked dense citations to be parenthesized to improve readability.
Q3: About mistake of PSNR.
A3: We thank the reviewer for identifying this issue. Upon re-examining our implementation, we discovered a bug in the PSNR calculation: although input images were normalized to the range [0, 1], the max_pixel value was inadvertently retained as 255—leading to an incorrect scaling in the PSNR formula. We have included both the original (buggy) and corrected implementations in the supplementary material for full transparency and reproducibility.
Q4: About xyz perturbations.
A4: For our free-view manipulation task, a new viewpoint is resampled in each episode. This viewpoint is randomly offset from the canonical viewpoint in the x, y, and z directions. We have explicitly demonstrated the free-view range in Appendix A.2 of the revised manuscript.