acceptodds
Under review as a conference paper at ICLR 2027

Free-View Manipulation via Dual-Flow Generation Policy: Canonical Images and Actions

Abstract

Visuomotor policies have become the mainstream action generation head in robot manipulation research. However, in open-world environments, the camera viewpoint varies arbitrarily and breaks the perception–action spatial alignment, leading to poor generalization and unstable execution. To address this issue, we propose a novel Dual-Flow Generation Policy (DFGP) that implicitly learns viewpoint-invariant visual representations to adapt to free-view observations and guide accurate manipulation action generation. Our framework consists of two collaborative pathways: an image mapping pathway that infers canonical-view images from free-view inputs via flow matching, and an action generation pathway that predicts robot actions conditioned on the shared canonical visual features. We further introduce a Bi‑directional Cross‑Attention (BiXT) module to adaptively fuse multi-source features for stronger cross-view action generalization. Extensive experiments conducted on two simulation benchmarks (Free-View and LIBERO) —as well as on physical world robotic platforms demonstrate that DFGP achieves statistically significant improvements over prior visuomotor policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Free-View Robot Manipulation, Visuomotor Policy, Imitation Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ecCFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DFPG (Dual-Flow Generation Policy), that implicitly learns viewpoint invariant visual representations as a way to guide accurate action generation under arbitrary viewpoints. It consists of two pathways: image mapping, that infers canonical views using flow matching, and action generation, that predicts robot actions condwitioned on the shared canonical visual features. The authors also introduce a bidirectional cross-attention module designed to fuse multi-source features for stronger cross-view action generation. Experiments on two simulation benchmarks and real-world robots show statistically significant improvements.

Critical strengths

The paper addresses what to me is a clear limitation of current policy methods: sensitivity to camera viewpoint. Most robotics datasets do not contain accurate camera information, which (1) difficults the study of the impacts that introducing such information could have in policy learning, and (2) limits the introduction of benchmarks that leverage such information and accurately measure the degradation of policy performance under different camera placements.

Critical weaknesses

Though interesting and appealing, I wonder if the choice of addressing the problem of image and action generation jointly is reasonable: in these days there are foundation models capable of generating spatio-temporally consistent images and 3D reconstructions from a single view. Such method should be able to shortcut the generation of canonical views, so a traditional policy can be trained without any modification. Even so, I wonder what are the drawbacks of foregoing viewpoint robustness in favor of abstracting it away, so all the heavy lifting is being done by the generative model. A proper study showing improvements in policy performance as image generation improves would be highly relevant here, as well as an analysis of what we lose or gain by relying on canonical images instead of free views (e.g., Are we losing relevant information by observing the scene from a different viewpoint? How far apart can free images be from the canonical view before results degrade? How much impact does hallucination have, since unobserved areas will be inpainted? How should one choose where to place the canonical view?).

Questions for authors

Minor comments:

  • In line 55 it is stated that the main cause for policies failing to generalize to other platforms is camera mounting variation. I understand that this is indeed an issue, but is this really the main cause?
  • The references are missing parentheses, and so author names are mixed with the text, making it hard to read citation-heavy sentences.
  • In Table 3, a PSNR of 80+ is astoundingly high, can you explain how you can generate predictions that close to ground-truth images?
  • What is the reasoning behind the selection for xyz perturbations in Section 4.1?
1 comment

We sincerely thank you for your careful review. Below, we provide a point-by-point response to each of your comments. All revisions in the updated manuscript are highlighted in red for clarity and ease of reference.

W1&Q1: Questions about motivation and drawbacks.

A1: We agree that foundation models have advanced rapidly and can produce temporally consistent images and 3D reconstructions from a single view. However, robotic manipulation still requires a geometric transformation between the robot base coordinate system and the camera coordinate system, namely camera extrinsic calibration. Even a high-fidelity point cloud reconstruction is insufficient for action execution unless it is accurately registered to the robot’s coordinate system. Prior generative visuomotor policies typically assume a fixed camera viewpoint and implicitly absorb the static camera–robot transformation into their learned representations during training. When the camera viewpoint shifts—even slightly—their internal spatial alignment degrades, leading to policy failure. This is one of the main causes for poor cross-platform generalization of visuomotor policies (we have revised this phrasing for rigor at Line 55, other factors include deviations in robot kinematic modeling and differences in low-level control logic).

Compared with a single fixed viewpoint, a free-viewpoint setting inevitably leads to a lower success rate in manipulation when the same amount of training data is used. Fairly speaking, our work trades a certain level of manipulation accuracy for viewpoint flexibility during practice. Following your suggestion, we have added an evaluation of the manipulation success rate under out-of-range viewpoints in Section 4.6, and discussed the relationship between image generation quality and policy success rate. As can be clearly observed in Appendix A.5 and Figure 10, the misalignment (hallucination) in the generated images becomes increasingly severe as the viewpoint offset increases.

Q2: References are missing parentheses

A2: We thank the reviewer for this suggestion. We have reworked dense citations to be parenthesized to improve readability.

Q3: About mistake of PSNR.

A3: We thank the reviewer for identifying this issue. Upon re-examining our implementation, we discovered a bug in the PSNR calculation: although input images were normalized to the range [0, 1], the max_pixel value was inadvertently retained as 255—leading to an incorrect scaling in the PSNR formula. We have included both the original (buggy) and corrected implementations in the supplementary material for full transparency and reproducibility.

Q4: About xyz perturbations.

A4: For our free-view manipulation task, a new viewpoint is resampled in each episode. This viewpoint is randomly offset from the canonical viewpoint in the x, y, and z directions. We have explicitly demonstrated the free-view range in Appendix A.2 of the revised manuscript.

Reviewer_iiUoOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Dual-Flow Generation Policy (DFGP) for calibration-free free-viewpoint manipulation. It pairs a 2D UNet flow-matching branch that reconstructs a 64×64 canonical-view image with a 1D UNet flow-matching branch that predicts action chunks, using BiXT (Hiller et al., 2024) to fuse visual and state features. Experiments on regenerated Free-View tasks, view-perturbed LIBERO suites, and two real-robot tasks report higher success rates than DP, FP, and Calibration DP.

Critical strengths
  1. Using paired canonical-view supervision during training to avoid test-time camera calibration is a practical setup, validated across two simulators and a real robot.
  2. Sharing a single affinity matrix in BiXT improves success rates over concatenation and dual cross-attention in Table 2 without extra attention overhead.
Critical weaknesses
  1. The headline canonical-image flow branch is never ablated, and in Figure 2 and Section 3.2 its output is not even fed into action generation since BiXT takes features directly from the encoder before the 2D UNet (c′=cc' = c). Worse, when BiXT is replaced by concatenation in Table 2 (Concat: 64% / 40% / 24%), the dual-flow model performs worse than the single-flow baseline in Table 1 (FP: 70% / 32% / 50%), suggesting the gains come from the borrowed BiXT block (Hiller et al., 2024) rather than image generation.
  2. Inferring a canonical view from a single uncalibrated RGB image is ill-posed unless the scene barely changes; Section 4.5 admits the target canonical images are highly similar and prone to overfitting, and the Table 3 PSNRs of 82.7 to 96.0 dB (even on blurry real images with missing objects in Figure 6) reflect an unnormalized metric bug.
  3. The main baseline Cali. DP is omitted on all Free-LIBERO and real-world tasks in Table 1 (and is within one deviation on 2 of 4 Free-View tasks), baselines carry no error bars, and Section 4.1 never states whether test viewpoints are unseen.
  4. Algorithm 1 and Section 3.2 contain notation errors that obscure the implementation: Line 10 writes linear training interpolation as Euler(Ic,Tf,t)\text{Euler}(I_c, T_f, t) without noise inputs, Lines 12–13 mix up uaactu_a^{\text{act}} and utactu_t^{\text{act}}, and Section 3.2 states the 1D UNet generates 16×716\times 7 trajectories with 8 already-observed past actions from noise (while Figure 2 shows 4 past and 4 future steps).
Questions for authors
  1. How does Flow Policy + BiXT perform when trained without the image-flow loss (λ1=0\lambda_1 = 0), and why does DFGP + Concat in Table 2 underperform vanilla FP in Table 1?
  2. Is the 2D UNet run at test time at all given that v^tact\hat{v} _ t^{\text{act}} in Figure 2 and Algorithm 1 Line 11 depends only on IfreeI _ {\text{free}} and ThT_h, and how was PSNR computed in Table 3 to reach 96.0 dB?
  3. Are test camera viewpoints held out from training viewpoints, and how does Cali. DP perform on Free-LIBERO where camera extrinsics are known in simulation?
3 comments

We sincerely thank you for your careful review. Below, we provide a point-by-point response to each of your comments. All revisions in the updated manuscript are highlighted in red for clarity and ease of reference.

Q1 & W1: About the ablation experiments of the image stream branches and the doubts about the experimental data in Table 1 and Table 2.

A1: We thank the reviewer again for your expertise. We re-examined and compared Table 1 and Table 2, and found that not only does DFGP+Concat underperform FP, but Dual Attention also appears unreasonably weaker than BiXT. BiXT is merely a variant of Dual Attention. We therefore rechecked the consistency of Free-LIBERO training settings between Table 1 and Table 2. We found that the Free-LIBERO experiments in Table 1 were trained with more samples (collects about 350-400 samples for each tasks) than the ablation experiments (fixed 300 samples) in Table 2.

In the revised manuscript, we re-ran the Free-LIBERO experiments in Table 1 (300 samples).We also added ablations for Concat and BiXT without the image branch in Table 2. All tasks suffer varying degrees of success-rate degradation when the image branch is removed. BiXT achieves comparable performance to Dual Attention while having fewer parameters.

Q2&W2: About 2D UNet at test time and the mistake of PSNR.

A2: The input to the action branch is the output of the shared image feature extractor, rather than the generated canonical-view image. Therefore, the image flow matching can be skipped during inference to reduce computational overhead.

We thank the reviewer for identifying this issue. Upon re-examining our implementation, we discovered a bug in the PSNR calculation: although input images were normalized to the range [0, 1], the max_pixel value was inadvertently retained as 255—leading to an incorrect scaling in the PSNR formula. We have included both the original (buggy) and corrected implementations in the supplementary material for full transparency and reproducibility.

Q3: About train viewpoints and test viewpoints; Add Cali. DP experiments on Free-LIBERO.

A3: The viewpoints of test samples are distinct from those of training samples. During testing, observation viewpoints are resampled within the training range (Table 1). In the revised manuscript, we have added out-of-range experiments in Section 4.6, where test viewpoints are sampled entirely beyond the camera range used for training data collection. DFGP exhibits partial robustness to viewpoint shifts under this setting.

At the same time, we added experiments with the Cali. DP method in Free-LIBERO. Although the Cali. DP and our DFGP are very close in terms of success rate, our method does not require additional camera calibration parameters as auxiliary inputs during testing.

W4: Some notation errors.

A4: In the revised manuscript, we have fixed these errors and marked the revisions in red. We have corrected Line 10 and Line 13 in Algorithm 1. Figure 2 has been re-annotated to reflect that 8 future actions are predicted (the generation of 8 past actions is omitted in the figure).

Thanks for the revision and additional ablations. After reading the updated paper, I still find the contribution and empirical scope too limited for acceptance, so I will keep my score at 2. First, the experimental positioning feels dated and narrow: the paper only compares against small from-scratch policies from a few years ago (BC-T, ACT, DP, FP) without evaluating or integrating with widely used modern VLA policies (such as OpenVLA-OFT, π0\pi _ 0, or π0.5\pi _ {0.5}), and the benchmark coverage is thin, relying on 300-episode subsets of three short-horizon LIBERO suites (omitting LIBERO-Long and other standard benchmarks) plus simple static tabletop tasks while requiring synchronized multi-camera setups during training. Second, inferring a canonical view from one uncalibrated RGB image remains fundamentally brittle: as the new real-world and out-of-range results show, object drift and missing objects quickly degrade performance with no test-time safeguard against hallucinated representations, and most of the gain over Flow Policy still comes from the borrowed BiXT fusion block rather than the canonical-image flow branch itself. Overall, while the revision is a step in the right direction, I feel the work still needs further refinement and broader validation before it is ready for publication.

We thank the reviewer for the careful evaluation and valuable feedback.

First, regarding comparisons with modern VLA policies such as OpenVLA-OFT, π0\pi_0, and π0.5\pi_{0.5}. Our method targets small policy models trained with limited robot demonstration data, which follows a distinct paradigm from VLA models built on pre-trained large vision-language backbones. Fine-tuning these VLA models requires substantial GPU resources, which are unavailable in our lab. Therefore, we choose to compare with lightweight, from-scratch visuomotor policies under the same small-data setting. We regret that further evaluation (e.g., on LIBERO-Long and other standard benchmark) requires collecting paired free-view and canonical-view images, which involves data collection work and cannot be completed within this rebuttal period.

Second, on the robustness of canonical view estimation and the source of performance gains. We acknowledge that predicting canonical views from a single uncalibrated RGB image has inherent limitations. However, compared with the original Flow Policy, our proposed canonical-view flow branch improves the policy’s robustness to viewpoint shifts, including out-of-distribution camera views. Although the BiXT fusion module was proposed in prior work, we are the first to introduce it into robot manipulation policy learning and combine it with our dual-flow policy. As validated by our new ablation studies, our image flow branch brings additional performance gains on top of the BiXT alignment module.

We sincerely appreciate your comments, which help us improve this work.

Reviewer_nYupOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the free-view problem in robot manipulation: changes in camera position alter the spatial correspondence between images and robot actions, degrading the performance of visual motor policies. The authors propose Dual-Flow Generation Policy (DFGP), which uses two generation branches based on flow matching to predict, respectively, the canonical-view image of the current scene and the robot action sequence. Here, the canonical view refers to the observation from a fixed reference camera; training requires synchronized free-view and canonical-view images, while at test time only the free-view observation and the robot's history states are used, without requiring canonical-view images or explicit calibration parameters.

The image branch provides supervision by reconstructing the canonical view into shared visual features, while the action branch fuses the visual features with history states through BiXT bidirectional cross-attention and then generates the action sequence. Experiments cover four Free-View tasks, three re-collected LIBERO task suites, and two real-robot tasks. The authors report success rate improvements over Diffusion Policy, Flow Policy, and, in some settings, Calibration Diffusion Policy, and analyze the method via fusion-module ablations, sampling-step comparisons, and image reconstruction results.

Critical strengths
  1. Using canonical-view supervision during training to improve viewpoint adaptation at deployment has a clear application motivation and a reasonable implementation path.

    Variation in camera mounting position is a real problem in deploying visual robot policies. DFGP uses paired images to provide a unified learning target across different observation viewpoints, and jointly optimizes this visual learning task with action prediction. This provides the model with additional supervision for learning manipulation-relevant cross-view structure, while keeping the deployment interface relatively simple.

    This design is particularly suited to scenarios where additional cameras can be arranged during demonstration collection, but the observation position may need to be adjusted at deployment. The authors also explicitly acknowledge the extra camera and synchronization costs of paired data, and discuss related limitations in the appendix. Its practical contribution can thus be understood as trading collection-stage cost for deployment flexibility, which has clear engineering value (§1, §3.2, Appendix A.5).

  2. The reported performance improvements span multiple experimental environments and include real-robot validation. According to Table 1, DFGP achieves an average success rate of 73% across the four Free-View tasks, compared with 66% for Calibration Diffusion Policy; on Free-LIBERO Spatial, it achieves 58%, higher than Diffusion Policy's 40%. The two real-robot tasks also show improvements: 36% on Pick Bottle, compared with 26% for the strongest reported baseline, and 56% on Select Cube, compared with 30%. These results indicate that the design has practical utility worth further verification.

    The fusion-module ablation also includes meaningful comparisons. In Table 2, BiXT achieves 78%/58%/54% on the three LIBERO suites, versus 70%/56%/46% for independent bidirectional attention, supporting the value of the shared attention structure within the current framework. The authors also show failure cases such as object displacement and omission in real scenes, which help to assess the method's practical boundaries.

Critical weaknesses
  1. The paper's main claim is that canonical-view image generation encourages the model to learn representations beneficial for free-view manipulation. However, Table 2 mainly compares fusion methods, and Table 4 mainly compares the action-generation backbone and sampling steps; neither directly tests performance when the canonical-view image loss is removed while keeping the encoder, BiXT, and action branch unchanged. There is also no deterministic image-reconstruction or cross-view feature-alignment baseline using the same paired images and the same action architecture. The current results therefore cannot distinguish whether the gain comes from the generative image objective, the additional canonical-view supervision, or changes to the fusion architecture.

    Moreover, according to Figure 2 and Algorithm 1, the action branch appears to consume shared encoder features before the image generation module, and the figure does not show the generated canonical-view image feeding into the action branch. This makes the method more consistent with an interpretation of "training a shared encoder with image reconstruction as auxiliary supervision," and it needs to be clarified whether image generation plays a direct role in action inference. The visual comparison of fused features in Figure 5 is also insufficient to demonstrate viewpoint invariance of the visual representation, since the robot history states remain identical across the comparison and may account for a substantial portion of the feature similarity.

    Given that prior work has already adopted cross-view feature alignment, and the shared similarity matrix and bidirectional update mechanism of BiXT also have clear precedents, what this paper most needs to establish is the added value of generative canonical-view supervision over these alternatives. This key evidence is currently still insufficient.

  2. §4.1 specifies the camera position perturbation range for Free-LIBERO and adjusts the camera orientation to keep the target visible, but does not clearly separate the training and test camera sampling distributions. The real-robot experiments only state that the camera mount was randomly moved during collection, without explaining how test camera positions were held out. It is therefore currently impossible to determine whether the results mainly reflect adaptation to newly sampled viewpoints within the training range or generalization to viewpoints beyond it. This distinction directly bears on the paper's conclusions about free-view adaptation, and the appendix does not provide a corresponding split protocol.

    The statistical reporting also limits judgment of the performance lead. The authors explicitly report 50 test episodes per trial and five random-seed repetitions, but Table 1 provides spread only for DFGP, while baselines are given as point estimates; moreover, the "±" denotes deviation between the median and the extremes, and cannot be interpreted as a confidence interval. For example, 78±4% vs. 74% on Arrange Bottle and 82±6% vs. 80% on Push Stick are not yet sufficient to directly support the "statistically significant improvement" stated in the abstract. Repeated results for all methods under consistent evaluation conditions, along with appropriate uncertainty analysis, are needed to determine whether these modest gains are stable.

Questions for authors

What does the action branch actually consume: the shared image encoder output, intermediate states of the image-stream network, or the generated canonical-view image? If the implementation matches Figure 2, would removing the image-generation branch at inference change the action outputs? With the encoder, BiXT, action branch, and training budget held constant, what results are obtained respectively from removing the image loss, and from replacing it with deterministic canonical-view reconstruction or cross-view feature alignment? These controls will directly affect judgment of the necessity of the dual-stream design and of the method's increment.

Please specify the training and test camera distributions in the Free-View, Free-LIBERO, and real-robot experiments. Were specific position or orientation regions held out, or were viewpoints re-sampled within the same range? What is the relationship between the camera positions at real-robot test time and those at collection time? Can you report success rates separately for new viewpoints within the training support and for viewpoints beyond it, to clarify the generalization boundary of the method?

Do all baselines in Table 1 use the same re-collected trajectories and test episodes, with matched visual inputs, history-state length, action prediction horizon, and training budget? Do the five random seeds correspond to independent training runs or repeated evaluation of a fixed checkpoint? Please provide per-seed results for the baselines and DFGP, along with confidence intervals or appropriate tests for the differences between methods, and in particular specify the basis for the claimed "statistically significant improvement."

The PSNR values in Table 3 are 96.0, 95.4, and 82.7 dB. These numbers are very high and need to be reconciled with the visible blur and reconstruction differences in Figure 9. Please clarify the numerical range of the images, the data_range setting used for PSNR, the scaling and normalization scheme, and the aggregation method across images and samples. Were these metrics computed on trajectories and viewpoints held out from training? Providing the corresponding raw MSE values or the evaluation implementation would help assess the reliability of the canonical-view reconstruction evidence.

2 comments

We sincerely thank you for your careful review. Below, we provide a point-by-point response to each of your comments. All revisions in the updated manuscript are highlighted in red for clarity and ease of reference.

Q1&W1: What does the action branch actually consume? & Ablation study of the image-stream network is missing.

A1: The input to the action branch is the output of the shared image feature extractor, rather than the generated canonical-view image. Therefore, the image flow matching can be skipped during inference to reduce computational overhead. It can indeed be understood as “training a shared encoder with image reconstruction as auxiliary supervision”.

However, image generation plays an important role. In the Table 2 of the updated manuscript, we add an ablation experiment of removing the image loss. After removing the image branch, the success rate of free-view task decreases. In the Section 4.6 of the updated manuscript, we show that the manipulation success rate is positively correlated with the image generation results. The visualization results are provided in Appendix A.5, where it can be seen that as the viewpoint offset increases, the object offset in the generated images also gradually increases.

Q2&W2: About training and test camera distributions.

A2: In the updated version, we have added Appendix A.2 to the supplementary material to record the free-viewing ranges for different tasks. For the within range settings (Tables 1 and 2), viewpoints are resampled within the same sampling range as training. Even so, the viewpoints in test episodes are unlikely to exactly coincide with those in the training samples. We have added a new section titled beyond range results in Section 4.6 to evaluate the model’s generalization to viewpoints completely outside the training range. In Table 4, as the viewpoint offset gradually increases, the success rate of the operation continues to decline. Overall, our method is more robust to viewpoint changes.

Q3: About experimental consistency and of “statistically significant improvement”.

A3: The baselines in Table 1 use the exact same training trajectories and test seeds, and the training configurations are identical. However, during verification, we noticed that Tables 1 and 2 do not use the same number of training trajectory samples. We therefore re-standardized and retrained Free-LIBERO experiments, evaluated them using the same random seeds as in Table 2, and provided confidence intervals for both the baselines and DFGP. The experimental results are evaluations of the same checkpoint under different random seeds.

We believe that, based on the experimental results, DFGP is more robust to viewpoint changes than methods that do not account for viewpoint transformation; compared with Cali. DP, our method does not require additional camera parameter inputs. For the sake of rigor, we have removed the word “significant” from the abstract.

Q4: About mistake of PSNR.

A4: We thank the reviewer for identifying this issue. Upon re-examining our implementation, we discovered a bug in the PSNR calculation: although input images were normalized to the range [0, 1], the max_pixel value was inadvertently retained as 255—leading to an incorrect scaling in the PSNR formula. We have included both the original (buggy) and corrected implementations in the supplementary material for full transparency and reproducibility.

Thanks. The action branch reading shared encoder features is now stated, and Table 2 supplies the control I asked for: the image loss removed with everything else fixed. Appendix A.2 and the beyond-range results settle the train/test split; the added confidence intervals and dropping "significant" settle Q3.

Comments on the paper

1 comment

Dear Reviewers,

Thanks for your time and efforts. Please have a look at the authors’ response and let the authors know whether it has addressed the concerns, as the rebuttal deadline is approaching.

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.