acceptodds
Under review as a conference paper at ICLR 2027

Bilinear Flow Policy: Distributional Extrapolation For Goal-conditioned Visuomotor Imitation

Abstract

Goal-conditioned imitation learning (GCIL) with flow matching is a promising framework that can represent multimodal behaviors while adapting to diverse, user-specified goals, yet often fails when goals lie outside the demonstration support. To extrapolate to such unseen goals without collapsing multimodality—a problem we call distributional extrapolation—we introduce Bilinear Flow Policy (BFP), a generative visuomotor policy that combines transductive retrieval with a bilinear conditional flow. Given an unseen observation-goal pair, BFP retrieves an “anchor” training example and transductively reformulates the unseen pair as this familiar anchor plus a residual term. For this decomposition to guide action prediction, the residual must compactly encode how the current observation-goal pair differs from the anchor, and the anchor must be chosen so that this difference is predictive of the corresponding action distribution. BFP achieves this with pretrained visual features and a novel learned anchor-selection algorithm. The novel bilinear flow then models how the anchor and the residual jointly determine the multimodal action distribution. We prove that, for bilinear flow, action distribution error at unseen goals is bounded by the in-distribution flow-matching error up to problem-dependent factors. Across five manipulation tasks in simulation, BFP achieves 2.63× the out-of-distribution success rate of a GCIL policy and 1.36× that of the strongest extrapolation-targeted baseline. On two real-world tasks, BFP improves over GCIL by 32 percentage points on average. Finally, our theory yields practical, pre-deployment diagnostics for predicting which trained policies will extrapolate well and to which unseen goals. Project website: https://bilinearflow.netlify.app

Primary area:
applications to robotics, autonomy, planning
Keywords:
Goal-conditioned imitation learning, Out-of-distribution generalization, Flow matching, Visuomotor policy learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wbsbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Bilinear Flow Policy addresses goal-conditioned visuomotor imitation when numerical goals lie outside the demonstrated range. It retrieves a demonstration anchor, computes the difference between the query and anchor representations, and conditions a flow-matching policy through a low-rank bilinear interaction between these two factors. A learned scorer selects anchors within phase-compatible candidates, while deployment-time filtering favors familiar goal differences. Experiments include controlled multimodal regression, five simulated manipulation tasks across seven settings, and two real-robot tasks. The paper also provides a conditional extrapolation bound and investigates flow-loss and residual-coverage diagnostics.

Critical strengths
  1. The architectural constraint is well motivated by the extrapolation problem. Rather than simply adding retrieved examples to a conventional policy, BFP explicitly separates anchor and residual processing and restricts their interaction. This provides a mechanism for reusing learned relationships across previously unseen combinations. Applying the structure to a flow field extends deterministic bilinear prediction to multimodal action generation, and the controlled regression experiment provides direct evidence for this capability.

  2. The empirical gains are substantial, and the paper investigates their contributing components. BFP achieves approximately 79% average visuomotor OOD success in simulation, compared with 30% for standard Flow and 58% for BRP, with additional improvements on two real-robot tasks. Beyond aggregate performance, the paper examines retrieval, pretrained visual representations, predictor backbones, and bilinear rank. The controlled regression experiment separately investigates multimodal prediction, making the evaluation informative about both effectiveness and design choices.

Critical weaknesses
  1. The contribution of the bilinear interaction is not fully separated from the retrieval procedure. BFP differs from standard Flow in both its conditioning inputs and architecture, while the BRP comparison changes both anchor selection and deterministic versus generative prediction. DARP provides a retrieval-based flow baseline, but uses a different retrieval procedure and extrapolation rule, so it does not isolate the bilinear interaction under identical conditioning. The retrieval ablations demonstrate that anchor selection matters, but do not by themselves establish how much of the improvement requires the bilinear flow formulation. A matched non-bilinear flow predictor receiving the same anchors and residuals would make the architectural claim more convincing.

  2. The reliability of anchor retrieval under deployment shift remains insufficiently characterized. BFP relies on phase-compatible references and on query–anchor differences remaining meaningful in the chosen representation. Although the retrieval ablations show an advantage over synchronized and random selection, they do not establish how sensitive the method is to phase-estimation errors, limited reference coverage, or visual differences between the query and stored demonstrations. These dependencies matter because the claimed extrapolation benefit requires a suitable anchor–residual decomposition. Deployment also requires retaining a compatible reference dataset, which limits portability relative to a standalone policy checkpoint.

Questions for authors
  1. Additional comparison: How does BFP compare with a non-bilinear flow predictor receiving the same anchor and residual, with the same retrieval procedure? This would help isolate the architectural contribution from the benefit of retrieval-based conditioning.

  2. Clarification: What per-task rules and inputs determine phase at deployment? How does anchor selection handle delayed progress or failed actions, and is there evidence measuring sensitivity to phase-estimation errors?

  3. Robustness evidence: How sensitive is performance to the size and composition of the demonstration reference set? Evidence from reduced reference sets or queries with poor anchor coverage would help establish when retrieval remains useful and when extrapolation breaks down.

6 comments
AuthorsOpenReview ↗

Response to reviewer wbsb (1/3)

We sincerely thank the reviewer for the careful reading and constructive suggestions. We are grateful that the reviewer recognized the well-motivated bilinear architecture, its extension to multimodal action generation, the substantial empirical gains, and the analysis of the method’s design choices. Below, we address the reviewer’s concerns.

W1. Missing Bilinear Head Control

We thank the reviewer for suggesting this experiment. To address this, we evaluated a matched non-bilinear flow predictor.

We froze BFP’s learned retrieval scorer and trained a parameter-matched, non-bilinear flow predictor using the same anchor and residual inputs. The non-bilinear predictor uses a flow action head conditioned on the concatenation of the resulting anchor and residual, in place of the bilinear product. We evaluate across seven simulation settings with state and visuomotor observations (3 training seeds, 100 OOD rollouts)

Result.

MethodOOD successOOD reward
Flow36.4 ± 4.835.1 ± 4.4
BFP (non-bilinear)40.9 ± 5.537.5 ± 3.7
BFP84.7 ± 5.167.9 ± 3.9

Takeaway. Compared with Flow, the matched non-bilinear control provides a modest OOD success improvement (40.9% vs. 36.4%), whereas BFP achieves 84.7%—43.8 percentage points higher than the control. This supports a contribution from the bilinear interaction beyond retrieval-based conditioning.

W2. Sensitivity to anchor retrieval

We appreciate reviewer for suggesting more analysis on sensitivity of BFP to anchor retrieval. We evaluate sensitivity to phase retrieval (W2a, W2b), reference coverage (W2c), visual shift (W2d). We additionally address the deployment concern (W2e).

W2a. Robustness under phase-estimation errors

TL;DR. BFP is robust to phase-estimation error

We agree that phase-estimation errors may lead to unsuitable candidates. However, the phase estimator proposes a coarse candidate window, and the learned retrieval policy selects an anchor within it, allowing BFP to tolerate errors in the tested settings. To examine this, we constructed a noisy estimator that sometimes selects the wrong phase and perturbs estimated progress, with errors increasing with noise scale σ. We evaluated BFP with a noisy estimator on PushCube-D0, PushCube-D1, Relocate-D0, Relocate-D1, StackCube, and PickCube (3 seeds, 100 OOD rollouts per seed) We measure temporal intersection over union (tIoU) between the noisy and clean candidate windows, using an interval-overlap metric common in temporal action localization [1, 2].

Result.

Method / Noise σtIoUOOD successOOD reward
BFP (σ = 0) (clean)1.0073.4 ± 1.457.0 ± 0.8
BFP (σ = 0.1)0.7970.8 ± 1.858.4 ± 0.7
BFP (σ = 0.25)0.5968.2 ± 1.156.7 ± 0.7
BFP (σ = 0.5)0.4366.2 ± 1.256.4 ± 0.8
BFP + Sync Retrieval—60.6 ± 1.649.4 ± 0.7
BFP + Random Retrieval—62.6 ± 1.452.1 ± 0.8
BT—58.2 ± 0.648.3 ± 0.4
Flow—26.1 ± 1.226.7 ± 0.4

At σ = 0.5, temporal IoU falls to 0.43, below 0.5, a commonly used matching threshold in temporal action localization. BFP achieves 66.2% OOD success versus 73.4% without noise and remains above the clean-clock BFP + Sync Retrieval (60.6% / 49.4), BFP + Random Retrieval (62.6% / 52.1), BT (58.2% / 48.3), and Flow (26.1% / 26.7) references on average across tasks.

Takeaway. The phase estimator proposes a coarse window, and the learned retrieval policy selects an anchor within it. The results suggest tolerance to the tested errors, although success declines as noise increases. However, severe misalignment may exclude all useful candidates and degrade performance, so selecting a good-enough phase estimator remains important. We provide further details in Appendix F.5 of the manuscript.

References.

[1] Idrees et al., The THUMOS Challenge, CVIU 2017.
[2] Caba Heilbron et al., ActivityNet, CVPR 2015.

AuthorsOpenReview ↗

Response to reviewer wbsb (2/3)

W2b. BFP with a learned phase estimator

TL;DR. BFP maintains performance with learned phase estimator

We agree that a hand-designed phase estimator may be difficult to use in complex tasks. However, BFP can also use learned alternatives. We evaluated SARM [1] and CaR [2] on PushCube-D1, Relocate-D1, PickCube, and StackCube (3 seeds, 100 OOD episodes per seed). Notably, SARM was developed for long-horizon, contact-rich manipulation, including T-shirt folding [1].

Result.

MethodOOD successOOD reward
BFP + hand-designed61.2 ± 2.050.6 ± 1.1
BFP + SARM60.7 ± 1.451.5 ± 0.7
BFP + CaR56.0 ± 2.049.5 ± 1.0
BT44.9 ± 1.541.1 ± 0.8
Flow26.8 ± 1.630.5 ± 0.6

Takeaway. BFP with SARM nearly matches the hand-designed estimator (60.7% vs. 61.2% OOD success), and both learned estimators outperform BT and Flow. These results support the use of learned phase estimators, while evaluating BFP on long-horizon, contact-rich tasks remains future work. We added more details in Appendix F.4 of the manuscript.

References.

[1] Chen et al., SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation, ICLR 2026
[2] Venuto et al., Code as Reward, ICML 2024.

W2c. Relation to reference coverage

TL;DR. (1) Dataset is the reference bank. (2) Query with less support extrapolates less. (3) BFP is better than FP in all data-regime.

  • We agree that reference coverage could be important for retrieval and extrapolation. We clarify in the manuscript that the reference bank is the training demonstration dataset itself; BFP does not require a separate reference dataset.

  • However, for a given training dataset, some queries may be poorly supported by that dataset; our delta-coverage diagnostic (Fig. 5a) associates poorer coverage with higher OOD flow-matching error.

  • A smaller training dataset may reduce this support, so we varied the demonstration fraction from 12.5% to 100% on six simulation tasks. We train BFP and Flow on the same subset of demonstrations across three seeds; BFP also uses this subset as its reference bank. We evaluate on 50 OOD episodes per seed on PushCube/Relocate and 100 on PickCube/StackCube.

Result. Task-averaged OOD success vs Dataset Size:

Method100%75%50%25%12.5%
BFP72.866.658.747.833.4
Flow24.025.819.715.08.1

BFP achieves higher task-averaged OOD success than Flow at every dataset size. With 12.5% of the demonstrations, BFP reaches 33.4% versus 8.1% for Flow.

Takeaway These results suggest that BFP can be effective in the low-data regime, although sensitivity varies by task. Because the training data and references shrink together, the experiment affects both the predictor and retrieval and does not isolate retrieval sensitivity. Please see Appendix F.9 for per-task details.

W2d. Robustness under visual shifts

We agree that visual differences between queries and stored demonstrations may affect retrieval. However, BFP can still outperform Flow under the tested shifts. We evaluated BFP and Flow on unseen goals under additional visual shifts in object color, light color, camera pose, and distractors on PushCube-D1 and Relocate-D1 (3 seeds, 100 OOD episodes per seed), ablating pretrained visual representations (PVRs), specifically frozen R3M and DINOv3 [1–3]. These shifts are introduced only at deployment and affect the test-time query observations.

Result. Task-averaged OOD success (%; mean ± SEM):

EncoderPolicyCleanObject colorLight colorCamera poseDistractors
R3MFlow25.8 ± 0.716.7 ± 0.65.3 ± 1.513.0 ± 0.619.2 ± 1.4
R3MBFP65.2 ± 1.736.5 ± 1.58.5 ± 1.229.8 ± 1.235.5 ± 2.2
DINOv3Flow32.2 ± 1.232.2 ± 0.630.7 ± 1.419.7 ± 0.625.0 ± 1.3
DINOv3BFP68.5 ± 1.071.3 ± 1.740.0 ± 3.631.3 ± 0.843.8 ± 2.9

Result. BFP exceeds Flow on average under each tested shift, but performance can decline substantially. For example, under light-color changes, success falls from 65.2% to 8.5% with R3M and from 68.5% to 40.0% with DINOv3. We provide detailed experimental results in Appendix F.8.

Takeaway. These results show that BFP can generalize to unseen goals under additional deployment-time visual shifts, while robustness to these shifts also depends on the encoder. A more robust representation can help, while severe visual changes may remain a limitation.

References.

[1] Pumacay et al., The Colosseum, RSS 2024. [2] Nair et al., R3M, CoRL 2022. [3] Siméoni et al., DINOv3, 2025.

AuthorsOpenReview ↗

Response to reviewer wbsb (3/3)

W2e. Deployment concern

We agree that retaining reference data adds a deployment cost. However, the memory/time cost is modest.

  • Memory. BFP reuses training demonstrations and caches compact latent features rather than raw images. For PnPMug and InsertPipette, 127 and 133 demonstrations occupy 173 MB and 313 MB, respectively. Feature-bank memory scales linearly, O(F), with the number of stored frames.
  • Time. At inference, BFP scores a small phase-compatible candidate window. For fixed task settings, per-plan retrieval cost is O(1) in F. On an RTX 4090 GPU, retrieval takes 21.9/22.7 ms and policy prediction takes 68.9/69.4 ms on PnPMug / InsertPipette, respectively.

The measured storage and retrieval overhead is modest in these settings, while feature-bank storage grows with the number of stored frames.

Questions

Q1. Bilinear head control

  • We provide additional ablation. Please see W1 for details.

Q2. Clarification of Retrieval Policy

  • Thank you for pointing out our misclarification on per-task phase rule. We updated the manuscript to include the used per-task hand-designed phase rule in Appendix D.
  • As reviewer pointed out, such hand-designed estimator is vulnerable to delayed progress or failed actions. Hence, we show BFP can be used with the learned phase estimators that can address delayed progress, failed actions (W2b).
  • We also provide sensitivity to phase-estimation errors (W2a).

Q3. Robustness evidence of BFP

  • We provide additional experiments. Please see W2c for details.
Reviewer_EhKFOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies goal-conditioned visuomotor imitation when test-time goals lie outside the demonstration support, while preserving multimodal action distributions (“distributional extrapolation”). BFP retrieves an anchor demonstration for each query and computes a residual in a frozen pretrained visual feature space. It then parameterizes the flow-matching velocity field as a low-rank bilinear interaction between anchor and residual features. Anchors are selected by a learned scorer over phase-compatible candidates, trained on held-out BFP loss. The authors derive a bound relating in-distribution flow-matching error to the Wasserstein error of generated actions at unseen goals, and use it to motivate two pre-rollout diagnostics. Experiments on seven simulated settings and two real-robot tasks report higher OOD success than flow-matching GCIL and several extrapolation baselines.

Critical strengths
  • The problem is clearly defined: the method must both extrapolate and handle multimodality. The toy example in Fig. 2 makes this intuitive.
  • The experiments are extensive: 7 simulation settings, 4,200 rollouts, standard statistical tests with FWER control, and two real-robot tasks.
  • The improvement over BRP, EquiDP, DARP, and CTA is clear.
Critical weaknesses
  1. The theoretical novelty is limited. It essentially applies an existing Theorem 4 directly to each (c, k) and adds a standard Gronwall argument. The assumptions are very strong and cannot be verified.
  2. There is no ablation isolating the contribution of the bilinear structure. Performance is nearly unchanged across ranks from 32 to 512, and r=512 is hardly low-rank. This weakens the core claim that low-rank structure enables extrapolation. The missing control is: use the same representation, the same learned retrieval, and the same (δ, z†) inputs, but replace the bilinear head with a non-bilinear one. DARP’s different retrieval method does not substitute for this control.
Questions for authors

Please refer to the weaknesses above.

3 comments
AuthorsOpenReview ↗

Response to reviewer EhKF

We sincerely thank the reviewer for the careful reading and constructive suggestions. We are grateful that the reviewer recognized the clearly defined problem, the illustrative toy example, the extensive simulation and real-robot experiments, and the improvements over baselines. Below, we address the reviewer’s concerns.

W1. Theoretical Novelty of BFP

We thank the reviewer for the careful assessment. We agree that Propositions 7 and 8 build on existing bilinear extrapolation bounds and standard ODE stability arguments. However, our contribution is to connect these ingredients to distributional generation with conditional flows.

We also identify the additional assumptions (Eqs. 58 and 59) and provide a Gaussian example in which bilinear structure in action means carries over to flow velocity (Proposition 11).

We agree that the assumptions are strong and not directly verified. Nevertheless, BFP demonstrates substantial empirical OOD improvements, including over the parameter-matched non-bilinear control (W2). We have clarified the flow-specific contribution, and these limitations in Sec. 4.1 and Appendix C.

W2. Low-Rank Bilinear Structure Clarification and Missing Bilinear Head Control

We apologize for the confusion about our core claim. We clarified in the manuscript that the low-rank bilinear structure enables extrapolation. The rank sweep (Fig. 5d) shows robustness across the tested ranks; it does not establish that low rank itself causes extrapolation.

We also agree that the original comparisons did not isolate the bilinear head from retrieval-based conditioning. To address this, we evaluated a matched non-bilinear flow predictor.

We froze BFP’s learned retrieval scorer and trained a parameter-matched, non-bilinear flow predictor using the same anchor and residual inputs. The non-bilinear predictor uses a flow action head conditioned on the concatenation of the resulting anchor and residual, in place of the bilinear product. We evaluate across seven simulation settings with state and visuomotor observations (3 training seeds, 100 OOD rollouts)

Result:

MethodOOD successOOD reward
Flow36.4 ± 4.835.1 ± 4.4
BFP (non-bilinear)40.9 ± 5.537.5 ± 3.7
BFP84.7 ± 5.167.9 ± 3.9

Takeaway. Compared with Flow, the matched non-bilinear control provides a modest OOD success improvement (40.9% vs. 36.4%), whereas BFP achieves 84.7%—43.8 percentage points higher than the control. This supports a contribution from the bilinear interaction beyond retrieval-based conditioning

—

Thank you again for the constructive feedback. We welcome further questions and clarification.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer EhKF,

As the discussion period ends soon, we wanted to kindly check whether our response has addressed your concerns. We would be happy to clarify anything further.

Best regards, The Authors

Hi, thanks for the detailed response. I will keep my original score

Reviewer_9DvmOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Bilinear Flow Policy to improve the generalization of goal-conditioned visuomotor imitation learning to unseen goals. The method combines anchor retrieval, anchor-residual decomposition, and bilinear conditional flow matching to generate multimodal action distributions. The authors also provide theoretical analysis and experiments on simulation benchmarks and real-world manipulation tasks demonstrate significant improvements over existing baselines.

Critical strengths

Combining bilinear transduction with conditional flow matching is interesting. The learned anchor retrieval mechanism also provides a useful way to improve extrapolation.

The paper provides a theoretical bound on distributional extrapolation, which helps explain the motivation behind the proposed method.

Generalizing to unseen goals is an important challenge in robot learning, and the proposed method achieves improvements over several baselines on simulation experiment.

Critical weaknesses

The retrieval policy relies heavily on phase compatible anchors. However, task phases may be difficult to identify, especially during contact-rich manipulation, long-horizon tasks, or failure recovery. The paper uses a hand-designed phase estimator but does not sufficiently evaluate the robustness of BFP to incorrect phase estimation.

Although BFP does not require predefined geometric transformations, the experiments still rely on explicitly specified goals, such as positions or poses. Real-world experiments also use RGB-D, SAM3, FoundationPose, and camera calibration to obtain structured goals. Therefore, it remains unclear how well BFP generalizes to language-conditioned goals, unseen objects, or more complex open-world manipulation tasks.

Limited real-world evaluation with only two manipulation tasks, only a single training seed per method. More diverse tasks and repeated training runs would help validate the robustness of the proposed method.

Questions for authors

How sensitive is BFP to inaccurate task-phase estimation? What happens when the retrieved anchor belongs to an incorrect task phase?

Can BFP be extended to more complex goals, such as language instructions or image-based goals, without explicit goal coordinates?

4 comments
AuthorsOpenReview ↗

Response to reviewer 9Dvm (1/2)

We sincerely thank the reviewer for the careful reading and constructive suggestions. We are grateful that the reviewer recognized the importance of generalizing to unseen goals, the combination of bilinear transduction and flow matching, and the improvements over simulation baselines. Below, we address the reviewer’s concerns.

W1. Robustness of BFP to incorrect phase estimation

We agree that phase estimation is challenging when task progress is ambiguous. However, BFP only needs a coarse window of candidates, from which its learned retrieval policy selects an anchor. We therefore evaluate alternative phase estimators (W1a) and controlled phase errors (W1b) below.

W1a. BFP with a learned phase estimator

TL;DR. BFP maintains performance with learned phase estimator

We agree that a hand-designed phase estimator may be difficult to use in complex tasks. However, BFP can also use learned alternatives. We evaluated SARM [1] and CaR [2] on PushCube-D1, Relocate-D1, PickCube, and StackCube (3 seeds, 100 OOD episodes per seed). Notably, SARM was developed for long-horizon, contact-rich manipulation, including T-shirt folding [1].

Result:

MethodOOD success (%)OOD reward
BFP + hand-designed 61.2 ± 2.050.6 ± 1.1
BFP + SARM60.7 ± 1.451.5 ± 0.7
BFP + CaR56.0 ± 2.049.5 ± 1.0
BT44.9 ± 1.541.1 ± 0.8
Flow26.8 ± 1.630.5 ± 0.6

BFP with SARM nearly matches the hand-designed estimator (60.7% vs. 61.2% OOD success), and both learned estimators outperform the baselines: BT and Flow. These results support the use of learned phase estimators, while evaluating BFP on long-horizon, contact-rich tasks remains future work. We added more details in Appendix F.4 of the manuscript.

References.

[1] Chen et al., SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulation, ICLR 2026
[2] Venuto et al., Code as Reward: Empowering Reinforcement Learning with VLMs, ICML 2024.

W1b. Robustness under phase-estimation errors

TL;DR. BFP is robust to phase-estimation error

We agree that phase-estimation errors may lead to unsuitable candidates. However, the phase estimator proposes a coarse candidate window, and the learned retrieval policy selects an anchor within it, allowing BFP to tolerate errors in the tested settings. To examine this, we constructed a noisy estimator that sometimes selects the wrong phase and perturbs estimated progress, with errors increasing with noise scale σ. We evaluated BFP with a noisy estimator on PushCube-D0, PushCube-D1, Relocate-D0, Relocate-D1, StackCube, and PickCube (3 seeds, 100 OOD episodes per seed). We measure temporal intersection over union (tIoU) between the noisy and clean candidate windows, using an interval-overlap metric common in temporal action localization [1, 2].

Result:

MethodtIoUOOD success (%)OOD reward
BFP (σ = 0)1.0073.4 ± 1.457.0 ± 0.8
BFP (σ = 0.1)0.7970.8 ± 1.858.4 ± 0.7
BFP (σ = 0.25)0.5968.2 ± 1.156.7 ± 0.7
BFP (σ = 0.5)0.4366.2 ± 1.256.4 ± 0.8
BFP + Sync Retrieval—60.6 ± 1.649.4 ± 0.7
BFP + Random Retrieval—62.6 ± 1.452.1 ± 0.8
BT—58.2 ± 0.648.3 ± 0.4
Flow—26.1 ± 1.226.7 ± 0.4

At σ = 0.5, the noisy candidate window has 0.43 tIoU with the clean window, and BFP achieves 66.2% OOD success versus 73.4% without noise. BFP also remains above the BFP + Sync Retrieval (60.6% / 49.4), BFP + Random Retrieval (62.6% / 52.1), BT (58.2% / 48.3), and Flow (26.1% / 26.7) references on average across tasks.

Takeaway. The phase estimator proposes a coarse window, and the learned retrieval policy selects an anchor within it. The results suggest tolerance to the tested errors, although success declines as noise increases. However, severe misalignment may exclude all useful candidates and degrade performance, so selecting a good-enough phase estimator remains important. We provide further details in Appendix F.5 of the manuscript.

References.

[1] Idrees et al., The THUMOS Challenge, CVIU 2017.
[2] Caba Heilbron et al., ActivityNet, CVPR 2015.

AuthorsOpenReview ↗

Response to reviewer 9Dvm (2/2)

W2. BFP with non-explicit goals

W2a. Scope clarification: single-task generalization

We agree that BFP currently uses compact, explicit goals, obtained through a perception pipeline in our real-world experiments, and does not establish open-world generalization. However, extrapolation to unseen positions and poses remains an important challenge within this scope, as noted by STAR-Gen [1]. We clarified the evaluated goal setting in the manuscript.

W2b. Image-goal extrapolation and bilinear adequacy

We agree that generalization to image goals is not established by the explicit-goal experiments. As noted in our limitations, BFP relies on compact goal representations. We investigate this limitation through a negative result with image goals and analyze a possible explanation. We therefore evaluated BFP and Flow with frozen R3M or DINOv3 image goals on PushCube-D1 and Relocate-D1. Image goals are demonstration final frames; at evaluation, they come from successful expert demonstrations with the same initial condition and target. We report OOD success and normalized reward across three seeds, with 100/120 OOD episodes per seed on the two tasks, respectively. Appendix F.7 provides the details. We also report BFP’s held-out ID flow-matching error as a theory-motivated diagnostic (Fig. 5a).

Result.

Goal inputMethodOOD success (%)OOD reward
Explicit goalBFP94.6 ± 2.771.3 ± 1.2
Explicit goalFlow35.3 ± 4.629.7 ± 3.7
R3M imageBFP13.9 ± 0.218.7 ± 0.7
R3M imageFlow18.4 ± 1.319.2 ± 1.2
DINOv3 imageBFP2.5 ± 0.35.3 ± 0.4
DINOv3 imageFlow5.2 ± 0.96.1 ± 0.5

BFP held-out ID flow-matching error (×10⁻³; mean ± SEM; lower is better):

Goal inputPushCube-D1Relocate-D1
Explicit goal0.90 ± 0.073.22 ± 0.16
R3M image1.98 ± 0.335.07 ± 0.33
DINOv3 image3.62 ± 0.3412.40 ± 0.40

Takeaway. BFP outperforms Flow with explicit goals, but this advantage disappears with image goals. Held-out ID flow-matching error also increases with image goals. We hypothesize that image-goal embeddings do not define a meaningful goal residual, making the anchor–residual-to-action relationship less compatible with (S) Bilinear Structure. The observed association does not establish the cause. We clarified this limitation in the manuscript and discuss a potential way to alleviate it in W2c.

W2c. Extension to language and open-world tasks

We agree that the current experiments do not test language instructions or unseen objects. However, we envision a modular system in which a vision-language model grounds instructions and objects into compact spatial goals and BFP serves as the action-generation head. RT-Affordance [2] and GHOST [3] illustrate related grounding approaches. Integrating such modules and evaluating unseen objects or instructions remain future work.

References (W2).

[1] Gao et al., STAR-Gen, RA-L 2025

[2] Nasiriany et al., RT-Affordance, ICRA 2025

[3] Krishna et al., GHOST, RSS 2026

W3. Limited hardware task diversity and training replication

We agree that two hardware tasks and one training seed per method limit the evidence for real-world robustness. Our simulations provide complementary statistical evidence across five tasks and seven settings, with three training seeds per method, 4,200 rollouts, and 5% family-wise error control. We agree that additional hardware tasks and training seeds would strengthen the evidence and are important directions for future work.

Q1. How sensitive is BFP to phase estimation error?

Please see W1a and W1b.

Q2. Can BFP take language / image-based goal?

Please see W2a-W2c.

AuthorsOpenReview ↗

Gentle Reminder

Dear Reviewer 9Dvm,

As the discussion period ends soon, we wanted to kindly check whether our response has addressed your concerns. We would be happy to clarify anything further.

Best regards, The Authors

AuthorsOpenReview ↗

Additional Result: Hardware Result

W3. Limited hardware task diversity

We introduce a third real-world task EraseWhiteboard, in which the robot picks up an eraser and wipes the word "OOD” from a whiteboard. We use the center position of the word as the policy goal. We collect 190 expert demonstration, and evaluate Flow Policy and BFP on 30 OOD initial conditions.

Result. Out-of-distribution performance: Success rate (SR, %) measures complete erases. Dense reward (×100) measures percentage of ink removed to a labeled partial erase.

MethodMetric
FlowSR2020
Reward2020
BFPSR33\mathbf{33}
Reward48\mathbf{48}

Due to limited time in the rebuttal period, our evaluation is less extensive. In the camera-ready paper, we will evaluate with 50 ICs, 2 trials per IC, and evaluate Bilinear Transduction (BT).

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their thoughtful feedback, which has helped us improve our paper.

Below, we summarize the key updates:

  • Bilinear-head ablation: Added a non-bilinear flow predictor using the same retrieved anchors and residuals.
  • Robustness to phase estimation: Evaluated BFP with learned phase estimators and controlled phase-estimation errors.
  • Additional robustness analysis: Evaluated BFP with deployment-time visual shifts and varying dataset size.
  • Manuscript clarifications: Clarified our theoretical assumptions, the role of bilinear structure, and the scope of extrapolation with explicit goals. We updated the manuscript and provided the response below.

If reviewers have any further questions, please feel free to ask—we would be very happy to discuss them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.