acceptodds
Under review as a conference paper at ICLR 2027

Dynamism of a Trajectory: Encoding Coarse Intent into the Source of Flow-Matching Policies

Abstract

Robot policies are often trained on a relatively small corpus of demonstrations of a few tasks. In such low-data regimes, generative policies do not need to learn how their embodiments move because reproducing demonstrated trajectories is enough to fit the data. As a result, such policies tend to be competent only along those routes. Therefore, to make best use of limited demonstrations, we separate coarse and fine motion by first fixing an orthonormal basis for the coarse part of an action chunk where the coordinates of the basis represent displacement over time, which we call the command. Then, we modify the initially drawn Gaussian noise by replacing its component along the coarse motion basis with the command. The command is constructed to be unchanged along the training path and absent from the regression target, leaving the flow to learn the residual, fine motion. While a learned head predicts the command by default, an external planner or human operator can override it, and a scalar input allows inference-time modulation of command adherence. We evaluate our method on a quadrotor platform on gate navigation and out-of-distribution maneuvers. On the demonstrated tasks, our policy improves success in simulation from 45% to 73%, where the images differ from those seen in training. Our method also enables flexible out-of-distribution command steering for drone maneuvers through and around gates, such as orbits and figure-eights, in simulation and on hardware. Finally, we show a preliminary demonstration in which an agent pilots the policy by approving or overriding its proposed commands.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action policies, flow-matching, quadrotor, source distribution, controllable generation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_KrLhOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a coarse-to-fine approach for pretrained flow-matching VLA policies by modifying the sampled Gaussian noise. The method constructs an orthonormal basis for coarse trajectory commands and trains the flow-matching model to generate residual fine-grained motion. This separates where to go from how to move. The authors evaluate the method on quadrotor gate navigation and out-of-distribution maneuvers in simulation and the real world.

Critical strengths
  1. Interesting mathematical formulation. The method uses the flow-matching interpolation to preserve coarse trajectory commands rather than relying on additional conditioning or inference-time guidance.

  2. Clear motivation. The separation of where to go and how to move is intuitive, and the mathematical formulation is easy to follow.

  3. Real-world evaluation. The authors evaluate the policy on a physical quadrotor, including unseen orbit and figure-eight trajectories. They also use Gaussian Splatting to reconstruct the environment for simulation and evaluate robustness under visual distribution shift.

Critical weaknesses
  1. Limited evaluation trials. In Figure 7, the real-world OOD experiments include only five trials per task and lack baseline comparisons. The simulation experiments also use only ten trials for orbit and figure-eight tasks. More trials and real-world baseline comparisons would strengthen the results.

  2. Choice of coarse motion basis. The authors manually construct the 16-dimensional command space. A comparison with PCA-based representations extracted from demonstrations would help validate this design choice.

  3. Limited task diversity. The evaluation mainly focuses on quadrotor gate navigation. Additional tasks with different motion requirements would make the generalization claims more convincing.

Questions for authors
  1. Missing ablation on command prediction. In Section 5.2, the authors claim that low-dimensional coarse motion is easier to predict under visual distribution shift. It would help to compare against a baseline that predicts the full action chunk with a comparable prediction head to verify this claim.

  2. Additional steering baselines. It would strengthen the paper to compare against other steering approaches, such as residual flow steering or learned noise-space steering methods.

2 comments
AuthorsOpenReview ↗

Response to Reviewer KrLh (Part 1/2)

We thank Reviewer KrLh for their reading and constructive feedback. We are glad the reviewer enjoyed the mathematical formulation and motivation of our work. We address each concern and question below.

W1: Limited evaluation trials

In Figure 7, the real-world OOD experiments include only five trials per task and lack baseline comparisons. The simulation experiments also use only ten trials for orbit and figure-eight tasks. More trials and real-world baseline comparisons would strengthen the results.

We agree with the reviewer and have expanded the simulated orbit and figure-eight cells of Table 1 to 50 flights per method (Table R3).

Table R3. Orbit and figure-eight routes of Table 1 (50 flights per method). Err.: median tracking error.

MethodOrbit: successOrbit: touchedOrbit: err. (cm)Fig.-eight: successFig.-eight: touchedFig.-eight: err. (cm)
π0\pi_0 + sketch (baseline)7/504374.43/504743.8
SDEdit37/5005.747/5036.1
Vel. Proj.50/5001.050/5001.4
Ours50/5004.550/5006.2

Our policy completes 50/50 orbits and 50/50 figure-eights, whereas the π0\pi_0 fine-tune with injected sketch completes 7/50 and 3/50, respectively. SDEdit (37/50, 47/50) and velocity projection (50/50, 50/50) maintain high success rates as well, consistent with Table 1. On hardware, we flew only our policy on the sketched routes for practical reasons: in simulation, the π0\pi_0 baseline with the same sketch injected touches a gate in 43/50 orbits and 47/50 figure-eights, with median tracking errors of 44 to 74 cm. Flights that far off a route through gates risk catastrophic collision, which we deemed too costly to attempt. We agree that five hardware flights per route is a small sample. We will be sure to state that the hardware results demonstrate feasibility (that our policy can fly these routes) and are more of a robustness test than a contribution evaluation.


W2: Choice of coarse motion basis

The authors manually construct the 16-dimensional command space. A comparison with PCA-based representations extracted from demonstrations would help validate this design choice.

We thank the reviewer for pointing this out, and we add a comparison of our command space against PCA bases (Table R4). We find that a 16-dimensional PCA basis captures more of the demonstrated motion and reproduces smooth translations more closely, but it cannot express motions rarely present in the training data. For example, commanding a 45 degree turn in place is off by about 34 degrees compared with 7 degrees for the windowed basis. We chose a basis that does not depend on the demonstrations because we want the command space to cover motions outside the training data. We would also like to note that the choice of command space is an interesting question in its own right, and we leave a systematic study as future work. We will add these comparisons in a revision.

Table R4. Expressing simple motions on the real demonstrations. We write each motion as a command, decode the command back into a path, and report the largest deviation from the intended motion (median over directions and over completing the motion in 20, 35 or 50 steps).

BasisStay still: error (cm / deg)Move 1 m horizontally: error (cm)Move 0.5 m up or down: error (cm)Turn 45° in place: error (deg)
PCA, K=16K = 160.3 / 22.02.434
Windows (ours), K=16K = 160.0 / 014.77.37
AuthorsOpenReview ↗

Response to Reviewer KrLh (Part 2/2)

W3: Limited task diversity

The evaluation mainly focuses on quadrotor gate navigation. Additional tasks with different motion requirements would make the generalization claims more convincing.

We agree that the evaluation centers on quadrotor gate navigation, and we will state this as a limitation. We note that our construction needs only a linear map UU from action chunks to a low-dimensional subspace, so other embodiments may use the same training objective with a customized UU. Within the drone setting, we acknowledge that our tasks are all navigation and trajectory-following tasks, though each captures a different facet of navigation:

  • Gate navigation requires precision, as the drone must pass through a narrow opening without touching the posts; the arbitrary gates test this at unseen gate placements.
  • The orbit and the figure-eight require dynamic movement absent from the training data, with continuous turning and repeated changes in direction.
  • The mannequin tasks add a semantic search component to the task as its position is not known in advance.

Q1: Missing ablations on command prediction

In Section 5.2, the authors claim that low-dimensional coarse motion is easier to predict under visual distribution shift. It would help to compare against a baseline that predicts the full action chunk with a comparable prediction head to verify this claim.

To verify this, we trained a baseline under the same conditions with the same head architecture to predict the full action chunk. Evaluated on 6,743 frames of real demonstrations, the 16-dimensional command is predicted with R2=0.39R^2 = 0.39, compared to R2=0.23R^2 = 0.23 for the full action chunk (Table R5).

Table R5. Prediction of each head against its own target on the real demonstrations.

HeadTargetR2R^2 (95% interval)
Ours16-dimensional command0.39 (0.36, 0.42)
Full chunk50×450 \times 4 action chunk0.23 (0.20, 0.25)

We also note that the π0\pi_0 baseline predicts the full action chunk on the same backbone, using the flow-matching action expert in place of a head. Relative to the baseline, the 16-dimensional command gets the drone through the gate opening more often, even when σ=0\sigma = 0. In the σ=0\sigma = 0 runs of Figure 6, our policy misses the opening in 44/200 flights (a breakdown we will add to the paper) whereas the π0\pi_0 baseline misses the opening 69 times (Section 5.2). Most of our policy’s remaining failures are touches of a post (Section 5.3), which the learned σ\sigma then corrects, raising success from 87/200 to 146/200, compared to 89/200 for the π0\pi_0 baseline.


Q2: Additional steering baselines

It would strengthen the paper to compare against other steering approaches, such as residual flow steering or learned noise-space steering methods.

We see methods like residual flow steering and learned noise-space steering as complementary to ours, since they address a different question. Learned steering methods such as latent-space reinforcement learning (Wagenmaker et al., 2025) and flow reversal steering (Tang et al., 2026) adapt a trained policy toward an objective through additional optimization or interaction, so a fair comparison would require a reward and a separate training setup. Our method changes what the flow is trained to generate so that the coarse motion of an action chunk becomes an input that some entity (e.g., a learned head, a person, or an agent) can author at test time without reward or further training. The two can be combined, for example by running noise-space reinforcement learning directly in our 16-dimensional command space. For out-of-distribution route following (Section 5.4), our baselines are training-free methods of imposing one command on the π0\pi_0 fine-tune through manipulation of the source noise. We will add this discussion to the related work.

Reviewer_CrJkOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a coarse-to-fine factorization of flow-matching robot policies. It achieves this by encoding a low-dimensional trajectory command into the source noise of an action-generating flow. The method constructs an orthonormal basis representing coarse motion and embeds these motion commands into the initial noise of the generative process. The paper shows that this allows the policy to focus on generating residual motion. Experiments on quadrotor navigation demonstrate improved performance under visual distribution shifts and successful execution of previously unseen, externally specified trajectories in both simulation and real-world settings.

Critical strengths

(1) Intuitive mathematical formulation. The proposed method uses an orthonormal basis to separate coarse motion from residual trajectory generation. The logic and math are straightforward and easy for an unfamiliar reader to understand.

(2) Empirical robustness under a visual distribution shift. The proposed method notably improves success rates over the standard π0 baseline on simulated gate-navigation tasks.

(3) Empirical generalization in one platform. On the dedicated quadrotor experimental platform, the proposed method successfully executes previously unseen trajectories (e.g., orbits, figure-eights) and the ability to use externally-provided commands without additional fine-tuning. Successful demonstrations are shown in both simulation and hardware.

Critical weaknesses

(I actually do not consider the following points to be critical flaws that undermine the proposed method. Instead, I included them here, partially because this is a required section, as an overview of questions that I would appreciate the authors clarifying in the next section.)

(1) Limited theoretical justification for the factorization. Although the proposed orthonormal decomposition looks to be mathematically sound, it doesn't necessarily guarantee that residual motion generalizes to unseen coarse trajectories. Furthermore, the command-invariance property derived in Section 4.1 is relaxed (and may not apply) to the uncertainty-conditioned formulation in Section 4.3.

(2) Limited validation of the uncertainty mechanism. Section 5.3 compares the adaptive uncertainty parameter σ\sigma only against two extreme fixed values 0 and 1.5. What about the in-between values? Furthermore, the relationship between predicted uncertainty and actual command errors is not sufficiently validated.

(3) Limited evaluation settings. The broader applicability to other environment and hyperparameter settings remain relatively unexplored. The experiments focus only on a single quadrotor platform and closely related navigation tasks. The paper also lacks sensitivity analyses for important design choices, e.g., the dimensionality of the command space, and the selected temporal windows.

Questions for authors

(1) Theoretical justification for the coarse-to-fine generalization. Eqs(4)–(7) establish an orthogonal decomposition of the action trajectory into coarse and residual components. However, the orthogonal decomposition alone may not guarantee that the learned residual motion generalizes to unseen coarse commands. Could the authors clarify whether the ability to reuse fine-grained motion across different trajectories is actually theoretically supported or is it mostly empirical observations?

(2) Command invariance under uncertainty conditioning. Eqs(5)–(7) establish that U⊤xt=cU^\top x_t=c and U⊤(z−a)=0U^\top(z-a)=0, i.e., command invariance along the noiseless training interpolant. However, under eq(8), these become U⊤xt=c+tσζ~U^\top x_t=c+t\sigma\tilde{\zeta} and U⊤(zσ−a)=σζ~U^\top(z_\sigma-a)=\sigma\tilde{\zeta}, where ζ~\tilde{\zeta} is the scaled command noise. That means exact invariance no longer holds for nonzero σ\sigma. Could the authors clarify how this affects the theoretical interpretation of their method?

(3) Effectiveness of adaptive uncertainty. Section 5.3 evaluates adaptive σ\sigma against only σ=0\sigma=0 and σ=1.5\sigma=1.5. How does adaptive σ\sigma compare against intermediate fixed values?

(4) Comparison with velocity projection. Section 5.4 and Table 1 show that velocity projection achieves more accurate tracking of externally specified trajectories, while the flawed-sketch experiment referenced in Appendix D demonstrates an advantage of the proposed method when commands are unsafe. Could the authors elaborate on the conditions under which their approach is expected to outperform velocity projection? For instance, how broadly might thsi advantage extend to different types of command error?

(5) Sensitivity to command-space design. How sensitive is the proposed method to the choice of command dimension K=16K=16 and the selected temporal windows? Could the authors provide further intuition for these choices? Maybe discuss whether alternative command-space constructions would be expected to produce similar results?

3 comments
AuthorsOpenReview ↗

Response to Reviewer CrJk (Part 1/2)

We thank Reviewer CrJk for their careful reading and positive assessment. We are glad the reviewer found the mathematical formulation intuitive and easy to follow, and appreciated the robustness under visual shift and the execution of unseen sketched routes in simulation and on hardware. We also appreciate the reviewer noting that the listed weaknesses are questions to clarify rather than critical flaws, and we address each below.

Q1: Theoretical justification of the coarse-to-fine generalization

Eqs(4)–(7) establish an orthogonal decomposition of the action trajectory into coarse and residual components. However, the orthogonal decomposition alone may not guarantee that the learned residual motion generalizes to unseen coarse commands. Could the authors clarify whether the ability to reuse fine-grained motion across different trajectories is actually theoretically supported or is it mostly empirical observations?

The reuse of fine motion across trajectories is mostly an empirical finding, and we will say so more plainly in the revision. Indeed, the guarantees from the construction do not extend to unseen commands. Empirically, we show in Table 1 that sketches written into the source of π0\pi_0 fine-tune, trained without the construction, complete only 24/65, 2/10, and 1/10 routes, while our policy completes 60/65, 10/10, and 10/10.


Q2: Command invariance under uncertainty conditioning

Eqs(5)–(7) establish that U⊤xt=cU^\top x_t = c and U⊤(z−a)=0U^\top(z - a) = 0, i.e., command invariance along the noiseless training interpolant. However, under eq(8), these become U⊤xt=c+tσζ~U^\top x_t = c + t\sigma\tilde{\zeta} and U⊤(zσ−a)=σζ~U^\top(z_\sigma - a) = \sigma\tilde{\zeta}, where ζ~\tilde{\zeta} is the scaled command noise. That means exact invariance no longer holds for nonzero σ\sigma. Could the authors clarify how this affects the theoretical interpretation of their method?

The reviewer is correct that exact invariance holds only when σ=0\sigma = 0, and we will make this explicit in Section 4.3. Under Equation 8, the projection of the interpolant onto the command subspace is U⊤xt=c+tσζ~U^\top x_t = c + t\sigma\tilde{\zeta}, and the regression target in the subspace is σζ~.\sigma\tilde{\zeta}. Because vθv_\theta is conditioned on (xt,t,o,σ)(x_t, t, o, \sigma) and not on ζ~\tilde{\zeta}, the minimizer of the flow-matching loss in the command subspace is the conditional expectation

E[σζ~∣xt,t,o,σ]=(U⊤xt−E[c∣xt,t,o,σ])/t.\mathbb{E}[\sigma \tilde{\zeta} \mid x_t, t, o, \sigma] = (U^\top x_t - \mathbb{E}[c \mid x_t, t, o, \sigma]) / t.

Integrating this field moves the command coordinates from the supplied value toward the policy’s estimate of cc given the observation. The supplied command therefore acts as a noisy measurement of cc with noise scale σ.\sigma. When σ=0\sigma=0, the exact invariance described in Section 4.1 is recovered. Note that the target in the orthogonal complement, (I−UU⊤)(zσ−a)(I - UU^\top)(z_\sigma - a) is unchanged from the noiseless case. Thus, when σ>0\sigma>0, the exact invariance becomes more of a soft constraint, where σ\sigma determines how far the flow may move the supplied command.


Q3: Effectiveness of adaptive uncertainty (with W2)

Section 5.3 evaluates adaptive σ\sigma against only σ=0\sigma = 0 and σ=1.5\sigma = 1.5. How does adaptive σ\sigma compare against intermediate fixed values?

We agree that comparing against only the two extremes leaves the question open. It is possible that the gain comes from predicting the command together with some moderate σ\sigma, rather than from adapting σ\sigma to the head's uncertainty. To address this, we ran fixed values of σ\sigma on the demonstrated tasks under the same protocol as Figure 6.

Table R2. Success out of 200 flights (100 per gate) for fixed and learned σ\sigma, under the same protocol as Figure 6.

σ\sigma0.25.5.751.01.5learned
Success (/200)871281261238151146

The learned σ\sigma outperforms the best fixed value, σ=.25\sigma=.25. We also note that the head's uncertainty and its command error have a Spearman correlation of 0.49 across 720 frames of held-out simulated demonstrations, rising to 0.82 in the final quarter of each flight. We will add this analysis to the paper.

AuthorsOpenReview ↗

Response to Reviewer CrJk (Part 2/2)

Q4: Comparison with velocity projection

Section 5.4 and Table 1 show that velocity projection achieves more accurate tracking of externally specified trajectories, while the flawed-sketch experiment referenced in Appendix D demonstrates an advantage of the proposed method when commands are unsafe. Could the authors elaborate on the conditions under which their approach is expected to outperform velocity projection? For instance, how broadly might this advantage extend to different types of command error?

Our method should outperform velocity projection when (1) the command is correct in its overall route but locally off, for example passing too close to an obstacle, and (2) the policy has learned a better alternative from its training data. The scenario presented in Appendix D demonstrates this as the command passes dangerously close (within 7 cm) to the gate. Thus, following the sketch exactly would be dangerous, hence why velocity projection has zero successes while our policy at σ=0.5\sigma=0.5 succeeds 8/10 times. Thus, the advantage should extend to errors the flow knows how to deal with from observations and has learned to correct but would not extend to out-of-distribution scenarios.

To test how broadly this extends, we evaluated 12 sketches with the policies of Appendix D where four of the sketches are feasible and eight are flawed. Of the flawed sketches, five are in-distribution left-then-center routes, while the other three are out-of-distribution tasks (orbit and figure-eight). We compared our method at σ=0.5\sigma=0.5 against velocity projection (s=0s=0 and s=0.3s=0.3) and SDEdit.

Table R3. Completed routes for the policies of Appendix D, 10 flights per sketch and method. VP: velocity projection.

SketchesOurs (σ = 0.5)VP (s = 0)VP (s = 0.3)SDEdit
Feasible (4)39/4040/4030/4022/40
Flawed, in-distribution (5)25/500/500/501/50
Flawed, out-of-distribution (3)3/300/300/300/30

We find that for in-distribution tasks, our method succeeds in 25/50 flights on flawed sketches, whereas velocity projection fails on all flights. For out-of-distribution tasks, the advantage of our method is limited, as shown by our method succeeding in only 3/30 flights. On feasible sketches, our method performs similarly to velocity projection (39/40 vs. 40/40).


Q5: Sensitivity to command-space design

How sensitive is the proposed method to the choice of command dimension K=16K = 16 and the selected temporal windows? Could the authors provide further intuition for these choices? Maybe discuss whether alternative command-space constructions would be expected to produce similar results?

The value K=16K=16 follows from the choice of windows. The command records cumulative displacement in x,y,z,x,y,z, and yaw over four horizons of 6,12,256, 12, 25 and 5050 steps, so the choice of KK and the temporal windows are chosen together (4×4=164 \times 4 = 16). We chose windows over several horizons because the bases we initially tried, 5-dimensional PCA and reduced-rank regression bases, had trouble with stopping. On the set of real demonstrations, they capture 56% and 29% of the variance of the action chunks at the end of each flight, where the drone comes to rest, compared to 77% for our windows. We find that a 16-dimensional PCA basis captures more of this variance (83%) and reproduces smooth translations more closely but cannot express rare motions in the training data. For example, commanding a 45 degree turn in place is off by about 34 degrees compared with 7 degrees for our windows. Further, because the windows basis represents displacement over a fixed horizon, the command predicted by the head can be interpreted directly, unlike a PCA basis, allowing for external authors to inspect a proposal before steering the policy. We will add these comparisons in a revision. We would also like to note that the choice of command space is an interesting question in its own right, and a systematic study would be useful future work.

Thank you, authors, for the additional clarifications. They are helpful. I will keep my score.

Reviewer_mcvvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a method that encodes coarse trajectory commands into the source noise of pi0-based flow-matching policies, with an uncertainty input regulating command adherence. Commands can be predicted or externally supplied. Quadrotor experiments use 100 demos and report improved simulated success under visual shift. The manuscript provides autonomous hardware performance; additional tests display unseen sketched route following and preliminary coding agent-based control.

Critical strengths

Interesting low-data regime experiments show that simulated success under visual shift improves (from 89/200 with pi0 to 146/200 with the proposed approach).

Critical weaknesses

The advantage under imperfect guidance is insufficiently established. Velocity projection matches or outperforms the proposed method on feasible sketched routes (Table 1), making correction of unreliable commands an important distinction. However, Appendix D evaluates only one flawed sketch, using expanded training data and specified σ settings. The experiment does not establish reliability across sketch errors nor how adherence should be selected when sketch quality is unknown.

Questions for authors

How should σ be selected for externally authored sketches with unknown errors? Could you compare tracking and collision avoidance across multiple feasible and flawed sketches using a predefined selection rule and matched training data? This would clarify whether the advantage over hard and partially relaxed velocity projection generalizes.

Why does the manuscript only use pi0, when newer better models are available? The evals on pi0 alone leave unclear whether the gains address a general limitation of flow-matching policies or limitations specific to that backbone

3 comments
AuthorsOpenReview ↗

Response to Reviewer mcvv (Part 1/2)

We thank Reviewer mcvv for their reading and constructive feedback. We are glad the reviewer appreciated the presentation and premise of the paper, along with the increases in simulated success.

To address W1 and Q1, we extended the study in Appendix D to 12 sketches, of which four are feasible and eight are flawed, with 10 flights per sketch and method. With a single pre-specified σ=0.5\sigma = 0.5 and no per-sketch tuning, our method retains 39/40 success on feasible sketches while achieving 28/80 success on flawed sketches, whereas exact velocity projection achieves 40/40 and 0/80, respectively.

Table R1. Completed routes of 10 flights per sketch and method for the policies of Appendix D. Clearance is the sketch’s distance to the gate point cloud. The drone’s radius is approximately 18cm. Flight failures of our method and velocity projection are due to gate impact. VP: velocity projection.

SketchClearance (cm)Ours (σ = 0.5)VP (s = 0)VP (s = 0.3)SDEditTracking err., cm (ours / VP, s = 0)
Feasible
Left then center, centered239/1010/109/101/108.9 / 1.3
Orbit5310/1010/1010/1010/1010.8 / 0.8
Figure-eight3210/1010/1010/106/107.3 / 1.2
Figure-eight, hand-drawn2510/1010/101/105/107.0 / 1.1
Flawed
Left then center, 15 cm from post148/100/100/100/108.6 / 1.4
Left then center, 7 cm from post (App. D)78/100/100/100/109.3 / 1.5
Left then center, 2 cm from post26/100/100/100/109.6 / 1.5
Left then center, wrong side of post31/100/100/100/1010.5 / 1.6
Left then center, too high122/100/100/101/1013.1 / 1.5
Orbit, tightened30/100/100/100/1012.3 / 0.9
Orbit, tightened further00/100/100/100/1011.7 / 0.9
Figure-eight, tightened163/100/100/100/107.3 / 1.1
Feasible total39/4040/4030/4022/40
Flawed total28/800/800/801/80

These results show two things:

  1. Velocity projection is a reference point for following a feasible sketch rather than a competing method.
  2. When an external author is unsure whether their sketch is correct, a predefined σ=0.5\sigma=0.5 keeps nearly all of the performance on feasible sketches while recovering many of the flawed sketches the policy has learned to handle.

We address each concern and question in more detail below:

W1: Imperfect guidance versus velocity projection

Why use imperfect guidance when velocity projection matches or outperforms the proposed method?

We agree that velocity projection is the best method for enforcing correct sketches exactly. Our method targets the situation where the command is not exactly correct. Table R1 shows this for flawed sketches, and the same property holds for commands predicted by our head.

Consider, for example, the autonomous setting of Section 5.2, where each command is derived from the learned head (and is therefore imperfect), which could be attributed to visual shift from the Gaussian-splat renders. Section 5.3 demonstrates what happens when the policy tries to follow commands strictly. With σ=0\sigma=0, our method adheres to the head’s command as closely as it was trained to. This strict adherence reaches success of 87/200, no better than the π0\pi_0 baseline. Allowing the flow to deviate from commands the head is unsure of raises success to 146/200. Thus, the gain in our main result comes from allowing the trained flow to correct a command it has reason to doubt, rather than leaving it unchanged.

To test whether exact adherence in fact behaves similarly to σ=0\sigma = 0, we ran our policy with velocity projection applied to the head’s predicted commands and found that success fell to 2/200 where the failures were due to missing the opening of the gate or touching a post. Thus, even when σ=0\sigma=0, the flow in our method corrects some of the head’s error, and it is clear that exactly adhering to the head’s command is inadvisable. We will revise Section 5.4 to present velocity projection as a reference point rather than a competing method.

AuthorsOpenReview ↗

Response to Reviewer mcvv (Part 2/2)

Q1: Selecting σ\sigma for external sketches

How should σ\sigma be selected for externally authored sketches with unknown errors? Could you compare tracking and collision avoidance across multiple feasible and flawed sketches using a predefined selection rule and matched training data? This would clarify whether the advantage over hard and partially relaxed velocity projection generalizes.

If an external author is unsure of how accurate their provided sketches are, our method allows for the external author to begin with a predefined default and probe the policy with safer sketches. For instance, a human providing such sketches learns which routes the policy handles well or poorly. On routes the policy handles well, they can increase σ\sigma and yield to the trained flow. On routes the policy handles poorly, they can decrease σ\sigma to keep control. We demonstrated the latter scenario in Table 1, where sketched routes were out-of-distribution and were flown at a fixed σ=0\sigma=0, completing 60/65, 10/10 and 10/10 routes. An author who remains uncertain can stick to conservative sketches, for example by providing sketches with more clearance around obstacles.

To answer the second question, we refer the reviewer to Table R1, where we extended the study performed in Appendix D using σ=0.5\sigma = 0.5, hard and partially relaxed velocity projection (s=0s = 0 and s=0.3s = 0.3), and SDEdit on 12 sketches. For in-distribution tasks (left-then-center routes), our method maintains success on the feasible sketch but dramatically increases success rates on flawed sketches, completing 25/50 whereas velocity projection completed 0/50 and SDEdit completed 1/50. We find the expected result for out-of-distribution tasks, the orbit and the figure-eight, where our method is not expected to improve a flawed sketch and find 3/30 success, though other methods are similarly poor and find no success.


Q2: Choice of backbone

Why does the manuscript only use π0\pi_0, when newer better models are available? The evals on π0\pi_0 alone leave unclear whether the gains address a general limitation of flow-matching policies or limitations specific to that backbone.

We use π0\pi_0 primarily due to compute constraints during hardware rollouts. In preliminary hardware tests, π0.5\pi_{0.5} inference was too slow for closed-loop flight over our ROS 2 and Wi-Fi setup, which we attribute to the highly dynamic nature of aerial flight, where slow decisions can lead to catastrophic failures. However, we take the concern seriously and will add π0.5\pi_{0.5} results in simulation on the demonstrated tasks and protocol to the final version.

Thank you for the additional results. The twelve-sketch evaluation extends the evidence beyond a single flawed sketch and uses a fixed σ without per-sketch tuning. The intermediate-σ comparison also reports higher success for learned uncertainty than for the tested fixed values.

The results support a qualified conclusion: correction succeeds in 25/50 trials on flawed familiar routes and 3/30 on flawed unseen maneuvers, using the expanded training dataset from Appendix D. Velocity projection remains competitive on feasible sketches. Please make these conditions and limitations explicit in the manuscript.

The stated hardware-latency constraint explains the choice of π0, but applicability to other backbones remains untested in the reported results; the proposed π0.5 experiments are still pending.

The expanded evaluation addresses the main evidential gap identified in my review, so I will update my recommendation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.