acceptodds
Under review as a conference paper at ICLR 2027

Dual-Expert Flow Matching for Dynamic Robot Manipulation

Abstract

Vision-Language-Action (VLA) models predict where a robot should move, but not how fast. The velocity profile is neither supervised nor predicted; the low-level controller reconstructs it from position targets. For quasi-static manipulation this is harmless. For dynamic manipulation, in which the robot tosses, flips, slides, or pushes, the velocity at contact decides the outcome, and position tracking distorts it. We show that accurate position does not imply accurate velocity and we establish the need to learn velocity from the demonstrations. We propose VIPER, which adds a velocity expert to a pretrained VLA and denoises velocity jointly with the VLA’s position expert from shared noise to ensure temporal alignment between the noise-free position-velocity actions. VIPER leaves the VLA’s backbone frozen and attaches to any VLA with a flow-matching action expert. We evaluate VIPER on three VLA backbones and six dynamic manipulation tasks in simulation and on a real robot, from tossing and flipping to catching and handover. VIPER raises mean success from 32% to 51%, and improves backbones that already reason about temporal context. Our project website is available at https://viper-robot.github.io/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Dynamic Robot Manipulation, Velocity Prediction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Yxg5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies an actionable question for dynamic robot manipulation: should a VLA explicitly predict velocity as well as position? VIPER adds a velocity expert to a position-based policy and trains the two experts jointly with shared flow noise and joint position–velocity conditioning. Their outputs define a continuous trajectory through Hermite interpolation. Experiments cover three policy configurations and six tasks, including three physical robot tasks. The results support a useful practical lesson: explicitly learning velocity can improve tasks in which the timing and speed of contact matter.

Critical strengths
  1. The problem is well chosen. Predicting a plausible sequence of positions is not necessarily enough for tossing, flipping, or interception. The paper connects the action representation to these concrete execution requirements, and its proposed modification is straightforward enough to be useful to people building dynamic manipulation systems.
  2. The empirical improvements are substantial and fairly consistent.
Critical weaknesses
  1. The spline comparison supports a narrower conclusion than the paper suggests. The paper clearly describes Spline-FF as fitting a cubic B-spline through predicted position targets and differentiating the curve. Hechtl et al.'s continuous variant instead trains the policy to predict B-spline control points, using control points fitted to demonstrations as training targets. Both use B-splines, but they learn different action representations. The reported comparison therefore establishes an advantage over post-hoc spline interpolation, not over the cited continuous-action method in its original form. The baseline attribution and conclusions should reflect this distinction.

  2. The specific role of shared noise remains unclear. As acknowledged in Appendix C, the alignment argument assumes that both experts recover the exact shape of their profiles, the only residual error is a single scalar temporal phase shared across all H timesteps and all joints, and the denoisers are population-optimal for the squared-error flow-matching objective. The experiments support the coupled design as a whole, but do not separate shared noise from joint position–velocity conditioning, leaving the contribution of shared noise itself to alignment unresolved.

  3. The generality of the learned velocity behavior remains unclear. An interesting question is whether the velocity expert learns motion behavior that can adapt to unfamiliar objects and substantially different trajectories. The reported results establish its usefulness in the evaluated task settings, but leave readers uncertain about how far this behavior extends beyond the demonstrated motion patterns. Coupling velocity with position is compatible with generalization. As a result, readers gain a useful practical lesson, but limited insight into whether the learned velocity behavior is broadly transferable.

  4. The consistently smaller gains on physical tasks remain insufficiently explained. In Table 1, all three backbones show smaller improvements on physical tasks than in simulation. Although the task sets differ and Section 5.3 offers an explanation for RTC, the broader pattern remains unresolved: does it reflect task demands or difficulty, or limitations of the proposed approach during physical execution? The paper provides no evidence from additional tasks or datasets to support a task-based explanation.

Questions for authors

See weakness and one more question: How much inference latency does the velocity expert add in the evaluated configurations, and does it change the policy replanning rate?

5 comments
AuthorsOpenReview ↗

Rebuttal to Reviewer Yxg5 [part 1]

We thank the reviewer for calling the problem well chosen and the gains substantial and fairly consistent, and for naming Hechtl et al.'s continuous variant and the shared-noise split so clearly.

Summary.

  1. (W1) We added Control-Point FF, a learned control-point baseline, on the full Table 1 grid. VIPER is higher than Control-Point FF in 17 of 18 cells and ties one.
  2. (W2) We isolated shared noise from joint conditioning. Both contribute: joint conditioning raises success, and shared noise raises it further and lowers the consistency error at every denoising step.
  3. (W3) Cube Slide at held-out target distances. VIPER keeps its Table 1 success rate on unseen distances across the training range.
  4. (W4) The smaller physical Δ\Delta appears for every source of velocity, and the physical baselines start higher. VIPER is the only method with a positive physical Δ\Delta on all three backbones.
  5. (Q1) +22.4 ms per action chunk on π0.5\pi_{0.5}; replanning rate unchanged.

W1.

"not over the cited continuous-action method in its original form"

We incorporate the suggestion and report new results on Control-Point FF: the policy predicts cubic B-spline control points, trained on control points fitted to the demonstrations, and the low-level controller receives the derivative of that spline, following the time-continuous action space of Hechtl et al. Recipe: 28 control points per action chunk, 32 knot vector, flow-matching loss on the control points. The frozen backbone, the demonstrations, and the low-level controller are the same as in Table 1. Mean success (%) over the three tasks on each side.

Methodπ0.5\pi_{0.5} Phys  π0.5\pi_{0.5} Sim  RTC Phys  RTC Sim  TEMPO Phys  TEMPO Sim  
Position-Only30.016.342.017.758.025.0
Spline-FF (post-hoc)31.327.340.028.056.031.0
[new] Control-Point FF28.022.732.720.351.324.3
VIPER47.351.744.752.068.044.7
  • VIPER is higher than Control-Point FF on all six averages and on 17 of the 18 backbone–task combinations. The one tie is RTC Ball Push at 36. Per-task numbers are in the revised Table 1.
  • Control-Point FF does not exceed post-hoc Spline-FF on any of the six averages.

Table 1 now covers two spline-based sources of velocity: Spline-FF fits the spline post hoc through the predicted position targets, and Control-Point FF learns the spline action space. In both, the controller receives the derivative of a position spline. VIPER is the only row whose velocity is supervised on the demonstrated joint velocities, and it is the highest on every average.

We have incorporated these changes into the revised manuscript in Sec. 2, Sec. 5.1, and Table 1 (blue).


W2.

"do not separate shared noise from joint position–velocity conditioning"

We incorporate the suggestion and report a new variant that keeps joint conditioning and co-fine-tuning as in VIPER and draws independent noise for the two experts, at training and at inference. Table 3 setting (π0.5(\pi_{0.5}, simulation, 100 rollouts, success %).

VariantBottle TossBread FlipCube SlideAvg
Independent-Experts (Table 3)28384737.7
[new] Joint conditioning, independent noise52415248.3
VIPER (joint conditioning + shared noise)57425651.7
  • Joint conditioning raises average success from 37.7 to 48.3.
  • Shared noise raises it further to 51.7 (+5 Bottle Toss, +1 Bread Flip, +4 Cube Slide).

The consistency error shows what shared noise contributes to alignment. Position–velocity consistency error EconsE_{\mathrm{cons}} (App. D.3) at intermediate denoising steps, Cube Slide, 100 rollouts:

VariantStep 3 ↓\downarrowStep 5 ↓\downarrowStep 10 ↓\downarrow
[new] Joint conditioning, independent noise0.960.810.71
[new] VIPER (shared noise)0.690.640.60
  • VIPER is lower at every step, and the gap is largest early (0.27 at step 3, 0.11 at step 10).
  • This pattern is consistent with App. C.2. With independent noise the difference of the two interpolants is (1−t)(Q˙−Q)+t(ϵv−ϵp)(1-t)(\dot{Q}-Q) + t(\epsilon_v-\epsilon_p), so the position–velocity offset is corrupted most at large tt, which is the early steps. Shared noise cancels that term.

We have incorporated both tables into the revised manuscript Appendix D.6 (blue). The consistency table measures directly the alignment effect that Appendix C motivates.

AuthorsOpenReview ↗

Rebuttal to Reviewer Yxg5 [part 2]

W3.

"substantially different trajectories"

We report Cube Slide at held-out target distances, π0.5\pi_{0.5}, 20 rollouts per band.

Goal bandDistance (m)Success [new]
Inside the training range0.43–0.7360% (12/20)
Outside, shorter0.28–0.3715% (3/20)
Outside, longer0.76–0.8130% (6/20)
  • Inside the training range, 60% is in line with Table 1 Cube Slide (56% on 100 rollouts), on target distances that are not in the demonstrations.
  • Outside the range, success is 15% and 30%, as expected for a policy trained on demonstrations between 0.43 and 0.73 m.
  • Table 1 already varies the goal within each physical task: the basket position (Ball Push), the walking path (Bottle Handover), and the release time (Drop Catch) change across trials (App. D.1). Ball Push on π0.5\pi_{0.5} goes 28% →\to 52% under that variation.

We have added the distance-band discussion to Section 6 (blue) of the revised manuscript.

"unfamiliar objects"

Object recognition is the role of the frozen backbone, which VIPER leaves unchanged: the velocity expert conditions on the same backbone output cc as the position expert. Our evaluation targets motion variation, and we have listed unfamiliar-object evaluation as a future direction in Section 6 (blue) of the revised manuscript.


W4.

"task demands or difficulty, or limitations of the proposed approach during physical execution"

Table 1 supports the task-based reading in two ways.

First, the smaller physical Δ\Delta is not VIPER-specific.

Δ\Delta success vs Position-Only:

Methodπ0.5\pi_{0.5} Phys  RTC Phys  TEMPO Phys π0.5\pi_{0.5} Sim RTC Sim  TEMPO Sim 
Spline-FF (post-hoc)+1.3−2.0−2.0+11.0+10.3+6.0
[new] Control-Point FF−2.0−9.3−6.7+6.3+2.7−0.7
VIPER+17.3+2.7+10.0+35.3+34.3+19.7

Δ\Delta is the mean of the per-task differences, as in Table 1.

Every source of velocity gains less on the physical tasks, which points to the tasks and not to VIPER. VIPER is the only one that stays positive on all three backbones; the two spline-based sources range from −9.3 to +1.3.

Second, Position-Only starts higher on the physical tasks (30.0 / 42.0 / 58.0) than in simulation (16.3 / 17.7 / 25.0), so less failure is left to remove.

Share of Position-Only failures that VIPER removes (%):

Settingπ0.5\pi_{0.5}RTCTEMPO
Physical25524
Simulation424226
  • On TEMPO, VIPER removes the same share of failures on the physical tasks as in simulation (24 vs 26).
  • On π0.5\pi_{0.5}, VIPER removes a quarter of the physical failures.
  • RTC is the case Sec. 5.3 discusses: RTC blends each new action chunk into the committed actions, which smooths the transitions between chunks, where the short velocity peak can fall.

We have added this reading to Sec. 5.3 (blue) of the revised manuscript.


Q1.

"How much inference latency does the velocity expert add in the evaluated configurations"

We report wall-clock time on π0.5\pi_{0.5}, RTC and TEMPO : A100, 10 denoising steps, per action chunk (Table 9). The backbone runs once and its output cc is shared, so the extra time is the velocity expert's denoising steps.

ConfigurationPosition-Only (ms) ↓\downarrowVIPER (ms) ↓\downarrowExtra (ms) ↓\downarrow
[new] π0.5\pi_{0.5}79.2101.622.4
[new] RTCRTC116.0142.226.2
[new] TEMPOTEMPO93.5112.919.4

We have added this to App. D.8 (blue) of the revised manuscript.

"does it change the policy replanning rate?"

No. Every backbone runs VIPER, Spline-FF, and Control-Point FF at the replanning rate of its Position-Only row in Table 1.


Revised PDF (blue): Sec. 2, Sec. 5.1, and Table 1 (Spline-FF is post-hoc interpolation; Control-Point FF is the learned control-point baseline), Sec. 5.3 (physical Δ\Delta), Sec. 6 (Cube Slide distances), App. D.6 (shared-noise split), App. D.8 (Inference latency). We are happy to elaborate on any implementation-level detail in the discussion phase.

Rebuttal Summary

We believe we have objectively answered each concern in the review, with new results for every one. All of them strengthen the findings of the paper.

Is our response satisfactory? If not, we are happy to run more experiments in the remaining 1.5 days.

  1. Learned control-point baseline (W1). We added the new baseline across the full Table 1 grid, covering 3 simulation and 3 real-robot tasks across 3 backbones. VIPER outperforms this baseline in 17 of 18 comparisons and ties in one. We also revised Sections 2 and 5.1 and Table 1 to clarify the distinction between baselines.

  2. Shared noise vs. joint conditioning (W2). New ablation: 37.7% →\to 48.3% with joint conditioning, →\to 51.7% with shared noise.

  3. Alignment during denoising (W2). Additional experiments: VIPER has a lower consistency error at every step (0.64 vs. 0.81 at step 5) when denoised from shared noise.

  4. Unseen target distances (W3). Additional experiments: VIPER achieves 60% success on Cube Slide at distances not in the demonstrations, in line with 56% in Table 1.

  5. Physical vs. simulation gains (W4). VIPER is the only method with a positive physical gain on all three backbones. On TEMPO, it removes the same share of failures on physical tasks as in simulation (24% vs. 26%).

  6. Latency (Q1). An additional 22.67 ms per action chunk on average across 3 different backbones; the replanning rate remains unchanged.

We thank the reviewer for the detailed review, which has improved both the framing and the evidence. Given these new results, we sincerely hope the reviewer will reconsider the overall assessment.

Reviewer_6qtROpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses dynamic robot manipulation—tasks such as tossing, flipping, sliding, and catching where contact velocity and momentum dictate task success. The authors reveal a fundamental limitation of current Vision-Language-Action (VLA) models: existing VLAs predict only chunked joint-position targets, leaving low-level controllers to reconstruct velocities by either tracking targets with zero desired velocity (which introduces periodic velocity ripples and dampens momentum) or differentiating position splines (which consistently undershoots peak velocities). The authors establish theoretically and empirically that accurate position prediction does not imply accurate contact velocity. To resolve this, they introduce VIPER (Velocity-Informed Policy Enhancement): a dual-expert flow-matching framework that appends a dedicated velocity expert to a pretrained, frozen VLA backbone. Crucially, the position and velocity experts are coupled during training and inference by denoising from the exact same shared Gaussian noise vector, ensuring exact cancellation of noise in their difference and maintaining strict temporal phase alignment. Commanded waypoints are interpolated via Cubic Hermite splines at 250 Hz for closed-loop PD tracking. VIPER is evaluated across three modern VLA backbones (general-purpose ω0.5, asynchronous RTC, and temporally conditioned TEMPO) on six dynamic tasks: three on a physical I2RT YAM robot arm (Ball Push, Bottle Handover, Drop Catch; 50 trials per condition) and three in RoboTwin simulation (Bottle Toss, Bread Flip, Cube Slide; 100 trials per condition). VIPER lifts the average success rate from 32% to 51%, with improvements up to +48% on momentum-critical tasks, and ablations confirm that both the dual-expert split and noise-coupling are indispensable.

Critical strengths
  1. Crisp Problem Formulation and Control-Theoretic Grounding: The paper pinpoints a critical, pervasive flaw in contemporary VLA research: the disconnect between quasi-static position chunk tracking and dynamic contact physics. The analytical derivation of tracking dynamics in Section 3.1 (Eq. 1-2) clearly proves why zero-velocity PD targets inevitably produce ripple and damp momentum. The proposed solution—coupling dual flow-matching experts through identical noise—is mathematically elegant, ensuring strict phase consistency across position and velocity without requiring heavy cross-attention architectures.

  2. Rigorous Physical Hardware Validation Across Multiple Modern Backbones: The empirical validation is exemplary. Rather than tuning on a single bespoke architecture, VIPER is benchmarked across three distinct, state-of-the-art VLA backbones (ω0.5, RTC, and TEMPO) across both simulation and real-world experiments on an I2RT YAM arm with substantial sample sizes (50 real rollouts per method/task). The ablations in Section 5.4 cleanly demonstrate that a single combined expert fails completely and that uncoupled independent experts suffer from severe acceleration spikes (>500 rad/s²), convincingly justifying the design.

Critical weaknesses
  1. Complete Omission of Validation on Standard Quasi-Static Manipulation Benchmarks: The paper tests VIPER exclusively on six dynamic manipulation tasks. In Section 6, the authors assert that "since the velocity expert can output zero, we expect VIPER to reduce to the base policy on quasi-static tasks, which is easy to verify." However, this assumption is never tested experimentally. In high-precision, quasi-static tasks (such as peg-in-hole insertion or delicate object stacking in LIBERO/CALVIN), active velocity feedforward could potentially induce overshoot, limit-cycle oscillations, or steady-state tracking error. Validating that VIPER does not harm quasi-static precision is critical for a general-purpose VLA enhancement.

  2. Reliance on Clean Demonstration Velocity Logs and Dataset Incompatibility: VIPER requires demonstrations to contain synchronized, high-quality joint-velocity logs. However, dominant large-scale open-source robot manipulation datasets (e.g., Open X-Embodiment, BridgeData, DROID) log only position waypoints or low-frequency end-effector poses. As demonstrated in Section 3.2, numerical differentiation of position data introduces high-frequency noise and fails to recover dynamic peaks. Consequently, VIPER cannot currently leverage the vast majority of existing robotics pretraining data without specialized velocity filtering or state-estimation pipelines.

Questions for authors

My questions can be found in the above weakness part.

2 comments

We thank the reviewer for calling the empirical validation "exemplary", for finding the shared-noise coupling "mathematically elegant", and for the close reading of the Sec. 3.1 analysis (Eqs. 1–2) and the Sec. 5.4 ablations. Both suggestions made the paper better, and we are glad to report the quasi-static experiment below.

Summary.

  1. (W1) We ran the quasi-static check the review asks for, on a real robot and in simulation. VIPER is at or above Position-Only on both tasks.
  2. (W2) VIPER keeps the full benefit of position-only pretraining. It needs velocity in only 30 to 90 task demonstrations.

"Validating that VIPER does not harm quasi-static precision is critical"

This is a great suggestion, and we were glad to run it. Sec. 6 called this check "easy to verify", and we can now show it. We evaluated two quasi-static tasks on π0.5\pi_{0.5}: Cube Stacking on the YAM arm and Mug Hanging in RoboTwin. Success requires the cube to remain stacked in Cube Stacking and the mug to remain hanging on the rack in Mug Hanging. The table reports success rates, where (n) denotes the number of evaluation trials per method.

MethodCube Stacking (real, n=50)Mug Hanging (sim, n=100)
[new] Position-Only (π0.5\pi_{0.5})68%26%
[new] VIPER72%30%
  • VIPER does no harm on either task. Success does not drop. It is 4 points higher on both, within one standard error, so we claim parity and not a gain.
  • VIPER learns when to be slow. The velocity expert is supervised on the demonstrated velocity, so where the demonstration slows down or holds still, it predicts a slow or zero velocity. App. Fig. 6 shows this on Bread Flip, where VIPER follows the stationary phases of the demonstration. The low-level controller therefore receives a near-zero desired velocity exactly where precision matters.
  • Placement stays stable. The rollouts show no added overshoot or oscillation at the put-down. Videos are at https://viper-robot.github.io/#comparison-quasi-static.

Both tasks use the same robot, simulator, and low-level controller as Table 1.


"VIPER cannot currently leverage the vast majority of existing robotics pretraining data"

VIPER keeps the full benefit of position-only pretraining. It needs velocity only in the task demonstrations.

  • VIPER starts from the pretrained VLA. The backbone and the position expert come from a VLA pretrained on large position-only collections such as Open X-Embodiment. VIPER keeps that backbone frozen (Sec. 4.1), and the velocity expert is initialized from the pretrained π₀.₅ checkpoint (App. A).
  • The velocity-labeled data is small. VIPER uses 30 to 90 demonstrations per task (App. A).
  • Velocity is easy to log from here on. Joint velocity is standard proprioception. The YAM arm and RoboTwin both report it, and we will release all our demonstrations with joint velocities (Sec. 6).
  • Position-only logs are the next step, not a dead end. An interpolating cubic B-spline differentiated through recorded positions stays 0.8 to 1.8 rad/s from the logged velocity (Table 6), so we do not use that fit as labels for the velocity expert. Sec. 6 gives the path for dumps that only have position: treat velocity as a latent that is supervised where it is logged and inferred elsewhere.

Revised PDF (blue): Sec. 5 adds the quasi-static table, and Sec. 4.1 and Sec. 6 state which stage needs velocity. We are grateful for both suggestions and are happy to discuss any of this further.

Rebuttal Summary

We thank the reviewer for recognizing that our paper “pinpoints a critical, pervasive flaw in contemporary VLA research,” describing our solution as “mathematically elegant,” and calling our empirical validation “exemplary.”

Below, we summarize the reviewer’s questions and our responses. We believe we have addressed 2/2 questions objectively through new results and clarification of the training setup.

Quasi-static performance (W1). Additional experiments: VIPER achieves 72% vs. 68% on real-world Cube Stacking (50 trials) and 30% vs. 26% on simulated Mug Hanging (100 trials). Neither task shows an observed decrease in success; rollout videos show no added overshoot or oscillation; the velocity profiles in Fig. 6 (App. D.4) of the original submission also illustrate that VIPER learns when to move slowly.

Leveraging existing robotics pretraining data (W2). VIPER builds on an existing VLA backbone pretrained on large-scale position-only data; velocity supervision is needed only during task adaptation, using 30–90 demonstrations per task; collecting these labels simply requires logging joint velocities alongside positions.

These results and clarifications are included in Sec. 5, Sec. 4.1, and Sec. 6 of the revised PDF (blue). We thank the reviewer for the detailed feedback, which has strengthened both the paper’s framing and its supporting evidence.

Reviewer_bDSKOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a dual-expert framework that explicitly predicts both position and velocity for dynamic robot manipulation. Building upon existing flow-matching VLA models, such as π0.5, the proposed method introduces an additional coupled velocity expert alongside the original position expert. During training, the two experts share the same flow-matching timestep and Gaussian noise, enabling coupled generation of temporally aligned position and velocity trajectories.

Experiments on 3 real-world robotic tasks and 3 simulation tasks demonstrate that VIPER consistently improves task success rates over position-only baselines and spline-based velocity estimation methods. The improvements are also observed when VIPER is integrated with RTC and TEMPO. Theoretical analyses are also implemented to illustrate the limitations of recovering velocity through position differentiation and how shared-noise coupling helps preserve temporal alignment between predicted position and velocity trajectories.

Critical strengths

The paper is clearly written, with a well-motivated problem formulation and thorough theoretical and experimental analysis. The authors provide a clear explanation of the limitations of existing position-based control approaches for dynamic manipulation. In particular, they analyze how position-only control can introduce repeated acceleration and deceleration when tracking consecutive position targets, resulting in velocity oscillations that are undesirable for highly dynamic tasks. They further demonstrate that differentiating interpolated position trajectories does not necessarily recover the desired velocity profile. These limitations are well illustrated in Figures 1 and 4, making the motivation for explicitly predicting velocity both intuitive and convincing.

The proposed coupled position–velocity generation framework is technically well-motivated and addresses an important challenge in dynamic manipulation. While introducing a separate velocity expert enables the policy to explicitly model velocity, it also raises the issue of temporal misalignment between independently generated position and velocity trajectories. The authors address this problem by using the same flow-matching timestep and Gaussian noise for both experts, while conditioning each expert on their joint noisy states. This provides a principled mechanism for promoting temporal alignment between the predicted position and velocity trajectories. The theoretical analysis and ablation studies further support the effectiveness of this design, particularly the comparison between joint, independent, and coupled experts.

The experimental evaluation is comprehensive and provides convincing evidence of the effectiveness and compatibility of the proposed method. The comparisons include relevant baselines, particularly spline-based velocity feedforward, which directly addresses the same problem of providing velocity references during execution. Furthermore, the integration with RTC and TEMPO demonstrates that the proposed method is complementary to existing approaches for improving real-time execution and temporal perception.

Critical weaknesses

One interesting question is how the proposed method would be applied to long-horizon dynamic manipulation tasks that require continuous velocity-sensitive interactions, such as repeated ball striking or balancing a ping-pong ball on a paddle, where precise velocity control and continuous closed-loop adjustments are required across multiple action chunks. As the authors also point out, RTC's action chunk blending may attenuate the velocity peaks that are critical for dynamic manipulation. A discussion of this issue, or additional experiments involving continuous dynamic interactions, would help clarify the potential of VIPER for long-horizon dynamic manipulation.

Questions for authors

How do the authors envision extending VIPER to long-horizon dynamic manipulation tasks involving continuous velocity-sensitive interactions, such as repeated ball striking? While combining VIPER with RTC or TEMPO seems promising, the paper also notes that RTC's action chunk blending may attenuate critical velocity peaks. Could the authors discuss how this might affect velocity consistency across consecutive action chunks in long-horizon tasks?

2 comments

We thank the reviewer for finding the motivation for predicting velocity "intuitive and convincing", for calling the shared-noise coupling "a principled mechanism for promoting temporal alignment", and for reading the RTC and TEMPO results as showing that VIPER is "complementary to existing approaches". The question on velocity consistency across action chunks sharpens that last point, and we answer it with a real-robot analysis below.

"velocity consistency across consecutive action chunks" … "RTC's action chunk blending may attenuate critical velocity peaks"

This is a great question to press, because it tests how VIPER composes with RTC. We went back to real-robot Ball Push logs of VIPER (on π0.5) with and without RTC and plotted the commanded joint-5 trajectory of the right arm (mean ± std over 50 episodes each, temporally aligned at episode start). Left: position. Right: velocity. Blue: VIPER without RTC. Orange: VIPER with RTC. The figure covers the first chunk of the run without RTC (0–0.8 s, before the arm hits the ball), during which RTC switches chunks at 0.06, 0.38, and 0.70 s (dotted lines). At the boundary between RTC chunks A and B, the position stays continuous, but the velocity falls from about 0.35 to 0.2 rad/s inside the blend window (shaded) and has recovered only to about 0.5 rad/s by the end of the window. Over the same interval without RTC, the velocity rises steadily from approximately 0.45 to 0.7 rad/s.

The figure is in App. D.7, for convenience, the same figure is available in an anonymous online viewer.

The position stays continuous. The velocity does not. That is the same lesson as Fig. 1, now at a chunk boundary: RTC inpaints the position and does not constrain the velocity VIPER already predicted.

This also helps explain the RTC block of Table 1. VIPER is the strongest method there (44.7 physical, 52.0 simulation). The gain is +34.3 in simulation, where the peak sits inside a chunk, and +2.7 on the physical robot, where a peak can sit in the blend window (Sec. 5.3).

"continuous closed-loop adjustments are required across multiple action chunks"

The other half of the question is whether VIPER can keep adjusting across chunks, not just fire one impulse.

  • VIPER is run closed-loop. Each new chunk is generated from the latest observation. Within that chunk the two experts are denoised together from shared noise (Sec. 4.2), so the new position and velocity describe the same moment.
  • Bottle Handover and Drop Catch already do this while a person or a ball is moving (8 and 39 chunks per episode). On TEMPO, the backbone built for these tasks, VIPER raises them 74% → 86% and 66% → 86% (Table 1). That is velocity-sensitive closed-loop control across chunks, and the closest setting in our suite to the reviewer's examples. Repeated striking would use this same loop; what remains is keeping a blend window from cutting the hit short, which the overlay above isolates.
  • The natural extension, and the one the review points to, is for RTC to inpaint the velocity VIPER already predicts, not only the position. We would freeze the committed prefix of both experts and denoise the rest with the same shared noise as in the paper. A peak that lands on a chunk boundary should then survive the blend.

Revised PDF (blue): Sec. 5.3 discusses the Ball Push overlay (figure in App. D.7), and Sec. 6 gives the inpainting recipe.

Rebuttal Summary

We thank the reviewer for recognizing that VIPER “addresses an important challenge in dynamic manipulation,” provides “a principled mechanism for promoting temporal alignment,” and is supported by “thorough theoretical and experimental analysis.”

Below, we summarize the reviewer’s suggestion and our response, which has further strengthened our work. We believe the additional analysis addresses all of the reviewer’s questions:

Velocity consistency across chunks. Our new analysis of Ball Push logs from VIPER w/ and w/o RTC shows how velocity attenuation occurs during RTC despite continuous position commands. Sec. 6 discusses joint position–velocity inpainting as a future extension.

Long-horizon dynamic manipulation. Bottle Handover and Drop Catch require repeated closed-loop adjustments in response to new observations. VIPER improves the TEMPO backbone’s success rate on both tasks.

The additions are included in the revised PDF (blue). We appreciate the reviewer’s feedback, which has helped us clarify the paper’s contributions and strengthen its experimental support.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.