- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
- This paper proposes FlowShift, a risk-aware test-time scaling framework for flow-based VLAs, while keeping the pretrained VLA backbone frozen.
- The method has three main components: (1) a test-time adaptive task-progress verifier that uses self-supervised state prediction to adapt its latent prompts, (2) verifier-guided stochastic flow sampling that injects noise and uses reward gradients to guide action generation, and (3) a risk-aware router that selectively activates additional sampling for high-risk action chunks.
- The authors evaluate FlowShift on LIBERO-Plus, RoboTwin 2.0, and real-world bimanual manipulation tasks.
Critical strengths
-
Interesting integration of verification and action generation. I like the idea of using verifier feedback not only for selecting completed actions but also for guiding intermediate flow states. The risk-aware router is also a useful addition, since applying test-time scaling to every action chunk may introduce unnecessary distribution shifts.
-
Solid experimental results, especially under challenging settings. The improvements on RoboTwin Hardand real-world manipulation are encouraging. I also appreciate the ablations on verifier adaptation, gradient guidance, and router design, which help demonstrate the contributions of the individual components.
Critical weaknesses
-
Missing comparisons with existing test-time scaling approaches. Although RoboMonkey, MG-Select, TapSampling, and VLS are discussed, the main experimental tables mostly compare against pretrained VLA policies rather than existing TTS methods. I think adding at least one matched TTS baseline under the same backbone and inference budget would strengthen the paper. QGF is another relevant approach, particularly for comparing gradient-guided flow generation.
-
Several components have been explored individually in prior work, including stochastic flow sampling, reward-gradient guidance, and test-time verifier adaptation. The paper would benefit from a clearer discussion of what is fundamentally new beyond combining these techniques. I'm also not fully convinced by the verifier supervision: assuming linear progress across successful demonstrations may not reflect actual task progress, and reversing action sequences seems to produce relatively easy negatives rather than challenging failure modes.
Questions for authors
- Could the authors add a matched baseline against at least one existing TTS approach, such as RoboMonkey, TapSampling, VLS, or QGF, using the same VLA backbone and a comparable inference budget? This would help clarify the effectiveness of the proposed method.
- Could FlowShift complement post-hoc sampling and verification methods such as RoboMonkey or TapSampling? It would be interesting to see whether combining these approaches provides additional gains over either method alone.
- What motivates the assumption of linear task progress along successful demonstrations? Also, why use reversed action sequences as negative examples instead of harder negatives from failed rollouts or alternative candidate actions? An ablation comparing different negative-sampling strategies would help establish the verifier's robustness.
- Which types of tasks benefit most from FlowShift, and what specific failure modes does it address? For example, is the improvement mainly from recovering from inaccurate grasping/placement, handling visual distribution shifts, or correcting long-horizon execution errors? A more detailed breakdown would help explain when verifier-guided flow refinement is useful.
4 comments
Q2: Complementarity with Post-hoc Sampling and Verification
R3: To investigate the complementarity between FlowShift and post-hoc sampling methods, we conducted an additional experiment applying TapSampling on top of π₀.₅ + FlowShift, using TapSampling's default configuration on LIBERO-Plus Goal.
Method SR ↑ (%) Latency ↓ (ms) π₀.₅ + FlowShift 80.1 252.99 π₀.₅ + FlowShift + TapSampling 79.4 880.91 The combined method achieves 79.4% success, but does not outperform FlowShift alone (80.1%), while substantially increasing inference latency. This overhead arises from stacking two sampling and verification procedures. One possible explanation for the lack of additional gains is that TapSampling's fixed verifier may not fully accommodate the action distribution refined by FlowShift's adaptive guidance.
These results suggest that naively combining the two methods does not guarantee complementary benefits. Developing a more efficient integration could be a promising direction for future work.
Q4: Task-Specific Benefits and Failure Modes Addressed by FlowShift
R4: As shown in Table 1 in the main paper, FlowShift achieves the largest improvements on LIBERO-Plus under Camera (+2.5%), Layout (+1.6%), and Background (+1.4%) perturbations, indicating that it is particularly effective under visual and spatial distribution shifts. The detailed improvements across perturbation categories are summarized below:
Perturbation π₀.₅ SR (%) + FlowShift SR (%) Gain Camera 70.9 73.4 +2.5 Layout 85.7 87.3 +1.6 Background 94.3 95.7 +1.4 Robot 75.2 76.1 +0.9 Noise 86.5 87.0 +0.5 Language 85.3 85.4 +0.1 Light 96.5 96.4 -0.1 Real-World Failure Modes: Our additional qualitative demonstrations show that FlowShift can identify risky action predictions involving inaccurate action poses and spatial misalignment, and refine the corresponding action trajectories. Furthermore, FlowShift improves real-world long-horizon task success from 27.5% to 40.0%, suggesting that risk-aware action refinement helps mitigate error accumulation across dependent subgoals. Since aggregate success rates cannot fully capture these intermediate interventions and recovery behaviors, we provide anonymous demonstration videos (link) illustrating risk-aware action correction and long-horizon execution in real-world settings.
Overall, FlowShift is particularly beneficial for visual and spatial distribution shifts, inaccurate action execution, and long-horizon manipulation tasks involving accumulated execution errors.
Thank you for the detailed response and for running the additional experiments! I think the overall framework is interesting and the additional experiments strengthen the paper. I will raise my score.
Thank you very much for your thoughtful review and constructive comments. Your valuable feedback has been very helpful in improving the quality and clarity of our paper. We are delighted that our responses and additional experiments have addressed your concerns. We sincerely appreciate your positive reassessment of our work and your willingness to raise the score.
Thank you for the thoughtful and constructive review. We have addressed your concerns through additional experiments and clarifications as follows. We would greatly appreciate it if you could consider giving us a higher rating.
W1 & Q1: Comparison with Existing Test-Time Scaling Methods
W2 & Q3: Methodological Novelty and Verifier Supervision