acceptodds
Under review as a conference paper at ICLR 2027

FlowShift: Risk-Aware Test-Time Scaling for Vision-Language-Action Models with Verifier-Driven Guidance and Selection

Abstract

Vision-Language-Action (VLA) models often degrade under test-time distribution shifts, when training and deployment distributions diverge. Existing test-time scaling methods mainly select among completed action candidates, leaving the generation process constrained by pretrained flow dynamics and the verifier fixed under deployment changes. To address these limitations, we propose **FlowShift**, a plug-and-play, risk-aware test-time scaling framework that adaptively shifts the action distribution while keeping the VLA backbone frozen. (1) FlowShift first introduces a **Test-Time Adaptive Task-Progress Verifier**, which adapts latent prompts through self-supervised state prediction to remain responsive to deployment changes. The adapted verifier provides two complementary signals for subsequent action generation: reward gradients for guidance and scalar rewards for evaluation. (2) Building on these signals, **Verifier-Guided Stochastic Flow Sampling** converts selected ODE updates into verifier-guided SDE transitions, using reward gradients to steer intermediate states toward higher-reward regions and scalar rewards to select promising particles. This process progressively shifts the action distribution toward a reward-aligned target. (3) To prevent excessive distribution shifts, FlowShift further introduces a **Risk-Aware Router** that models multi-timestep flow features and trajectory history to estimate execution risk. It activates stochastic flow sampling for high-risk action chunks while retaining the original ODE path for low-risk ones, thereby controlling distribution shift. FlowShift improves from 83.9% to 85.0% on LIBERO-Plus, from 63.9% to 68.9% and 39.0% to 46.2% on RoboTwin 2.0 under the *Easy* and *Hard* settings, respectively, and from 56.7% to 69.2% in real-world manipulation.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotic Manipulation

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_NeiDOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • This paper proposes FlowShift, a risk-aware test-time scaling framework for flow-based VLAs, while keeping the pretrained VLA backbone frozen.
  • The method has three main components: (1) a test-time adaptive task-progress verifier that uses self-supervised state prediction to adapt its latent prompts, (2) verifier-guided stochastic flow sampling that injects noise and uses reward gradients to guide action generation, and (3) a risk-aware router that selectively activates additional sampling for high-risk action chunks.
  • The authors evaluate FlowShift on LIBERO-Plus, RoboTwin 2.0, and real-world bimanual manipulation tasks.
Critical strengths
  • Interesting integration of verification and action generation. I like the idea of using verifier feedback not only for selecting completed actions but also for guiding intermediate flow states. The risk-aware router is also a useful addition, since applying test-time scaling to every action chunk may introduce unnecessary distribution shifts.

  • Solid experimental results, especially under challenging settings. The improvements on RoboTwin Hardand real-world manipulation are encouraging. I also appreciate the ablations on verifier adaptation, gradient guidance, and router design, which help demonstrate the contributions of the individual components.

Critical weaknesses
  • Missing comparisons with existing test-time scaling approaches. Although RoboMonkey, MG-Select, TapSampling, and VLS are discussed, the main experimental tables mostly compare against pretrained VLA policies rather than existing TTS methods. I think adding at least one matched TTS baseline under the same backbone and inference budget would strengthen the paper. QGF is another relevant approach, particularly for comparing gradient-guided flow generation.

  • Several components have been explored individually in prior work, including stochastic flow sampling, reward-gradient guidance, and test-time verifier adaptation. The paper would benefit from a clearer discussion of what is fundamentally new beyond combining these techniques. I'm also not fully convinced by the verifier supervision: assuming linear progress across successful demonstrations may not reflect actual task progress, and reversing action sequences seems to produce relatively easy negatives rather than challenging failure modes.

Questions for authors
  • Could the authors add a matched baseline against at least one existing TTS approach, such as RoboMonkey, TapSampling, VLS, or QGF, using the same VLA backbone and a comparable inference budget? This would help clarify the effectiveness of the proposed method.
  • Could FlowShift complement post-hoc sampling and verification methods such as RoboMonkey or TapSampling? It would be interesting to see whether combining these approaches provides additional gains over either method alone.
  • What motivates the assumption of linear task progress along successful demonstrations? Also, why use reversed action sequences as negative examples instead of harder negatives from failed rollouts or alternative candidate actions? An ablation comparing different negative-sampling strategies would help establish the verifier's robustness.
  • Which types of tasks benefit most from FlowShift, and what specific failure modes does it address? For example, is the improvement mainly from recovering from inaccurate grasping/placement, handling visual distribution shifts, or correcting long-horizon execution errors? A more detailed breakdown would help explain when verifier-guided flow refinement is useful.
4 comments

Thank you for the thoughtful and constructive review. We have addressed your concerns through additional experiments and clarifications as follows. We would greatly appreciate it if you could consider giving us a higher rating.

W1 & Q1: Comparison with Existing Test-Time Scaling Methods

R1: To address this concern, we conducted additional comparisons with TapSampling and QGF using the same π₀.₅ backbone on LIBERO-Plus Goal under comparable end-to-end inference latency. For TapSampling, we use K=2 with 14 additional VAE-generated candidates (16 candidates in total), while QGF uses R=7 candidate action chunks.

MethodSR ↑ (%)Latency ↓ (ms)
π₀.₅ + TapSampling79.1293.73
π₀.₅ + QGF (R=7)79.2256.87
π₀.₅ + FlowShift (Ours)80.1252.99

FlowShift outperforms TapSampling and QGF by 1.0 and 0.9 percentage points, respectively, while achieving lower inference latency. Unlike post-hoc candidate selection or conventional gradient-based guidance, FlowShift integrates test-time adaptive verification, verifier-guided stochastic flow refinement, and risk-aware routing to selectively allocate additional computation. These results demonstrate the effectiveness of FlowShift over existing TTS approaches under comparable inference budgets.

W2 & Q3: Methodological Novelty and Verifier Supervision

R2: Methodological Novelty: FlowShift's novelty lies in the coordinated design of three components, rather than introducing stochastic sampling or gradient guidance in isolation:

  • Risk-Aware Stochastic Sampling: Unlike methods that apply stochastic refinement indiscriminately, FlowShift introduces a risk-aware router that selectively activates verifier-guided SDE transitions for high-risk action chunks while preserving the original ODE path for low-risk predictions, limiting unnecessary distribution shifts.
  • Adaptive Gradient Guidance: Rather than relying on a fixed reward function, FlowShift uses a test-time adaptive verifier whose reward values and gradients evolve with deployment observations. The same verifier provides gradient guidance during flow generation and scalar rewards for candidate selection, enabling deployment-aware action refinement.
  • Verifier-Level TTA: Unlike existing VLA test-time adaptation methods that adapt policy-side representations or prediction modules, FlowShift adapts only the verifier's latent prompts through self-supervised state prediction. This enables lightweight online adaptation to influence action generation through reward gradients without modifying the pretrained VLA backbone.

Verifier Supervision: Our task-progress supervision draws on the progress supervision strategy of TapSampling, using linearly increasing progress labels along successful demonstrations and reversed expert actions for negative supervision. Similar principles have been explored in SARM, ReWiND, and WARP-RM, which leverage temporal progress assumptions and reversed temporal sequences to model task advancement and regression. Thus, our supervision design is grounded in established progress-supervision principles rather than constituting our primary methodological novelty. The practical effectiveness of the verifier is supported by the ablations in Tables 3 and 4 in the main paper. While more fine-grained progress supervision is possible, it often requires costly manual annotations or large-scale VLM-based labeling, introducing additional data preparation overhead.

Negative-Sampling Ablation: To further examine verifier robustness, we conducted an additional ablation on LIBERO-Plus Goal. Specifically, we collected failed π₀.₅ rollouts on the original LIBERO-Goal and manually identified action chunks that cause task stagnation or regression, obtaining 500 challenging negative samples. We compared three strategies under identical verifier architectures and training budgets: (1) reversed-action negatives, (2) failed-rollout negatives, and (3) a 1:1 mixture of both.

Negative Sampling StrategySR ↑ (%)
Reversed Actions (Original)80.1
Failed-Rollout Negatives79.5
Reversed + Failed (Mixed)79.7

The original reversed-action supervision achieves the best performance, demonstrating the effectiveness of our annotation-free design. The benefits of failure-derived negatives may depend on data scale and annotation quality, which also introduce additional collection and labeling costs. We will clarify these findings in the revision.

Q2: Complementarity with Post-hoc Sampling and Verification

R3: To investigate the complementarity between FlowShift and post-hoc sampling methods, we conducted an additional experiment applying TapSampling on top of π₀.₅ + FlowShift, using TapSampling's default configuration on LIBERO-Plus Goal.

MethodSR ↑ (%)Latency ↓ (ms)
π₀.₅ + FlowShift80.1252.99
π₀.₅ + FlowShift + TapSampling79.4880.91

The combined method achieves 79.4% success, but does not outperform FlowShift alone (80.1%), while substantially increasing inference latency. This overhead arises from stacking two sampling and verification procedures. One possible explanation for the lack of additional gains is that TapSampling's fixed verifier may not fully accommodate the action distribution refined by FlowShift's adaptive guidance.

These results suggest that naively combining the two methods does not guarantee complementary benefits. Developing a more efficient integration could be a promising direction for future work.

Q4: Task-Specific Benefits and Failure Modes Addressed by FlowShift

R4: As shown in Table 1 in the main paper, FlowShift achieves the largest improvements on LIBERO-Plus under Camera (+2.5%), Layout (+1.6%), and Background (+1.4%) perturbations, indicating that it is particularly effective under visual and spatial distribution shifts. The detailed improvements across perturbation categories are summarized below:

Perturbationπ₀.₅ SR (%)+ FlowShift SR (%)Gain
Camera70.973.4+2.5
Layout85.787.3+1.6
Background94.395.7+1.4
Robot75.276.1+0.9
Noise86.587.0+0.5
Language85.385.4+0.1
Light96.596.4-0.1

Real-World Failure Modes: Our additional qualitative demonstrations show that FlowShift can identify risky action predictions involving inaccurate action poses and spatial misalignment, and refine the corresponding action trajectories. Furthermore, FlowShift improves real-world long-horizon task success from 27.5% to 40.0%, suggesting that risk-aware action refinement helps mitigate error accumulation across dependent subgoals. Since aggregate success rates cannot fully capture these intermediate interventions and recovery behaviors, we provide anonymous demonstration videos (link) illustrating risk-aware action correction and long-horizon execution in real-world settings.

Overall, FlowShift is particularly beneficial for visual and spatial distribution shifts, inaccurate action execution, and long-horizon manipulation tasks involving accumulated execution errors.

Thank you for the detailed response and for running the additional experiments! I think the overall framework is interesting and the additional experiments strengthen the paper. I will raise my score.

Thank you very much for your thoughtful review and constructive comments. Your valuable feedback has been very helpful in improving the quality and clarity of our paper. We are delighted that our responses and additional experiments have addressed your concerns. We sincerely appreciate your positive reassessment of our work and your willingness to raise the score.

Reviewer_qvWzOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To improve flow-matching based VLA's performance, three components are proposed; (1) task-progress verifier, (2) verifier-guided stochastic flow sampling, and (3) risk-aware router. In two simulation based benchmarks (LIBERO-Plus, RoboTwin 2.0), performance gain in LIBERO-Plus is marginal but one in RoboTwin 2.0 is noticeable.

Critical strengths
  • Using a single verifier for both guidance and selection of action candidates is efficient design.
  • Empirical gains in RoboTwin 2.0 are noticeable.
  • Real world robot experiments are a plus.
Critical weaknesses
  • Lack of additional computational analysis. Newly proposed methods will add computational cost but it is not analyzed. If the cost is significant, practicality of the proposal is limited.
  • Following verifier gradient on the generated actions may not be reliable especially when the verifier is updated in online manner.
  • As the method is plug and play, how much the empirical gain differs with different backbone architectures (eg, GR00T, other than π0.5\pi_{0.5})
Questions for authors
  • How to train router?
  • Contrast the novelty of the proposal against other guidance based methods and SMC.
  • How much is the additional computational cost (compare the end-to-end latency to base VLA)
  • Any case that the guidance harms the performance (worse than base policy)?
5 comments

Thank you for the thoughtful and constructive review. We have addressed your concerns through additional experiments and clarifications as follows. We would greatly appreciate it if you could consider giving us a higher rating.

W1 & Q3: Computational Overhead and Practicality

R1: End-to-End Latency Comparison: We conducted additional inference latency experiments on LIBERO-Plus Goal, comparing FlowShift with the original π₀.₅ and π₀.₅ + TapSampling, an existing test-time scaling (TTS) method. For TapSampling, we adopt its original default inference configuration, where the original policy generates 4 action candidates and Action-VAE generates 20 additional candidates (24 candidates in total).

MethodMean Latency ↓ (ms)
π₀.₅ (Base)103.88
π₀.₅ + TapSampling386.83
π₀.₅ + FlowShift (Ours)252.99

Although FlowShift introduces additional computation compared to the base policy, it achieves 34.6% lower average latency than TapSampling. This efficiency stems from our risk-aware router, which selectively activates computationally intensive TTS for potentially unreliable action predictions rather than applying TTS to every action chunk. Notably, the standard ODE path incurs only 4.41 ms (4.2%) additional latency compared to the base policy (108.29 vs. 103.88 ms), preserving near-baseline inference efficiency when TTS is not triggered.

Component-Level Computational Analysis: We further quantify the computational costs of individual components:

ComponentODE (ms)TTS (ms)
Visual encoding and prefix prefill35.1952.14
Action expert forward51.73107.09
Router4.405.11
Search and scheduler8.3217.31
Policy guidance backward—101.32
Verifier gradient RPC—381.53
Verifier scoring RPC—170.54
I/O and other overhead6.2212.42
Policy subtotal105.87847.45
Client communication2.423.09
Total108.29850.54
Additional Cost Accounting (Non-additive)
Verifier backward-related overhead—82.54
TTA overhead (amortized)—99.40

The verifier backward-related overhead (82.54 ms per TTS request, including server-side overhead) and amortized test-time adaptation cost (99.40 ms per TTS request) are already included in the measured end-to-end latency. While individual TTS requests incur substantial computational overhead, FlowShift selectively allocates these costs to high-risk predictions. Overall, FlowShift reduces average inference latency by 34.6% compared to the evaluated TTS baseline while maintaining near-baseline efficiency along the standard ODE path, demonstrating a more practical balance between computational cost and selective test-time refinement.

W2: Reliability of Gradient Guidance under Online Verifier Adaptation

R2: As shown in Table 4 in the main paper, combining TTA with gradient guidance achieves the best performance, improving LIBERO-Plus success from 83.9% to 85.0% compared to gradient guidance without TTA, and real-world generalization success from 70.0% to 83.8%. These results empirically support the effectiveness of gradient guidance with online verifier adaptation. We further address potential reliability concerns through two complementary mechanisms:

  • Controlled Online Adaptation: To avoid frequent updates that may destabilize verifier gradients, FlowShift adopts a buffer-based TTA strategy, where only TTS-selected action chunks are collected for adaptation. The verifier is updated only after accumulating 32 such chunks, rather than after every action prediction. We observe that updates typically occur in the middle-to-late stages of the episodes that trigger TTA, after sufficient high-risk samples have accumulated. This selective, periodic adaptation limits frequent changes to the verifier while allowing it to accommodate deployment shifts.
  • Risk-Aware Gradient Guidance: Verifier gradients provide soft steering during stochastic flow sampling rather than directly replacing generated actions. Moreover, gradient guidance is complemented by verifier-based candidate scoring and selection, while the risk-aware router preserves the original ODE path for low-risk predictions. These mechanisms jointly limit unnecessary exposure to potentially inaccurate gradients and reduce excessive deviations from the pretrained policy.

Overall, the empirical improvements in Table 4 in the main paper, together with controlled online adaptation and risk-aware sampling, support the effectiveness and practical reliability of gradient-guided refinement under online verifier updates.

W3: Generalizability Across Different VLA Backbones

R3: To evaluate FlowShift's generalizability beyond π₀.₅, we conducted additional experiments on GR00T, a VLA model with a different backbone architecture, using LIBERO-Plus Goal. Since GR00T supports configurable flow-matching inference steps, we adopt 10 denoising steps for both the original GR00T and GR00T + FlowShift to ensure a consistent comparison.

MethodSR ↑ (%)
GR00T72.3
GR00T + FlowShift73.1 (+0.8)

FlowShift improves GR00T's success rate from 72.3% to 73.1%, achieving a 0.8 percentage-point gain. These results provide additional empirical evidence that FlowShift's test-time refinement is not restricted to π₀.₅ and can be integrated with different flow-based VLA architectures, supporting its plug-and-play applicability.

Q1: Router Training and Threshold Calibration

R4: As described in Sec. 3.4 and Appendix C.2, the risk-aware router is trained offline using policy rollouts with the pretrained VLA backbone frozen. Specifically, for LIBERO-Plus evaluation, training rollouts are collected from the original LIBERO suites, while for RoboTwin 2.0, they are collected under the clean setting, without using test-time perturbations. We further clarify the training details as follows:

  • Supervision and Optimization: Each rollout is assigned a binary outcome label (1 for failure and 0 for success), which supervises the router's risk prediction at every action chunk without requiring additional chunk-level annotations. The timestep aggregation module, Mamba backbone, and risk prediction head are jointly optimized using class-balanced binary cross-entropy, with inverse-frequency class weights and padded sequence positions excluded from the loss.
  • Feature Extraction and Training Data: For each action chunk, final-layer action features from denoising steps {1, 3, 5, 7} are aggregated by a temporal Transformer, followed by a causal Mamba module that models cross-chunk execution history to predict failure risk. The collected trajectories are split into training and held-out calibration sets within each task.
  • Threshold Calibration: Following SAFE, we calibrate position-dependent risk thresholds using one-sided functional conformal prediction on held-out successful trajectories, with a miscoverage level of α = 0.1. During inference, risk scores exceeding the calibrated threshold activate TTS, while low-risk action chunks retain the original ODE sampling.

We will incorporate these additional training details into the revised manuscript to improve reproducibility.

Q2: Novelty Compared with Gradient-Based Guidance and SMC

R5: The novelty of FlowShift lies in integrating deployment-adaptive reward modeling, verifier-guided stochastic flow refinement, and risk-aware computation allocation, rather than introducing gradient guidance or particle sampling in isolation. We clarify the distinctions as follows:

  • Compared with Gradient-Based Guidance (e.g., QGF): Existing methods typically rely on a pretrained, fixed critic to guide action generation. In contrast, FlowShift introduces a test-time adaptive task-progress verifier that updates its latent prompts through self-supervised state prediction to accommodate deployment shifts. Importantly, the same adaptive verifier provides both reward gradients for guiding stochastic flow transitions and scalar rewards for evaluating and selecting candidate actions, enabling coordinated guidance and selection under changing environments.
  • Compared with SMC (e.g., DAS): Conventional SMC methods approximate reward-aligned distributions through sequential particle propagation, importance weighting, and resampling. FlowShift instead integrates verifier-gradient steering directly into selected stochastic flow transitions and performs reward-based candidate selection. More importantly, its risk-aware router selectively activates TTS for high-risk action chunks while preserving the original ODE path for low-risk predictions, avoiding indiscriminate sampling interventions. Together with online verifier adaptation, this establishes a deployment-aware refinement framework distinct from conventional SMC.

Latency-Matched Comparison with QGF: We further reproduced QGF on the same π₀.₅ backbone using 7 candidate action chunks (R=7), achieving comparable inference latency on LIBERO-Plus Goal:

MethodSR ↑ (%)Latency ↓ (ms)
π₀.₅ + QGF (R=7)79.2256.87
π₀.₅ + FlowShift (Ours)80.1252.99

FlowShift outperforms QGF by 0.9 percentage points while maintaining slightly lower inference latency. These results further support the effectiveness of combining adaptive verification, gradient-guided stochastic sampling, and risk-aware routing beyond conventional gradient-based flow guidance.

Q4: Cases Where Guidance Negatively Affects Performance

R6: Yes, we observe cases where FlowShift negatively affects previously successful executions. On LIBERO-Plus Spatial, 94 episodes that succeeded with the original π₀.₅ fail after applying FlowShift, while 128 previously failed episodes become successful, yielding a net gain of 34 successful episodes.

These results indicate that although stochastic refinement may occasionally disrupt successful trajectories, it recovers more failed executions than it introduces. Our risk-aware router mitigates such negative interventions by preserving the original ODE path for low-risk predictions and selectively activating TTS for high-risk ones, resulting in an overall performance improvement.

Comments on the paper

3 comments

Thank you for the thoughtful and constructive review. We have addressed your concerns through additional experiments and clarifications as follows. We would greatly appreciate it if you could consider giving us a higher rating.

W1 & Q1: Compute-Aware Comparison with Simpler TTS Strategies

R1: To address this concern, we conducted additional experiments on LIBERO-Plus Goal, comparing FlowShift against ODE + Best-of-2 and Full-step SDE + Best-of-2, using the same π₀.₅ backbone and verifier. Both baselines generate two action candidates and select the highest-scoring one, while the latter applies SDE sampling across all flow steps.

MethodSR ↑ (%)Latency ↓ (ms)
ODE + Best-of-278.8333.71
SDE + Best-of-278.7332.55
FlowShift (Ours)80.1252.99
  • Computational Efficiency: FlowShift achieves 24.2% and 23.9% lower average latency than the two baselines, respectively. Its risk-aware router activates expensive SDE refinement for only 19.49% of action chunks, avoiding unnecessary computation.
  • Effective Test-Time Scaling: FlowShift improves success rates by 1.3 and 1.4 percentage points over the two baselines, respectively. Unlike Best-of-N methods that perform only post-hoc candidate selection, FlowShift directly incorporates verifier gradients into action generation, enabling targeted refinement of potentially unreliable actions.

A detailed latency breakdown, including NFE, verifier computation, and TTA overhead, is provided in our response to W4 & Q4.

W2 & Q2–Q3: Effectiveness and Frequency of Test-Time Verifier Adaptation

R2: We conducted additional experiments to evaluate verifier ranking accuracy and clarify the online adaptation protocol.

Verifier Ranking Accuracy: We evaluated 8 candidates per observation using demonstration-derived progress labels as ground truth. After four steps of self-supervised latent-prompt adaptation, Top-1 Accuracy improves from 33.0% to 58.5% (+25.5 points), without using reward labels during adaptation.

MetricWithout TTAWith TTA
Top-1 Accuracy ↑ (%)33.058.5 (+25.5)

Component Complementarity: As shown in Table 4 in the main paper, TTA improves verifier discrimination, while gradient guidance utilizes the adapted reward signals to refine action generation. Their combination improves LIBERO-Plus average SR from 84.1% to 85.0% and real-world generalization SR from 75.0% to 83.8%.

Adaptation Timing and Frequency: Only TTS-selected action chunks enter the buffer. Once 32 chunks accumulate, four SGD updates are performed; the buffer is cleared while the adapted prompts are retained until the episode ends, when both are reset. On LIBERO-Plus Goal, 19.49% of action chunks trigger TTS, and 12.5% of episodes trigger TTA. We observe that updates typically occur in the middle-to-late stages of these episodes, after sufficient high-risk samples have accumulated. This selective adaptation enables the verifier to better accommodate deployment shifts during critical task stages, while limiting frequent updates that could destabilize action sampling. We will clarify the protocol in the revision.

W3 & Q5–Q6: Empirical Distributional Control and Theoretical Justification

R3: To address these concerns, we conducted two additional experiments using the actual π₀.₅ backbone and FlowShift sampler on 24 fixed contexts from 8 LIBERO-Plus tasks covering 7 perturbation categories. We generated 64 paired ODE/TTS samples per context with λ=0.01λ=0.01, varying the router threshold (risk>γ+Δγrisk > γ + Δγ). We measure MMD² against the original ODE distribution and an independently constructed empirical reference for the reward-aligned target distribution in Eq. (3). Since this fixed-context sampling evaluation differs from the full LIBERO-Plus Goal rollout evaluation, the observed TTS activation rate under the default threshold is slightly different (21.16% vs. 19.49%).

Threshold Offset ΔγTTS Rate (%)MMD² to ODEMMD² to ptarp_{tar}
Original ODE0—0.013879
FlowShift (Δγ=0, default)21.160.0008950.010078
−0.01012841.020.0025180.011643
−0.02514655.010.0053480.014755
−0.02522571.090.0078570.017416
−0.16551187.110.0115510.020255
Always-TTS1000.0169190.025360

Empirical Distributional Control: MMD² to the original ODE distribution increases monotonically with TTS activation across the evaluated thresholds, demonstrating that the router regulates the degree of distributional deviation. Importantly, FlowShift under our original routing configuration (Δγ=0) activates TTS for approximately 21% of action chunks in this experiment and achieves a lower MMD² to the empirical reward-aligned target (0.010078) than both the original ODE (0.013879) and Always-TTS (0.025360). These results provide empirical evidence that selective refinement shifts the implemented sampling distribution closer to the reward-aligned target while avoiding excessive deviation.

Theoretical Justification and Algorithmic Distributional Control: Our theoretical analysis establishes two complementary connections between the theoretical formulation and the implemented algorithm:

  • Reward-Aligned Target (Eq. 3): The KL-regularized objective yields the exact reward-tilted optimum, providing a principled target for verifier-guided generation. FlowShift approximates this target through reward-gradient guidance and particle selection, with empirical alignment evaluated above.

  • Controlled Stochastic Flow Dynamics: The same-marginal ODE–SDE equivalence holds for continuous-time dynamics with accurate score estimation. In FlowShift, stochastic transitions are restricted to selected flow steps with fixed noise and guidance schedules and a finite particle-search budget. For Gaussian proposal kernels with the same covariance, the guidance-induced conditional KL relative to the unguided SDE transition is

Thisexpressionexplicitlycharacterizestheguidance−induceddistributionaldeviationateachsamplingstate.Sincethenoiseschedule,guidancecoefficients,andstepsizearefixedandbounded,thedeviationisgovernedbytheverifier−gradientmagnitude.Overthefinitesetofevaluatedsamplingstates,thegradientnormshaveafinitemaximum,yieldingafinitesingle−stepKLbound.Thesecontrolledstochastictransitionsregulatethemagnitudeofindividualinterventions,whiletherisk−awarerouterfurtherlimitstheiractivationfrequency.Together,ourtheoreticalanalysisprovidesaprincipledreward−alignedtargetandcharacterizesthedistributionaleffectofindividualguidedstochastictransitions.Ouradditionalexperimentsfurtherdemonstratethatrisk−awareroutingeffectivelyregulatesdistributionaldeviationintheimplementedsampler.Wewillclarifythecorrespondingtheoreticalassumptionsandtheirapplicabilityintherevisedmanuscript. This expression explicitly characterizes the guidance-induced distributional deviation at each sampling state. Since the noise schedule, guidance coefficients, and step size are fixed and bounded, the deviation is governed by the verifier-gradient magnitude. Over the finite set of evaluated sampling states, the gradient norms have a finite maximum, yielding a finite single-step KL bound. These controlled stochastic transitions regulate the magnitude of individual interventions, while the risk-aware router further limits their activation frequency. Together, our theoretical analysis provides a principled reward-aligned target and characterizes the distributional effect of individual guided stochastic transitions. Our additional experiments further demonstrate that risk-aware routing effectively regulates distributional deviation in the implemented sampler. We will clarify the corresponding theoretical assumptions and their applicability in the revised manuscript.

W4 & Q4: Sampling Details and Inference Latency Breakdown

R4: We provide the complete sampling settings, pseudocode, and detailed latency analysis on LIBERO-Plus Goal.

Sampling Hyperparameters:

ParameterSetting
Diffusion schedule τ10 uniform steps, τ: 0 → 1, Δτ=0.1; g(τ)=0.3(1−τ)
Guidance coefficient ω(τ)min(1, 1.07^k−1), with ω₈=0.7182 and ω₁₀=0.9672
Regularization λ0.01
Candidate count4 proposals per step, with budget rollover
Total search budget8 proposals

Sampling Pseudocode:

Algorithm 1: FlowShift Sampling
Input: observation O, instruction l, state s,
       router history H, latent prompts P, buffer B
Parameters: T=10, Δτ=0.1, S={8,10}, M=4,
            total_budget=8, λ=0.01

A ~ N(0,I)                              # τ = 0
A, F = ODE_Sample(A, steps=1...7)
risk, γ = Router(F[{1,3,5,7}], H)

if risk <= γ:
    return ODE_Sample(A, steps=8...10)

# Test-time adaptation
A_pred = A + (1-τ) * vθ(A,τ)
B.append_once(O,l,stopgrad(A_pred),s)
if |B| == 32:
    P = SGD(P,L_state(B),steps=4,microbatch=4)
    B.clear()
C = CacheVerifierContext(O,l,P)

# Verifier-guided stochastic sampling
R_best = -inf; q = M; used = 0
for k = 8...10:
    τ = (k-1)/T
    if k not in S:
        A = A + Δτ*vθ(A,τ)
        continue

    A_pred = A + (1-τ)*vθ(A,τ)
    G = ∇_A Rφ(A_pred|C) / λ
    g = 0.3*(1-τ)
    ω = min(1,1.07^k-1)
    local_best = empty
    accepted = false

    for j = 1...min(q,total_budget-used):
        ε ~ N(0,I)
        Y = A + Δτ*fθ(A,τ)
              + g*sqrt(Δτ)*ε + ω*g²*Δτ*G
        r = Rφ(TweedieEstimate(Y,τ+Δτ)|C)
        used += 1
        UpdateLocalBest(local_best,Y,r)

        if r > R_best:
            A = Y; R_best = r
            q = M + (q-j)             # rollover
            accepted = true
            break

    if not accepted:
        A = local_best.action
        q = M

return A                              # τ = 1

Inference Latency and NFE: FlowShift achieves an average end-to-end latency of 252.99 ms on LIBERO-Plus Goal, with mean and tail latency across routing paths reported below. Actual NFE counts action-denoiser forward evaluations, including guidance-related forwards.

RouteMean ↓ (ms)P50 ↓ (ms)P95 ↓ (ms)P99 ↓ (ms)NFE ↓
ODE108.29109.87122.88139.1210
TTS850.54714.69967.986,658.1812
Overall252.99112.56863.13966.8510.39

Component-Level Breakdown: We further quantify the additional costs of verifier backpropagation (82.54 ms per TTS request, including other server-side overhead) and test-time adaptation (99.40 ms per TTS request, amortized). The detailed latency breakdown is:

ComponentODE (ms)TTS (ms)
Visual encoding and prefix prefill35.1952.14
Action expert forward51.73107.09
Router4.405.11
Search and scheduler8.3217.31
Policy guidance backward—101.32
Verifier gradient RPC—381.53
Verifier scoring RPC—170.54
I/O and other overhead6.2212.42
Policy subtotal105.87847.45
Client communication2.423.09
Total108.29850.54
Additional Cost Accounting (Non-additive)
Verifier backward-related overhead—82.54
TTA overhead (amortized)—99.40

The verifier backward-related overhead includes synchronization and other server-side costs. Both additional costs are already included in the measured end-to-end latency and are not added again.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.