acceptodds
Under review as a conference paper at ICLR 2027

Robustness Margins for Local Visual Perturbations in Vision–Language–Action Policies with Flow Matching

Abstract

Vision-language-action (VLA) policies often remain effective under visual disturbances such as changes in lighting, but it is not well understood why their behavior is insensitive to some perturbations and highly sensitive to others. As a first step toward understanding this robustness, we study a more local question: given a fixed observation and a perturbation to its visual conditioning, how much can the distribution of executed action chunks change? We formulate this question probabilistically and derive a local bound on the shift of the action distribution under small perturbations. The bound depends on the magnitude and direction of the input perturbation together with the sensitivity of the policy’s flow to its conditioning variable, characterized through the Jacobian. This provides a way to relate perturbations in observation space to changes in the distribution of sampled actions without making assumptions about downstream task success. We empirically study this relationship in LIBERO using several classes of visual perturbations and compare the predicted sensitivity with the observed shifts in action distributions. Our results provide an initial framework for analyzing VLA robustness through the geometry of how conditioning perturbations propagate through the generative action policy.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Flow matching, Local Robustness Analysis

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_t1KMOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper studies how a specified visual perturbation changes the executed-action distribution of a flow-matching VLA at a fixed observation. It propagates the measured context change through clean-trajectory Jacobians and bounds the nonlinear remainder using recursive curvature tubes. A Wasserstein bound is then converted into a clean-action retention bound, with a refinement accounting for predicted mean translation. Experiments with a LIBERO-finetuned π0.5 policy examine six perturbation families at two observations. They show directional differences at matched context magnitudes, close numerical agreement for small perturbations, and increasing conservatism at larger magnitudes.

Critical strengths

The recursive tube construction provides a coherent way to control unknown perturbed trajectories using the clean trajectory and measured context change. I could follow the containment argument in Equations (14)–(16) and Appendix A.1.1. The analysis also explicitly distinguishes generated chunks from executed coordinates. Its assumptions and the distinction between exact bounds and numerical estimates are clearly acknowledged.

The experiments support the importance of perturbation direction. At context magnitude 500, patch noise produces a sixfold larger empirical shift than contrast for the black-bowl observation, while their ordering reverses for cream cheese (Table 1). Translation adjustment also improves the retention estimates: for contrast near magnitude 180, the estimate increases from 22.7% to 68.7%, against observed coverage of 95.2% (Table 3). Separate fitting, calibration, and evaluation samples are used for the clean-action region.

Critical weaknesses
  1. The reliability of the numerical remainder control is not established. The theoretical construction requires curvature upper bounds and enclosing tubes. Appendix A.2.1 acknowledges that finite searches and power iteration can underestimate the required quantities, so the computed tubes may fail to contain the actual displacements. The numerical study does not check this containment or report sensitivity to the search budgets. Agreement with empirical Wasserstein distance does not establish control of the shared-noise prediction error, since that distance can be smaller than the displacement being bounded. Moreover, the candidate-set construction and optimization procedure are not specified on pages 15–16. The timing uses five interpolation points, whereas the main-study description specifies 16. These omissions prevent me from assessing whether the reported tight estimates survive more thorough numerical evaluation.

  2. The experiments do not isolate the value of recursive curvature control beyond first-order sensitivity. Figure 3 does not separately report the first-order term in Equation (17), the estimated remainder, and the actual linearization error. The unrestricted joint-direction comparison also changes the evaluation domain and pools maxima across 16 noise draws, so it does not isolate the effect of discarding direction. This matters because the estimates closely match empirical distances for very small perturbations but can become extremely loose: cream-cheese patch warp gives 9.5 against an empirical distance of 0.070 at magnitude 250 (Table 2). The retention application is encouraging, but its KDE bandwidth and explicit margin-lower-bound calculations are also missing. Thus I cannot verify the probability estimates or determine where the additional curvature calculation provides useful control.

Questions for authors
  1. On the existing paired trajectories, does the actual prediction error remain within the numerical radius at every flow step? How does containment change when increasing displacement candidates, interpolation points, and power iterations? Please specify the search algorithms and configurations used for each plot and the timing.

  2. Could you show the first-order term, estimated nonlinear contribution, and actual linearization error together for the existing observations? Please also clarify how differences in evaluation domain and aggregation across noise draws affect Figure 3.

  3. What KDE bandwidth and explicit formulas, constants, and grid parameters produce the margin lower bounds in Appendix A.3? These are needed to reconstruct the retention estimates, including the 82% value in Figure 4.

4 comments
AuthorsOpenReview ↗

Response to t1KM: Q1

[Q1] Tube containment. How does containment change when increasing displacement candidates, interpolation points, and power iterations?

Thank you for raising this concern. We have also added the containment table and its evaluation protocol to Appendix A.2.3 of the revised paper, and reproduce the table below for convenience. Our tube construction bounds the nonlinear error around the first-order displacement prediction. Here we show that tube containment failures are infrequent and small in absolute magnitude given the numerical approximation methods used. We'd like to emphasize that theory guarantees tube containment. The true displacement can lie outside of the tube in practice due to numerical underestimation from finite searches and power iteration. The inequality below describes how we checked numerical tube containment with an absolute tolerance τ=10−5\tau=10^{-5} in normalized full-state units:

∥δk−δ^k∥F≤r^k+τ,\Vert\delta_k-\hat{\delta}_k\Vert_F \leq \hat r_k+\tau,

where δk\delta_k is the actual clean-to-perturbed displacement, δ^k\hat{\delta}_k its first-order prediction, and r^k\hat r_k the numerical radius.

For the two observations and six perturbation families in the main study, the paper settings use eight Gaussian initial-noise samples per observation, shared between clean and perturbed runs, and 10 power iterations. The table reports the percentage of checks that fail the above criterion. Each entry represents 96 checks: two observations ×\times six perturbations ×\times eight initial-noise samples.

Containment failures (%) with τ=10−5\tau=10^{-5}. Column headings 1–10 denote flow steps.

∥Δc∥F\Vert\Delta c\Vert_F12345678910
50000000000
10000002.11.0000
25000002.11.0000
500000000000
1000000000000
1500000000000
2500000000000
3750000000000
5000000000000

Across the nine tested magnitudes from ∥Δc∥F=5\Vert\Delta c\Vert_F=5 to 500, 6 of 8,640 checks (0.069%) fail this criterion. These six violations are small in absolute magnitude and occur only at ∥Δc∥F=10\Vert\Delta c\Vert_F=10 and 25 with maximum excess of 1.1×10−51.1\times10^{-5} and 3.77×10−53.77\times10^{-5} respectively.

Numerical convergence. We have added a numerical convergence study in Appendix A.2.2, with results and computation times measured on an NVIDIA RTX 5090 GPU in Tables 2–4. The theory guarantees containment under its assumptions. These experiments assess the accuracy and cost of its numerical implementation.

  • Table 2. At the paper's budget of 10 power iterations, median operator-norm errors are 1.37% for black bowl and 3.27% for cream cheese, with norm-estimation time about 33×33\times lower than explicit Jacobian construction and SVD. At 100 iterations, the median errors decrease to 0.03% and below 0.01%.

  • Table 3. Increasing the direction count from six for black bowl and four for cream cheese to 66 changes sampled curvature by at most 0.037% and the final estimate BB by less than 0.0013%.

  • Table 4. Increasing the interpolation grids from seven and five points, respectively, to 49 changes BB by at most 0.0086% across the tested conditions.

Tables 3 and 4 cover both observations and all six perturbations at ∥Δc∥F=100\Vert\Delta c\Vert_F=100, operator norms computed with 100 power iterations. The small changes under substantially larger searches support the paper's direction and interpolation budgets. Table 2 separately quantifies the power-iteration accuracy–cost trade-off, and direct containment results are reported above.

Clarifications on configurations used for the plots and timing. Thank you for pointing out the inconsistency between the interpolation counts in Appendix A.2.1 and the timing description. The stated value J=16J=16 in the appendix was a mistake. The main perturbation study uses J=7J=7 for black bowl and J=5J=5 for cream cheese, now documented in Appendix A.2.2. These runs use eight initial-noise draws and ten power iterations. The curvature ablation uses J=5J=5. We have corrected and clarified this mistake in Appendix A.2.1.

AuthorsOpenReview ↗

Response to t1KM: Q2

[Q2] Could you show the first-order term, estimated nonlinear contribution, and actual linearization error together for the existing observations? Please also clarify how differences in evaluation domain and aggregation across noise draws affect Figure 3.

Equation (17) provides a rigorous upper bound under the assumptions of Theorem 1 by controlling the nonlinear remainder. We agree that the first-order approximation is often numerically accurate in practice in many of the tested conditions in the low perturbation regime, although it can underestimate the measured displacement. Tables 6 and 7 in Appendix A.2.4 compare the first-order prediction, full estimate, and measured vector remainder using matched samples at magnitudes 25, 100, and 250. The magnitude-250 results are shown below.

Let aia_i and a^i\hat a_i be the actual and first-order predicted executed-action displacements for the same initial-noise draw. We report the paired RMS displacement D=RMS⁡(ai)D=\operatorname{RMS}(a_i), its first-order prediction D^=RMS⁡(a^i)\hat D=\operatorname{RMS}(\hat a_i), and the vector error R=RMS⁡(ai−a^i)R=\operatorname{RMS}(a_i-\hat a_i). Here DD is a matched-sample diagnostic, distinct from the empirical Wasserstein distance W2W_2. Here BB is the full numerical estimate. Its nonlinear part is B−D^B-\hat D, which can be obtained directly from the table. Vector error includes both magnitude and direction differences, which a comparison of magnitudes alone can miss.

All quantities use the original eight curvature draws and ten power iterations. We retain the original nonlinear term and average the first-order term over these same eight draws. Table 9 uses 128 draws for its first-order average, which explains the small remaining differences in BB. The curvature search uses six directions and seven interpolation points for black bowl, and four directions and five interpolation points for cream cheese. The first-order prediction uses the measured context change and derivatives along the clean flow trajectory. All values are in normalized action units.

Black bowl, ∥Δc∥F=250\Vert\Delta c\Vert_F=250

PerturbationPaired RMS DDFirst order D^\hat DFull estimate BBVector error RR
Brightness0.126730.121160.211690.01157
Contrast0.133390.136460.198460.00882
Patch noise0.249170.242720.267120.01182
Patch blur0.287070.281370.355330.02276
Patch saturation0.184050.173690.419470.04357
Patch warp0.193740.186990.261260.01526

Cream cheese, ∥Δc∥F=250\Vert\Delta c\Vert_F=250

PerturbationPaired RMS DDFirst order D^\hat DFull estimate BBVector error RR
Brightness0.050620.051480.062820.00233
Contrast0.286010.284080.877020.02134
Patch noise0.075390.072910.155600.00539
Patch blur0.090320.091050.145770.00401
Patch saturation0.108580.109840.169260.00418
Patch warp0.070080.069059.511130.00470

At ∥Δc∥F=250\Vert\Delta c\Vert_F=250, first order underestimates DD by more than 10−510^{-5} in 8 of 12 conditions. Across all 36 conditions in Tables 6 and 7, the full estimate covers DD and its nonlinear part covers RR. For black-bowl patch blur at magnitude 250, R=0.02276R=0.02276 and B=0.35533B=0.35533, showing one example of informative nonlinear control even when the first-order magnitude is close.

Regarding Figure 3, now Figure 4 in the updated paper, our tube construction evaluates curvature along candidate directions, whereas the unrestricted joint-direction calculation searches arbitrary directions at clean trajectory points. These differences affect the resulting estimates. The convergence results in Appendix A.2.2 support the numerical stability of our tube estimates under substantially larger search budgets, complementing the figure’s evidence for the usefulness of retaining perturbation-direction information.

AuthorsOpenReview ↗

Response to t1KM: Q3

[Q3] What KDE bandwidth and explicit formulas, constants, and grid parameters produce the margin lower bounds in Appendix A.3? These are needed to reconstruct the retention estimates, including the 82% value in Figure 4.

The KDE underlying Figure 5 (Figure 4 in the original submission) can be reproduced using scipy.stats.gaussian_kde(X) with default settings, where X contains the first five principal-components of 2048 clean action samples. We use 2048 separate clean flow samples to estimate the 95%-coverage density threshold. We define the retained region by thresholding the negative log KDE density at a level calibrated for 95% clean coverage. For Figure 5, we compute margin lower bounds analytically using the density gradient and a curvature bound determined by the Gaussian kernel covariance. The displayed contours are two-dimensional projections of this five-dimensional region, and the retention estimates apply the margin and retention calculations in Appendix A.4 of the updated paper to this KDE. We have added pseudocode for the five-dimensional margin and retention calculation to Appendix A.4, including the threshold, curvature constant, and residual budget that gave the 82% reported in the main paper. We plan to release the code after paper decisions as well.

AuthorsOpenReview ↗

Rebuttal follow-up

Dear reviewer t1KM,

Thank you again for your helpful suggestions on numerical reliability and reproducibility. As the discussion deadline approaches, please let us know if our response has addressed your questions and concerns. We'd also be happy to provide further clarifications if you have any more questions. Thank you for your time.

Best,

The Authors

Reviewer_2SkAOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This submission introduces a novel theoretical framework to analyse and quantify the robustness of VLA models, using flow matching for action generation. The main question addressed in this work is how small visual perturbations in input observations affect the distribution of output actions generated by the policy. This is derived through a rigorous local bound on the Wasserstein-2 distance between the clean and perturbed action distributions, that can be derived using only the unperturbed model's flow trajectories and local curvature information, avoiding the need to perform inference on the perturbed model. The proposed method shows that this distribution-shift bound can be used to establish a lower bound on the probability that the policy's output will remain within a pre-defined region of "typical" actions.

Critical strengths
  • This work tackles a critical problem in robot learning. Understanding failure modes and robustness characteristics in learned robot policies is of significant importance for safe and reliable real-world deployment, and this work proposed a novel theoretical framework in this direction.
  • The key findings of the work are insightful. Specifically, the proposed bound analytically demonstrates that the policy's sensitivity is not uniform but depends on the specific direction of the visual perturbation in the conditioning space. Additionally, the paper provides a method to translate the distribution-shift budget into a probabilistic guarantee on "action retention". Both findings can guide future work in the field.
  • The ablation study, and comparison to the oracle, effectively highlight the utility of the bound in the local regime, as well as its limitations at larger perturbation magnitudes.
Critical weaknesses

The main limitation of this work is the scope of the conducted experiments. The practical applicability of the proposed method remains unclear, given the lack of real-world robotic experiments. More specifically:

  • It is unclear how realistically the adopted synthetic perturbations capture actual real-world variations, such as changes in scene lighting, organic object textures, camera noise, or the addition of distractor objects.
  • The exact limits of the perturbation scale—beyond which the derived bounds become too conservative and lose their informativeness—are not clearly established. Further evaluation under these larger, more realistic environmental changes is needed.
  • It remains unverified how transferable the findings of this analysis are to other VLA model architectures (beyond π_(0.5)) and alternative robot manipulation datasets (beyond LIBERO).
  • The numerical estimation of the bound is computationally intensive, requiring iterative searches and multiple model forward passes to calculate the curvature. This bottleneck limits its practical utility for real-time safety verification, and a more detailed analysis of the underlying computational trade-offs is required.
Questions for authors

Please consider replying to the concerns raised in the weaknesses section above.

2 comments
AuthorsOpenReview ↗

Response to 2SkA

[W1] The main limitation of this work is the scope of the conducted experiments. The practical applicability of the proposed method remains unclear, given the lack of real-world robotic experiments. It is unclear how realistically the adopted synthetic perturbations capture actual real-world variations, such as changes in scene lighting, organic object textures, camera noise, or the addition of distractor objects.

[W3] It remains unverified how transferable the findings of this analysis are to other VLA model architectures (beyond π0.5\pi_{0.5}) and alternative robot manipulation datasets (beyond LIBERO).

Thank you for the recommendation to use other diverse observations and datasets. We address weaknesses 1 and 3 together by adding a third observation setting from DROID using the finetuned π0.5\pi_{0.5} checkpoint lerobot/pi05_droid. Appendix A.3 of the updated paper shows both camera views, all six perturbation families, and the bounds in Figures 10 and 11.

At context magnitude 250, a 55.4% global brightness gain gives a bound B=1.764W2B=1.764W_2, while patch blur with a standard deviation of 3.22 pixels gives B=1.202W2B=1.202W_2. Patch warp remains a tight bound even at magnitude 500, with B=1.275W2B=1.275W_2.

This extends the evidence to recorded real-robot imagery from a different dataset, with synthetic perturbations representing brightness, contrast, and local image changes. Table 8 reports our bound across all conditions. Closed-loop real-robot validation and empirical transfer to other VLA architectures remain outside the present evaluation. The theoretical framework is applicable to other flow-matching VLA models under the stated assumptions, making GR00T N1 and SmolVLA natural extensions to try. Here we simply chose to focus on π0.5\pi_{0.5} on several different diverse observations as one example applying the derived theoretical bound.

[W2] The exact limits of the perturbation scale-beyond which the derived bounds become too conservative and lose their informativeness are not clearly established. Further evaluation under these larger, more realistic environmental changes is needed.

Unfortunately, there is no rule of thumb for a useful perturbation range that still provides an accurate bound. It depends heavily on the particular observation and perturbation of interest. For the DROID towel observation, patch saturation gives B=5.09W2B=5.09W_2 at ∥Δc∥F=250\Vert\Delta c\Vert_F=250, while patch warp remains tight at magnitude 500 with B=1.275W2B=1.275W_2. Thus, larger context changes do not necessarily produce less informative estimates. Please see Appendix A.2.5 and A.3 for visual examples and the DROID results through magnitude 500, which illustrate this variation across perturbations.

[W4] The numerical estimation of the bound is computationally intensive, requiring iterative searches and multiple model forward passes to calculate the curvature. This bottleneck limits its practical utility for real-time safety verification, and a more detailed analysis of the underlying computational trade-offs is required.

We agree that the current computational cost limits real-time use. Our method only provides a rigorous framework for offline robustness analysis. We have added an accuracy-cost study in Appendix A.2.2, Tables 2-4, with timings on an NVIDIA RTX 5090.

Table 2 shows that ten power iterations cost 1.40 seconds per trajectory for one-step Jacobian norm estimation, compared with 47.0 seconds for explicit Jacobian construction and SVD, with median relative errors of 1.37% and 3.27% for the two observations. Increasing to 100 iterations costs 14.0 seconds and reduces these errors to 0.03% and below 0.01%.

Tables 3 and 4 assess curvature search stability. Increasing the direction counts from six for black bowl and four for cream cheese to 66 raises search time from 8.23 to 69.07 seconds and from 4.49 to 49.25 seconds per trajectory, respectively, while changing BB by less than 0.0013%. Refining the interpolation grids to 49 points changes BB by at most 0.0086%. These results support the chosen search budgets over the tested conditions.

The table timings are average costs per ten-step initial-noise trajectory. The complete calculation requires multiple samples for Monte Carlo estimation, with norm estimation and curvature search accounting for most of the computation. This supports offline analysis or applications that can accommodate that delay, while faster real-time verification remains future work.

Thank you the great effort to address all raised concerns, and the provided clarifications and additional results. It is encouraging that the paper's findings remain consistent on a more realistic dataset, such as DROID. Further experiments with real-world deployment and more models would further strengthen the contribution of the work. Having considered all the discussions in the rebuttal phase, I am maintaining my WA score.

Reviewer_Sn5mOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how a specified visual perturbation changes the executed-action distribution of a flow-matching VLA at a fixed observation. Using the measured change in multimodal conditioning and Jacobians along clean flow trajectories, it separates the directional first-order response from the nonlinear remainder. A recursive tube construction controls the remainder and yields a conditional Wasserstein-2 bound without evaluating perturbed flow trajectories. The paper then applies a standard transport-based probability bound, with a predicted-translation adjustment, to estimate retention within a region defined by clean-action projections. Numerical experiments use a LIBERO-finetuned π0.5 policy at two fixed observations with six visual perturbation families. The contribution concerns local action-distribution sensitivity, rather than closed-loop task-success guarantees.

Critical strengths
  1. A principled, direction-sensitive framework for action-distribution analysis. The shared-noise coupling, directional linearization, and recursive remainder control provide a coherent way to relate visual-conditioning changes to the distribution of generated actions. Preserving the perturbation direction goes beyond a magnitude-only sensitivity measure, while the probability-bound construction gives an interpretable consequence for a specified clean-action region. This is a useful theoretical perspective on the behavior of conditional generative policies (Sections 4.1–4.3).

  2. Empirical comparisons that reveal relevant geometry and sources of conservatism. The matched-context-magnitude comparisons show that different perturbation families can induce substantially different action shifts, with the ordering also depending on the observation. Comparing paired RMS displacement with empirical Wasserstein distance helps distinguish the cost of the shared-noise coupling, and the translation adjustment improves retention estimates in several tested settings. These analyses provide useful observations beyond reporting task success alone (Figures 2–4; Table 1).

Critical weaknesses
  1. The reliability of the numerical tube approximation is not sufficiently established. The theoretical construction relies on curvature and operator-norm controls that ensure the true displacement remains inside the recursively constructed tube. The numerical implementation instead uses finite candidate searches, interpolation grids, and power iteration. The paper does not sufficiently quantify how often or how far the actual perturbed trajectories leave the estimated tubes, or whether the radii and final estimates stabilize as the candidate count, search budget, and power-iteration budget increase. Because these estimates enter the radius recursion, underestimation could produce an overly small tube and apparent tightness. Agreement with empirical Wasserstein distances or the numerical oracle does not distinguish reliable remainder control from insufficient search. This is a central concern because controlling the nonlinear remainder is a principal technical contribution. Detailed search specifications and containment/convergence diagnostics are needed to support the numerical results (Eqs. 14–17; Figure 3; numerical procedures in Appendix A.2.1).

  2. The added value of the tube correction and its useful operating range remain insufficiently characterized. The curvature ablation compares several curvature estimates but does not directly compare the first-order-only prediction with the full tube construction under matched samples. Consequently, the favorable small-perturbation results do not isolate the empirical value of the added nonlinear control. The relevant question is whether the correction remains reliable and informative when the first-order approximation is materially inaccurate—not merely whether the full estimate is numerically close to the measured shift. The main evidence comes from only two fixed observations of one policy. It does not establish whether the informative local range is stable across observations. Beyond the example images, the correspondence between context-displacement norms and interpretable visual severity is not sufficiently quantified, nor is it clear how often the estimates remain informative under deployment-relevant perturbation strengths. Table 1 shows that the estimates can become highly conservative for larger changes, making characterization of this operating range particularly important. Language and proprioception are held fixed. Their interactions with the visual input can already be encoded in the measured joint context, but the experiments do not characterize how those conditioning variables affect the perturbation rankings or useful ranges. The current results therefore describe conditional sensitivity at selected multimodal contexts, rather than establishing transfer across instructions and robot states. Together, these limitations leave the practical information gained from the proposed construction insufficiently demonstrated (Sections 5–6; Figure 3; Table 1).

Questions for authors
  1. Tube containment. Using the paired perturbed trajectories already generated for evaluation, how often does the true displacement lie outside the estimated tube at each flow step, and by how much? Please report the frequency and magnitude of containment failures and whether the numerical remainder estimate actually controls the observed linearization error.
  2. Search-budget stability and reproducibility. Do the estimated curvature, tube radii, and final budgets stabilize when increasing candidate counts, search iterations/restarts, and power-iteration budgets? Please specify the candidate-generation procedure and sampling budgets, and compare settings with matched random draws to distinguish numerical-search effects from Monte Carlo variation. These diagnostics would help separate genuine tightness from underestimation.
  3. Incremental contribution of nonlinear control. Under matched observations, perturbations, and initial-noise samples, how do the first-order-only prediction, full tube estimate, and actual paired displacement/remainder compare? Is there an identifiable regime where the first-order approximation is inaccurate but the nonlinear correction still provides reliable, non-vacuous control? The first-order baseline is requested as a diagnostic, not as an alternative certificate.
  4. Meaningful and transferable local range. Across additional pre-specified observations, how stable is the range in which the estimates remain informative? Please relate context-displacement magnitudes to interpretable perturbation parameters or visual severity, and indicate whether informative bounds persist at deployment-relevant strengths rather than only for very small context changes.
  5. Multimodal conditioning and interactions. How do the response to a matched visual perturbation, its directional ranking, and the informative range depend on the language instruction and proprioceptive state? Controlled comparisons across valid observation–instruction–state combinations, using a corresponding clean reference for each, would clarify whether the conclusions are specific to the selected conditioning or transfer across multimodal contexts.
3 comments
AuthorsOpenReview ↗

Response to Sn5m: Q1–Q2

[Q1] Tube containment. How often does the true displacement lie outside the estimated tube at each flow step, and by how much? Does the numerical remainder estimate control the observed linearization error?

Thank you for raising this concern. We have also added the containment table and its evaluation protocol to Appendix A.2.3 of the revised paper, and reproduce the table below for convenience. Our tube construction bounds the nonlinear error around the first-order displacement prediction. Here we show that tube containment failures are infrequent and small in absolute magnitude given the numerical budgets used. We'd like to emphasize that theory guarantees tube containment. The true displacement can lie outside of the tube in practice due to numerical underestimation from finite searches and power iteration. The inequality below describes how we checked numerical tube containment with an absolute tolerance τ=10−5\tau=10^{-5} in normalized full-state units:

∥δk−δ^k∥F≤r^k+τ,\Vert\delta_k-\hat{\delta}_k\Vert_F \leq \hat r_k+\tau,

where δk\delta_k is the actual clean-to-perturbed displacement, δ^k\hat{\delta}_k its first-order prediction, and r^k\hat r_k the numerical radius.

For the two observations and six perturbation families in the main study, the paper settings use eight Gaussian initial-noise samples per observation, shared between clean and perturbed runs, and 10 power iterations. The table reports the percentage of checks that fail the above criterion. Each entry represents 96 checks: two observations ×\times six perturbations ×\times eight initial-noise samples.

Containment failures (%) with τ=10−5\tau=10^{-5}. Column headings 1–10 denote flow steps.

∥Δc∥F\Vert\Delta c\Vert_F12345678910
50000000000
10000002.11.0000
25000002.11.0000
500000000000
1000000000000
1500000000000
2500000000000
3750000000000
5000000000000

Across the nine tested magnitudes from ∥Δc∥F=5\Vert\Delta c\Vert_F=5 to 500, 6 of 8,640 checks (0.069%) fail this criterion. These six violations are small in absolute magnitude and occur only at ∥Δc∥F=10\Vert\Delta c\Vert_F=10 and 25 with maximum excess of 1.1×10−51.1\times10^{-5} and 3.77×10−53.77\times10^{-5} respectively.

[Q2] Search-budget stability. Do the curvature estimates, tube radii, and final estimates stabilize as the numerical budgets increase?

We have added a numerical convergence study in Appendix A.2.2, with results and computation times measured on an NVIDIA RTX 5090 GPU in Tables 2–4. The theory guarantees containment under its assumptions. These experiments assess the accuracy and cost of its numerical implementation.

  • Table 2. At the paper's budget of 10 power iterations, median operator-norm errors are 1.37% for black bowl and 3.27% for cream cheese, with norm-estimation time about 33×33\times lower than explicit Jacobian construction and SVD. At 100 iterations, the median errors decrease to 0.03% and below 0.01%.

  • Table 3. Increasing the direction count from six for black bowl and four for cream cheese to 66 changes sampled curvature by at most 0.037% and the final estimate BB by less than 0.0013%.

  • Table 4. Increasing the interpolation grids from seven and five points, respectively, to 49 changes BB by at most 0.0086% across the tested conditions.

Tables 3 and 4 cover both observations and all six perturbations at ∥Δc∥F=100\Vert\Delta c\Vert_F=100, operator norms computed with 100 power iterations. The small changes under substantially larger searches support the paper's direction and interpolation budgets. Table 2 separately quantifies the power-iteration accuracy-cost trade-off. The chosen numerical budgets are supported by these numerical checks.

AuthorsOpenReview ↗

Response to Sn5m: Q3

[Q3] Incremental contribution of nonlinear control. Under matched observations, perturbations, and initial-noise samples, how do the first-order-only prediction, full tube estimate, and actual paired displacement/remainder compare? Is there an identifiable regime where the first-order approximation is inaccurate but the nonlinear correction still provides reliable, non-vacuous control? The first-order baseline is requested as a diagnostic, not as an alternative certificate.

We agree that the first-order approximation is often numerically accurate in practice in many of the tested conditions in the low perturbation regime. Equation (17) provides a rigorous upper bound under its assumptions by controlling the nonlinear remainder. Tables 6 and 7 in Appendix A.2.4 compare the first-order prediction, full estimate, and measured vector remainder for both observations and all six perturbations at ∥Δc∥F∈{25,100,250}\Vert\Delta c\Vert_F\in\{25,100,250\}. The magnitude-250 results are shown below.

Let aia_i and a^i\hat a_i be the actual and first-order predicted executed-action displacements for the same initial-noise draw. We report the paired RMS displacement D=RMS⁡(ai)D=\operatorname{RMS}(a_i), its first-order prediction D^=RMS⁡(a^i)\hat D=\operatorname{RMS}(\hat a_i), and the vector error R=RMS⁡(ai−a^i)R=\operatorname{RMS}(a_i-\hat a_i). Here DD is a matched-sample diagnostic, distinct from the empirical Wasserstein distance W2W_2. Here BB is the full numerical estimate. Its nonlinear part is B−D^B-\hat D, which can be obtained directly from the table. Vector error includes both magnitude and direction differences, which a comparison of magnitudes alone can miss.

All quantities use the original eight curvature draws and ten power iterations. We retain the original nonlinear term and average the first-order term over these same eight draws. Table 9 uses 128 draws for its first-order average, which explains the small remaining differences in BB. The curvature search uses six directions and seven interpolation points for black bowl, and four directions and five interpolation points for cream cheese. The first-order prediction uses the measured context change and derivatives along the clean flow trajectory. All values are in normalized action units.

Black bowl, ∥Δc∥F=250\Vert\Delta c\Vert_F=250

PerturbationPaired RMS DDFirst order D^\hat DFull estimate BBVector error RR
Brightness0.126730.121160.211690.01157
Contrast0.133390.136460.198460.00882
Patch noise0.249170.242720.267120.01182
Patch blur0.287070.281370.355330.02276
Patch saturation0.184050.173690.419470.04357
Patch warp0.193740.186990.261260.01526

Cream cheese, ∥Δc∥F=250\Vert\Delta c\Vert_F=250

PerturbationPaired RMS DDFirst order D^\hat DFull estimate BBVector error RR
Brightness0.050620.051480.062820.00233
Contrast0.286010.284080.877020.02134
Patch noise0.075390.072910.155600.00539
Patch blur0.090320.091050.145770.00401
Patch saturation0.108580.109840.169260.00418
Patch warp0.070080.069059.511130.00470

At ∥Δc∥F=250\Vert\Delta c\Vert_F=250, first order underestimates DD by more than 10−510^{-5} in 8 of 12 conditions. Across all 36 conditions in Tables 6 and 7, the full estimate covers DD and its nonlinear part covers RR. For black-bowl patch blur at magnitude 250, R=0.02276R=0.02276 and B=0.35533B=0.35533, showing one example of informative nonlinear control even when the first-order magnitude is close.

AuthorsOpenReview ↗

Response to Sn5m: Q4–Q5

[Q4] Meaningful and transferable local range. Across additional pre-specified observations, how stable is the range in which the estimates remain informative? Please relate context-displacement magnitudes to interpretable perturbation parameters or visual severity, and indicate whether informative bounds persist at deployment-relevant strengths rather than only for very small context changes.

Thank you for your question. We have added sections A.2.5 and A.3 in the appendix to address these questions. A.2.5 relates context magnitudes to visible image changes more directly. For the LIBERO cream-cheese observation, a 25.9% global brightness gain gives ∥Δc∥F≈250\Vert\Delta c\Vert_F\approx250 and our bound gives B=1.24W2B=1.24W_2. For black-bowl patch blur, a standard deviation of 4.67 pixels gives the same context magnitude and B=1.23W2B=1.23W_2. These ratios use the main perturbation study, and W2W_2 is the empirical Wasserstein-2 distance.

Appendix A.3 adds a third observation from the DROID dataset for a real robot towel grabbing frame across the same perturbation families in the main study in Figures 10 and 11 and Table 8. At magnitude 250, a 55.4% global brightness gain gives B=1.764W2B=1.764W_2, and patch blur with a standard deviation of 3.22 pixels gives B=1.202W2B=1.202W_2. These examples demonstrate visible changes with informative estimates beyond the two LIBERO observations.

[Q5] Multimodal conditioning and interactions. How do the response to a matched visual perturbation, its directional ranking, and the informative range depend on the language instruction and proprioceptive state? Controlled comparisons across valid observation-instruction-state combinations, using a corresponding clean reference for each, would clarify whether the conclusions are specific to the selected conditioning or transfer across multimodal contexts.

Our study is limited to robustness to visual perturbations, keeping the language instruction and proprioceptive state fixed within each clean-perturbed comparison. This choice was partly motivated by our observations that the LIBERO-finetuned π0.5\pi_{0.5} model sometimes responded weakly to instruction changes that altered the intended task. Related language insensitivity on the LIBERO dataset has been reported in the LIBERO-PRO paper by Zhou et al. In this setting, a small action-distribution shift following a task-changing instruction would not necessarily indicate desirable robustness.

We therefore focused on visual changes under fixed task conditioning. We agree that this does not establish the interaction between multimodal contexts. Such comparisons are left for future work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions and responses

We would like to thank all reviewers for their thoughtful feedback. It is encouraging that the reviews recognize the value of a direction-sensitive framework for analyzing VLA action distributions, the coherence of the recursive tube construction, and the insights provided by the perturbation comparisons and action-retention analysis. We appreciate the constructive suggestions for strengthening the numerical evaluation and clarifying the method’s practical scope and limitations.

We have responded individually to each reviewer to address the weaknesses and concerns brought to our attention. Please don't hesitate to leave a comment if there are any follow-up questions or remaining concerns, particularly those that would need to be addressed for you to recommend acceptance.

Please note that the main paper and appendix have been updated thanks to reviewer feedback. The additions are as follows:

  • A new image has been added at the top of page 2 (Figure 1) to visualize the robustness bound during the denoising process. We hope this image clarifies the theory and provides intuition as to why bounding nonlinear terms is more informative than a first order prediction of action shift alone.

  • We have added numerical convergence tables in the appendix (recommended by reviewers Sn5m and t1KM) in order to validate the reliability of the numerical approximation of the theoretical bound.

  • Appendix A.3 adds a DROID dataset analysis section to show how our bound behaves on a real-world frame. Thanks to reviewer 2SkA for the suggestion to add analysis for alternative robot manipulation datasets.

  • We specify the KDE settings and provide pseudocode for the reported 82% action retention estimate in Appendix A.4. Thanks to reviewer t1KM for this suggestion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.