acceptodds
Under review as a conference paper at ICLR 2027

Uncertainty Quantification for Flow-Based Generalist Robot Policies

Abstract

Generalist robot policies, such as vision-language-action models (VLAs) and world-action models (WAMs), combine powerful pretrained backbones with expressive generative action heads trained via flow matching on large-scale robotic datasets. Despite their strong empirical performance in robotic manipulation, these policies lack mechanisms to quantify confidence in their predictions and to detect when their actions may be unreliable. This presents a critical limitation for real-world deployment in non-stationary environments, where models inevitably encounter scenarios outside their pretraining distribution and may fail without warning. To address this, we derive an efficient method to quantify epistemic uncertainty in flow-matching models by leveraging velocity-field disagreement (VFD) across a small ensemble. We successfully use this uncertainty estimate for detecting failures during deployment and active fine-tuning of flow-based generalist policies. For the latter, we propose SAVE, a simple yet effective method for uncertainty-guided active multitask fine-tuning that reduces the number of costly expert demonstrations required to adapt generalist policies to new tasks. We conduct experiments in simulation and the real world, across VLAs and a WAM. VFD yields better-calibrated uncertainty estimates predictive of downstream performance and detects failures with 8 pp higher overall accuracy than existing methods. Across three real-world tasks, SAVE improves final average success from 39% to 47% with a fixed demonstration budget. Our results show that measuring epistemic uncertainty with VFD enhances both failure awareness and adaptation of generalist robot policies.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Uncertainty Quantification, Active Learning, Failure Detection

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_tnbrOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VFD, an epistemic uncertainty estimation method for flow-matching-based generalist robot policies. It approximates the pairwise KL divergence between ensemble members via their weighted velocity-field disagreement along sampling trajectories, and applies it to runtime failure detection and uncertainty-guided active multitask fine-tuning (SAVE). Across three experiment types (calibration, active fine-tuning, failure detection) in simulation and the real world, it outperforms existing methods.

Critical strengths
  1. Important problem: the lack of uncertainty quantification in flow-matching generalist policies is a real gap for deployment safety; both failure detection and active fine-tuning are practically valuable.
  2. Clear derivation: from Bayesian epistemic uncertainty to a velocity-field expression of KL (Theorem 4.1) to a lightweight estimator, without modifying the training procedure.
  3. Comprehensive and internally controlled experiments: three policy scales with full baseline coverage per task, ranking first throughout; statistically significant on Push-T with 10 seeds (Wilcoxon); code released.
Critical weaknesses
  1. Application-level novelty: the velocity-field KL expression is a new increment, but the ensemble-disagreement framework itself is a general method; the paper contains no design specific to robot policies.
  2. Limited practicality and high ensemble cost: M=2 members are fine-tuned from the same base on shuffled versions of the same data, offering limited diversity; there is no direct verification separating epistemic from aleatoric uncertainty. Active fine-tuning requires maintaining M members across rounds and evaluating uncertainty for every candidate state, with cost scaling linearly with model size — yet the paper reports only inference-side overhead.
  3. Coarse-grained calibration evaluation: the task-level Spearman correlation is computed over only ~10 points and 3 seeds; the 0.05–0.17 margins between methods carry no significance test, and VFD's 0.44±0.17 on FastWAM is noisy.
  4. Non-standard protocol without an anchor: for LIBERO-Long, Dpre mixes all 30 tasks from LIBERO-Goal/Spatial/Object with demonstrations from 3 LIBERO-Long tasks, followed by active fine-tuning under a budget of 15 rounds × 5 demonstrations (75 in total). This protocol is not comparable to standard LIBERO training (full suites, hundreds of demonstrations per task), and the paper does not report the models' success rates under the standard protocol, leaving absolute performance unanchored.
Questions for authors
  1. What design choices are specific to robot policies, and in which scenarios does the method shine?
  2. How do you trade off the cost and diversity of M? Can the training-free alternative be validated on the active fine-tuning axis?
  3. Can you provide significance tests for the margins in Table 1?
  4. Can you report baseline success rates under the standard protocol?
6 comments
AuthorsOpenReview ↗

Response to Reviewer tnbr (1)

Thank you for your constructive feedback, which we believe has significantly strengthened our paper. We have uploaded a revision with the changes highlighted in blue and have provided detailed responses to your comments below.

W1/Q1: Novelty and robot-specific design.

Theorem 4.1 is a novel, general formulation for flow-matching models (e.g., image/video generation) that holds beyond policies, and we believe that applying our proposed VFD score to flow-matching models in other domains represents an interesting avenue for future work. However, several design choices in our work are motivated by the specific challenges of robot learning:

  • Robotics data is much scarcer than images and videos, so even "generalist" robot policies are much more prone to high epistemic uncertainty (lack of data) than other types of generative models, such as image- or video-generation models.
  • Since robot data is expensive to obtain, our method for iterative active fine-tuning, SAVE, aims to minimize data acquisition effort by guiding data collection using uncertainty estimates, as opposed to more expensive signals like failure rates (see our response to W3 for Reviewer FffU).

SAVE w/ VFD particularly shines in active fine-tuning scenarios with very diverse possible initial conditions (e.g., object layouts), such as in our real-world experiments. There, SAVE w/ VFD achieves a final success rate of 48±448 \pm 4 and an AULC of 36±236 \pm 2, indicating that guiding data acquisition based on the model's current epistemic uncertainty is significantly more efficient than collecting data uniformly at random (34±734 \pm 7 and 25±125 \pm 1).

W2/Q2: Ensemble diversity and cost.

Epistemic vs. aleatoric uncertainty: To directly verify that VFD separates the two, we have conducted a new targeted experiment on our toy example (Appendix C.1, Figure 10), in which an ensemble of M=5M=5 flow-matching models learns a 2D conditional Gaussian mixture. To vary the aleatoric uncertainty, we have made the standard deviation of the mixture components depend on the input. Over a 50×5050\times50 grid of inputs and five independently trained ensembles, VFD is strongly correlated with the epistemic uncertainty (Spearman ρ=0.84±0.02\rho=0.84\pm0.02), but not with the aleatoric uncertainty (ρ=0.00±0.08\rho=0.00\pm0.08), whereas the Action-L2 baseline mainly follows the aleatoric uncertainty (ρ=0.65±0.02\rho=0.65\pm0.02). These results are in line with our theoretical derivation, which establishes VFD as a method for estimating specifically the epistemic component of model uncertainty.

Ensemble cost: Excluding parallelization, the cost of training an ensemble scales linearly with the ensemble size MM. We provide details about the computational cost of iterative ensemble fine-tuning in Appendix B.1. However, we find that increasing the ensemble size beyond two yields only minor (calibration, Figure 11) to modest (failure detection, Table 4) performance gains in our active fine-tuning experiments. We argue that for active fine-tuning, the additional computation of training a second ensemble member amortizes particularly well in applications where collecting expert demonstrations is very expensive, such as surgery or care. Moreover, we find that even without training an ensemble, VFD can achieve strong performance. For example, as shown in Table 4, perturbing the weights of the policy to obtain a second model and reducing the computational overhead to a minimum (using only a single ODE timestep and batch size 1 for VFD computation) reduces failure-detection accuracy in the real world by only 55 pp, from 0.730.73 to 0.680.68.

Ensemble diversity: In our experiments, random shuffling of the mid-training data is sufficient to obtain an ensemble that, with VFD as the uncertainty signal, is well-calibrated at a negative Spearman rank correlation −ρ-\rho of 0.480.48 to 0.720.72 (see Table 1). As a potential way to further improve the diversity between the models, we have mid-trained an ensemble by randomly splitting DpreD_\mathrm{pre} into two disjoint halves (subsampling). This additional injection of diversity improves calibration for SmolVLA only slightly from −ρ=0.72±0.09-\rho=0.72\pm0.09 to −ρ=0.74±0.07-\rho=0.74\pm0.07 (Appendix C.2.4). This ablation supports our claim that our ensemble-creation strategy already yields policy ensembles sufficiently diverse across the scenarios we consider.

Ensemble-free active fine-tuning: We have evaluated SAVE w/ VFD without an ensemble, instead perturbing weights with layer-normalized Gaussian noise, for iterative active fine-tuning on SmolVLA. Despite its simplicity, this approach achieves AULC and Last-3 success rates of 47.8±5.947.8 \pm 5.9 and 57.2±4.857.2 \pm 4.8, tied with AMF (AULC: 47.3±3.247.3\pm3.2, Last-3 success rate: 57.4±5.257.4\pm5.2) and above Random (AULC: 41.8±3.141.8\pm3.1, Last-3 success rate: 52.6±4.652.6\pm4.6), the two ensemble-free baselines.

AuthorsOpenReview ↗

Response to Reviewer tnbr (2)

W3/Q3: Significance of calibration results.

We have extended the calibration analysis in Table 1 from three to five seeds, and the updated results are given in the table below.

PolicyAction-L2ACEDECUGUEntropyPerplexityVFD (ours)
SmolVLA0.43 ± 0.120.40 ± 0.140.31 ± 0.110.65 ± 0.060.08 ± 0.120.13 ± 0.130.72 ± 0.09
X-VLA0.29 ± 0.210.10 ± 0.070.03 ± 0.060.44 ± 0.07N/AN/A0.48 ± 0.06
FastWAM-0.07 ± 0.08-0.30 ± 0.090.03 ± 0.060.23 ± 0.08N/AN/A0.49 ± 0.17

VFD provides superior calibration, with gains ranging from 0.04 to 0.26 relative to the strongest baseline, GU. We have also analyzed the statistical significance of the gains and calculated one-sided Wilcoxon signed-rank p-values for each VFD-baseline comparison pair. The results are shown in the table below, with each cell indicating the number of pairs in which VFD beats the baseline and its corresponding p-value.

VFD vs.SmolVLAX-VLAFastWAMAll 15 pairs together
Action-L25/5, p=0.031p = 0.0314/5, p=0.094p = 0.0945/5, p=0.031p = 0.03114/15, p=9×10−5p = 9 \times 10^{-5}
ACE5/5, p=0.031p = 0.0315/5, p=0.031p = 0.0315/5, p=0.031p = 0.03115/15, p=3×10−5p = 3 \times 10^{-5}
DECU5/5, p=0.031p = 0.0315/5, p=0.031p = 0.0315/5, p=0.031p = 0.03115/15, p=3×10−5p = 3 \times 10^{-5}
GU5/5, p=0.031p = 0.0313/5, p=0.16p = 0.165/5, p=0.031p = 0.03113/15, p=6×10−4p = 6 \times 10^{-4}
Entropy / Perplexity5/5, p=0.031p = 0.031N/AN/A—

The gains of VFD reach p<0.05p<0.05 in 11 out of 13 per-model comparisons. When considering all three policies together, the gains of VFD over the baselines have p-values ranging from 0.000030.00003 to 0.00060.0006. This statistical analysis highlights that the improved calibration of VFD is consistent and statistically significant across different generalist policies. We have added this discussion to Appendix C.2.1.

W4/Q4: Non-standard LIBERO protocol.

The standard LIBERO protocol is to train on all available data from a single suite (Goal/Spatial/Object/Long) or all suites at once. In contrast, we consider the case that data is not yet available but needs to be collected, and our objective is to achieve maximum performance under a limited demonstration budget. To evaluate the impact of iterative fine-tuning (as opposed to a single fine-tuning run), we have conducted an additional experiment, in which the policies are trained only once on 75 episodes, using the cumulative number of gradient steps of all iterative fine-tuning rounds. This setting corresponds to the common setting in which data across tasks are collected only once, uniformly at random, without uncertainty guidance. This approach yields a final success rate on LIBERO-Long of 58.1±1.858.1 \pm 1.8 for SmolVLA (SAVE w/ VFD: 63.0±2.763.0 \pm 2.7) and 51.3±1.551.3 \pm 1.5 for X-VLA (SAVE w/ VFD: 54.3±3.754.3 \pm 3.7). This experiment demonstrates that, given the same data budget, iterative active data acquisition guided by VFD uncertainty can achieve higher absolute performance than common one-shot fine-tuning. We have added this discussion as a new Section C.7 to the Appendix.

Thank you for the detailed review and the positive evaluation of our work. We believe that integrating the suggested experiments has strengthened our work, and we hope it has addressed your concerns. Please do not hesitate to ask any further questions.

I thank the authors for their response. The added experiments are solid, and my assessment has changed considerably.

W2 and W3 are withdrawn. The synthetic experiment separates epistemic from aleatoric uncertainty; the disjoint-subset ablation (0.72→0.74) shows random shuffling already suffices; computational cost is now reported; and the initial-state calibration in C.2.5 (0.48 vs 0.42, 10/10, p=0.001) is more convincing than the task-level analysis.

W1 is downgraded. SAVE does have domain specificity (task-level aggregation, temperature-based sampling, selection over candidate initial states), so I withdraw "no robot-specific design" and reframe it as "the contribution's main body is a general estimator, with SAVE as its domain instantiation". It no longer counts as a major deduction.

Three points remain.

First, the calibration significance is constrained by design. With n=5 the smallest attainable one-sided Wilcoxon p is 0.031, and 5/5 lands exactly on that floor, so the value reflects sign agreement only, not effect size; the smallest p among 13 comparisons is 0.031, above the Holm threshold 0.05/13 ≈ 0.0038, so the family is unlikely to survive multiple-comparison correction. X-VLA is also non-significant against the strongest baseline GU (3/5, p=0.16, 0.48 vs 0.44). The pooled p=6×10⁻⁴ over 15 pairs is weakened by differing effect sizes across models (+0.04 to +0.26) and by seeds nested within models. Reporting per-model effect-size intervals, or a mixed-effects model with model as a random effect, would be more robust.

Second, the ensemble-free variant should not be framed positively. In C.5 it reaches AULC 47.8 and Last-3 57.2, against 52.5/63.0 for the full model and 47.3/57.4 for AMF — it degrades to the AMF level, consistent with the +5 pp from M=2→4. The gain depends on training one additional member, which reads better as a boundary condition.

Third, absolute performance still lacks a scale reference. I accept the C.7 comparison, and I agree that full-data training is not an appropriate baseline here; but C.7 answers "is this only more gradient steps", not "where does 63.0 sit". A single cited value for SmolVLA on full LIBERO-Long would suffice — no new run needed. I would also suggest cross-checking C.7 against C.8: random-selected one-shot 58.1, VFD-selected one-shot 70.6, VFD-selected iterative 67.1, i.e. selection ≈ +12.5 and iteration ≈ −3.5. If that reading is right, the effective ingredient in this section is selection rather than iteration, and saying so explicitly would make the method's operating conditions clearer.

Reviewer_FffUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a method for estimating epistemic uncertainty in flow-based generalist robot policies, called VFD. It measures disagreement between the velocity fields of ensemble members. Building on VFD, the authors propose SAVE, an uncertainty-guided active fine-tuning framework that prioritizes tasks and initial states for expert demonstration collection. VFD is also applied to runtime failure detection. Experiments across simulated and real-world robotic tasks, involving both VLA and WAM architectures, demonstrate improved uncertainty-performance correlation, more data-efficient fine-tuning, and more accurate failure detection compared to existing baselines.

Critical strengths
  1. The paper addresses uncertainty quantification for flow-based generalist robot policies, an important problem for reliable robotic deployment and adaptation.
  2. The proposed VFD provides a theoretically motivated uncertainty estimator based on velocity-field disagreement, with a practical implementation using a small ensemble.
  3. Experiments evaluate multiple VLA and WAM architectures across simulation and real-world tasks, demonstrating the usefulness of VFD for both active fine-tuning and runtime failure detection.
Critical weaknesses
  1. The ensemble is constructed by fine-tuning the same pretrained checkpoint on the same dataset with random shuffling. It is unclear whether such an ensemble can reliably capture epistemic uncertainty, particularly when all members share similar failure modes and produce consistently incorrect predictions.
  2. The paper primarily evaluates uncertainty calibration through its correlation with task success rates. However, task difficulty and failure probability are not necessarily equivalent to epistemic uncertainty. More controlled experiments distinguishing epistemic from aleatoric uncertainty would better support the paper's central claim.
  3. SAVE prioritizes data collection based on epistemic uncertainty, but it is unclear whether this is more effective than simpler signals, such as task failure rates or training data coverage. In particular, task- or state-level coverage of existing demonstrations could provide a direct and inexpensive signal for identifying underrepresented scenarios.
  4. The performance of SAVE appears sensitive to the task-sampling temperature, which is selected based on downstream success rates in the experiments. However, such performance feedback may not be available when adapting to new tasks, where uncertainty-guided acquisition is intended to operate without additional environment interaction.
Questions for authors

See weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer FffU (1)

We thank you for highlighting the relevance of our paper's topic, its theoretical contributions, and broad evaluation, as well as for your constructive feedback. We have incorporated your suggestions into the revised manuscript and believe they have significantly strengthened our paper. We provide detailed answers to your feedback below.

W1: Ensemble diversity for capturing epistemic uncertainty.

Our main methodological contribution is a novel method for quantifying epistemic uncertainty in flow-based generalist policies when different model weights sampled from the posterior are given (e.g., ensembling, Laplace approximation, or perturbations). As a secondary contribution, to obtain such models in practice, we additionally propose a simple strategy for turning a pretrained policy into an ensemble. In our experiments, this strategy yields an ensemble from which VFD can capture epistemic uncertainty well, as highlighted by its superior calibration (Table 1, Figure 4), active fine-tuning (Table 2, Figure 5) and failure-detection performance (Table 3). For example, the superior active fine-tuning performance supports the fact that VFD captures the uncertainty component that can be reduced by collecting more data, i.e., epistemic uncertainty.

However, we would like to point out that we do not claim that our ensemble creation via random shuffling is the best possible strategy. In fact, more sophisticated ensembling techniques (e.g., bootstrapping) might yield even better results. To investigate this further, we have mid-trained an ensemble on different subsets of DpreD_\mathrm{pre} and compared its calibration during active fine-tuning with the shuffling-only approach used in our main experiments.

Specifically, we randomly split DpreD_\mathrm{pre} into two disjoint halves and mid-train each ensemble member on one of them (i.e., subsampling), such that the members never see the same mid-training data. On LIBERO-Long with SmolVLA, this additional source of diversity improves the calibration of VFD only slightly, from −ρ=0.73±0.09-\rho=0.73\pm0.09 to −ρ=0.74±0.07-\rho=0.74\pm0.07 (Appendix C.2.4). We conclude that random shuffling already yields sufficiently diverse ensemble members for VFD to capture epistemic uncertainty in our setting, while being simpler and using all of the available mid-training data for each member.

Regarding shared failure modes, we agree that as a disagreement-based estimator, VFD would struggle to flag situations in which all ensemble members confidently agree on the same incorrect action. However, our results indicate that such shared failure modes are not dominant in practice: VFD is predictive of task failure both before execution (calibration, Table 1) and during deployment, where it detects failures with the highest average balanced accuracy (0.720.72) and AUROC (0.820.82) among all methods, including the Action-L2 and GU baselines that use the same ensemble (updated Table 3). If the ensemble members systematically shared failure modes, VFD would miss a large fraction of failures, which is inconsistent with these results.

AuthorsOpenReview ↗

Response to Reviewer FffU (2)

W2: Separating epistemic from aleatoric uncertainty.

At a high level, the central objective of our work is to enable robots controlled by generalist policies to recognize when they lack sufficient knowledge to solve a task. We agree that failure probability is not equivalent to epistemic uncertainty, but we find the two to be highly correlated. This holds both at the initial state, as shown by the calibration analysis in Table 1 (task-success vs. VFD score negative Spearman −ρ-\rho between 0.480.48 and 0.730.73) and during deployment, as shown by the failure detection results in Table 3 (overall accuracy of 0.720.72). We believe these results show that epistemic uncertainty is a powerful proxy for assessing a policy's ability to solve a task from the current state. In contrast, aleatoric uncertainty quantifies ambiguity in the expert policy and does not generally reflect task difficulty.

For a clean evaluation of whether VFD can distinguish epistemic from aleatoric uncertainty, we have conducted a new targeted experiment on our toy example setup that learns a 2D conditional distribution q(x∣y)q(\boldsymbol{x}\mid\boldsymbol{y}) with an ensemble of M=5M=5 flow-matching models (Appendix C.1, Figure 10). To vary the aleatoric uncertainty, we have made the standard deviation of the two components of the Gaussian mixture depend on the input. Over a 50×5050\times50 grid of inputs and five independently trained ensembles, VFD is strongly correlated with the epistemic uncertainty (Spearman rank correlation of ρ=0.84±0.02\rho =0.84 \pm 0.02), but not with the aleatoric uncertainty (ρ=0.00±0.08\rho=0.00\pm0.08). In contrast, the Action-L2 baseline, which measures disagreement in action space, mainly follows the aleatoric uncertainty (ρ=0.65±0.02\rho=0.65\pm0.02) and is even negatively correlated with epistemic uncertainty (ρ=−0.45±0.02\rho=-0.45\pm0.02). This experiment demonstrates that VFD measures specifically the epistemic (reducible) component of predictive uncertainty, as established in our theoretical derivation in Section 4. We have expanded Section C.1 of the Appendix to include this discussion and added a condensed version of the toy example to Section 4.

W3: Simpler acquisition signals (failure rates, coverage).

When a robot faces a set of new tasks in the real world, obtaining reliable estimates of their failure rates requires many rollouts, which are expensive. Moreover, if a task is highly unfamiliar to the policy, collecting many failure rollouts may even be dangerous. For these reasons, one of our key motivations is to enable generalist policies to estimate a priori whether they can successfully solve a task. Nonetheless, we have evaluated a failure-rate baseline in our iterative active fine-tuning setup on LIBERO-Long, which has privileged information about the per-task failure rates at each stage and samples tasks weighted by (1-Success Rate). This baseline achieves a Last-3 success rate of 61.9±3.661.9 \pm 3.6 (SAVE w/ VFD: 63.0±2.763.0 \pm 2.7) with SmolVLA and 49.2±2.749.2 \pm 2.7 (SAVE w/ VFD: 52.9±2.952.9 \pm 2.9) with X-VLA. This experiment highlights that per-task failure rates, i.e., privileged information that is expensive to obtain in the real world, do not provide a more effective signal than epistemic uncertainty.

The Diversity baseline in our experiments selects new expert demonstrations to greedily maximize state-level data coverage, i.e., without considering the epistemic uncertainty of the policy being trained with that data. Compared to SAVE w/ VFD, this method achieves a lower final success rate (2828 vs. 4848) and AULC (2222 vs. 3636) in the real-world experiments, and the Last-3 success rates are 3.9−10.93.9-10.9 pp lower in LIBERO-Long and 8.18.1 pp lower in Push-T. For task-level coverage, we have additionally evaluated a count-balancing baseline that allocates each demonstration to the task with the fewest demonstrations so far, which achieves Last-3 success rates of only 55.8±3.055.8 \pm 3.0 with SmolVLA and 51.0±1.551.0 \pm 1.5 with X-VLA on LIBERO-Long. These results demonstrate that uncertainty guidance is a more powerful approach for active fine-tuning than data coverage alone.

AuthorsOpenReview ↗

Response to Reviewer FffU (3)

W4: Sensitivity to the task-sampling temperature.

We appreciate your comment about the selection of the task-sampling temperature τ\tau. We have tuned this hyperparameter on SmolVLA, using the same grid as for the uncertainty-guided baselines (SAVE w/ Action-L2, SAVE w/ GU) and SAVE w/ VFD. For X-VLA and FastWAM, which are more expensive to train than SmolVLA, a full grid search over τ\tau is too computationally expensive for us to run. For those models and in the real-robot experiments, each uncertainty-based method uses the hyperparameter that performed best in the tuning experiment: temperatures τ=1.5\tau=1.5 for SAVE w/ Action-L2, τ=1\tau=1 for SAVE w/ GU, and τ=2.5\tau=2.5 for SAVE w/ VFD, and noise σ=0.01\sigma=0.01 for AMF. Based on our results, we recommend τ∈[2,2.5]\tau\in[2,2.5] for SAVE w/ VFD. In the single-task Push-T experiment, the task sampling temperature is not used.

To avoid selecting τ\tau based on downstream success rates in the SmolVLA evaluation, we have rerun the evaluation with three new independent seeds and updated the two corresponding columns in Table 2. The new values are also given in the following table:

MethodAULC [%]Last-3 SR [%]
Random41.8 ± 3.152.6 ± 4.6
Diversity48.1 ± 1.155.4 ± 2.1
AMF47.3 ± 3.257.4 ± 5.2
SAVE w/ GU50.0 ± 3.559.3 ± 5.1
SAVE w/ Action-L248.5 ± 0.458.4 ± 1.9
SAVE w/ VFD52.5 ± 1.963.0 ± 2.7

We observe that on SmolVLA, SAVE w/ VFD also performs best on seeds that are different from those used to tune τ\tau. This is similar to the X-VLA and FastWAM active fine-tuning results, which remain unchanged.

Thank you for the clear and actionable review. We would like to ask whether our new empirical results and responses fully address your concerns, and we are happy to continue this discussion during this rebuttal phase.

Dear Reviewer,

Thanks again for the helpful feedback. Since the end of the rebuttal phase on November 18th is approaching, we would like to ask whether we were able to address your comments with our rebuttal. In case any questions remain, we would be happy to answer them. Otherwise we would be interested in any feedback you might have on our additional experiments, and in whether they change your evaluation of the paper.

Reviewer_8TYaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes velocity-field disagreement (VFD), an epistemic-uncertainty score for flow-matching policies. It bounds the mutual-information form of epistemic uncertainty by the average pairwise KL divergence between ensemble members (Eq. 4c). Theorem 4.1 shows that, for flow-matching models with Gaussian OT paths, the KL divergence between two models equals an integral of their squared velocity-field difference, weighted by s/(1−s), along one model's probability path. VFD is an Euler discretisation of that integral and uses a two-member ensemble fine-tuned from one pretrained checkpoint. The paper evaluates VFD for predicting per-task success from initial observations ("calibration"; LIBERO-Long with SmolVLA, X-VLA and FastWAM); active multitask fine-tuning through SAVE; and runtime failure detection. SAVE samples tasks with probability proportional to (task-mean VFD)^τ and queries the most uncertain initial state within the sampled task, with tests on Push-T, LIBERO-Long and three real-robot tasks. Runtime failure detection uses conformal thresholds set from successful rollouts and is tested on Push-T, LIBERO-Plus and 152 real-robot rollouts. Reported results include higher rank correlation with task success than six other uncertainty estimates, higher AULC and Last-3 success than random, diversity, AMF and SAVE run with other scores, and 72% vs 64% average balanced accuracy in failure detection.

Critical strengths
  1. A cheap, principled and architecture-agnostic epistemic score for flow-based policies. Theorem 4.1 turns ensemble disagreement into a quantity with an information-theoretic meaning. That quantity can be computed alongside action sampling. Cost: +32 to +89 ms per action-generation step for SmolVLA with two members, or 1.19–1.53× the base policy (Table 4). It applies unchanged to VLAs and to a 6B world-action model. Why it matters: flow-based generalist policies have no native uncertainty, and one score here serves both data collection and runtime monitoring.

  2. A broad and careful evaluation. Coverage: three applications, three policy classes (SmolVLA 550M, X-VLA 900M, FastWAM 6B), and real-robot experiments for both active fine-tuning and failure detection. Ablations: ensemble size, a Laplace approximation, weight perturbation and the choice of flow times. Code is released (anonymous repository), and a paired significance test is reported on Push-T (Table 9; Wilcoxon p = 0.004 for Last-3 success). The ordering VFD > GU > Action-L2 holds both for calibration (Table 1) and for active fine-tuning (Table 2). This supports the thesis that better-ranked uncertainty leads to better data selection.

Critical weaknesses
  1. The gains over the strongest baselines are not yet established, and parts of the comparison favour SAVE. Small margins, three seeds. Last-3 success on LIBERO-Long against the strongest baseline: SmolVLA: 66.1±2.8 vs 61.0±2.9; X-VLA: 52.9±2.9 vs 51.0±1.3; FastWAM: 74.8±3.3 vs 71.6±5.2. Only Push-T has a significance test. Temperature chosen on the evaluation runs. τ was selected by the final success of the same runs (Table 11, marked "*"), and VFD's chosen τ = 2.5 is the edge of the grid. VFD's lead over GU depends strongly on τ. At round 15, VFD minus GU is +4.4, −4.8, +0.9, +3.6 and +7.0 pp at τ = 0, 1, 1.5, 2 and 2.5 (Table 11; differences derived). VFD leads at four of the five values, but the size of the reported margin comes from tuning on the evaluation runs. Random and Diversity have nothing to tune. Missing count-balancing baseline. In LIBERO-Long, three of the ten target tasks are included in D_pre (App. B.3.1). Most of SAVE's gain comes from choosing which task to query (App. C.6, Fig. 13). A simple count-balancing baseline is therefore needed: allocate each demonstration to the task with the fewest demonstrations so far, pretraining data included. In fairness, this concerns LIBERO-Long. On the real robot all three tasks are new, so Random already behaves like count balancing there. The real-robot headline's uncertainty is not reported. The headline is 47% vs 39%, where 39% is Random. It is evaluated with 30 rollouts per checkpoint (Table 6). Figs. 4 and 12 show neither error bars nor a seed count, although Section 6.1 says experiments use three seeds. Random is above SAVE on Stack at 120 demonstrations (Fig. 12). If this is a single run with about 10 rollouts per task, the standard error of an 8 pp difference would be roughly 10–13 pp (derived, binomial upper bound). Why this matters: the claim that SAVE beats the strongest baselines, and the abstract's real-world gain, both rest on these comparisons.

  2. The theory supports the estimator less directly than claimed, and the failure-detection comparison does not isolate VFD's key ingredient. The theorem is exact only for ideal fields. Theorem 4.1 relies on Lemma A.1, ∇log p_s = (s·u_s − x)/(1−s). This holds for the exact marginal velocity of a Gaussian OT path. A trained network's ODE does not in general induce marginals of that form. For learned fields, Eqs. (6)–(7) therefore replace the score difference in Eq. (36) with a scaled velocity difference. The error of that replacement is not analysed. For score-based ODE models, first-order score matching is known to leave a gap to the ODE likelihood (Lu et al., ICML 2022). So the "approximate upper bound" of Eq. (4c) does not transfer to VFD as computed. Two fine-tunes from one checkpoint are also a weak stand-in for posterior samples. Fig. 8 compares VFD with the KL to the ground truth, not with the KL between the two learned models. The failure-detection comparison leaves out the same-ensemble baselines. Table 3 compares VFD with OOD detectors and single-model scores. It omits the baselines that share VFD's ensemble and appear in Table 1 (Action-L2, GU, DECU). Those are the comparisons that would separate velocity-field disagreement from action-space disagreement at equal cost. VFD's advantage is in balanced accuracy at one conformal threshold. On early detection it ties on average (TWA 0.60 vs 0.59 for logpZO). In the real world it trails: TWA 0.58 vs 0.64 for logpZO and 0.61 for RND-OE, and detection time 0.51 vs 0.14 (Table 14). Threshold-free metrics such as AUROC are not reported. Why this matters: "mathematically grounded" and "detects failures more accurately" are two of the paper's three headline claims.

Questions for authors
  1. Add the count-balancing baseline from Weakness 1 on LIBERO-Long, with the same budget: allocate demonstrations to tasks with the fewest demonstrations in D_pre ∪ D_new, and pick initial states uniformly. Does SAVE with VFD still lead?

  2. Report results with τ fixed in advance (for example τ = 1 for every score) or chosen on separate seeds. Was τ re-tuned for Push-T, X-VLA and FastWAM? Table 11 covers SmolVLA only.

  3. For the real-robot experiment: Do Figs. 4 and 12 average three seeds, as Section 6.1 states for all experiments? If so, please add the spread. How many evaluation rollouts per task per checkpoint were used? Were "expert queries" retrieved from a pre-collected pool? App. B.4 says all strategies "retrieve demonstrations from the same expert dataset". If a configuration is selected in several rounds, is the same trajectory added again, and does it count against the budget?

  4. Does VFD still beat GU in calibration without the three D_pre tasks, or at the level of initial states within a task? That would show it measures more than the seen/unseen split.

  5. In the toy example or on Push-T, how close is VFD to the pairwise KL between the two learned models computed from exact likelihoods? The low dimension makes the divergence term tractable. How sensitive is VFD to the step size δs?

  6. Failure detection: please add Action-L2 and GU on the same ensemble, plus AUROC, for Table 3.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 8TYa (1)

We are grateful for your detailed feedback, which has been very helpful in improving our work. We have conducted new experiments and added more detailed discussions to the paper. In the uploaded revision, all changes with respect to the original manuscript are highlighted in blue. We summarize the main changes below.

W1/Q2: Task-sampling temperature selection.

We have tuned the task-sampling temperature τ\tau on SmolVLA in the LIBERO-Long experiments, with SAVE w/ VFD and the two baselines SAVE w/ Action-L2 and SAVE w/ GU all using the same grid τ∈{0,1,1.5,2,2.5,3}\tau \in \{0,1,1.5,2,2.5,3\} for fairness. We have added a note to Section 6.3 to clarify this. Since τ=3\tau=3 did not perform best for any method, we did not include it in Table 13. We have added these rows in the revised manuscript to make clear that τ=2.5\tau=2.5 is not the edge of the grid.

To avoid tuning on the runs we evaluate, we have rerun the SmolVLA active fine-tuning evaluation on three new independent seeds. This slightly changed the values in two columns of Table 2, which are now as follows:

MethodAULC [%]Last-3 SR [%]
Random41.8 ± 3.152.6 ± 4.6
Diversity48.1 ± 1.155.4 ± 2.1
AMF47.3 ± 3.257.4 ± 5.2
SAVE w/ GU50.0 ± 3.559.3 ± 5.1
SAVE w/ Action-L248.5 ± 0.458.4 ± 1.9
SAVE w/ VFD52.5 ± 1.963.0 ± 2.7

We observe that these results are consistent with previous ones: SAVE w/ VFD performs best, and SAVE w/ GU is second. For X-VLA and FastWAM, which are significantly larger models than SmolVLA, running the full grid over all values of τ\tau for iterative active fine-tuning would be too computationally expensive. Therefore, for these policies, we have fixed τ\tau in advance to the value that performed best in the SmolVLA tuning experiment, which we consider to be the fairest approach. Since Push-T is a single task, SAVE selects data only at the episode level and does not require a task selection parameter in those experiments.

W1: Statistical significance on LIBERO-Long.

We agree that the margins for the individual policies are small relative to the spread over three seeds. As in our calibration analysis (Appendix C.2.1 and response to Reviewer tnbr W3/Q3), we have pooled the nine policy-seed pairs of SmolVLA, X-VLA, and FastWAM and applied a one-sided Wilcoxon signed-rank test to the paired differences between SAVE w/ VFD and each baseline. For both metrics, the table below gives the number of seeds per policy on which SAVE w/ VFD outperforms the baseline (SmolVLA/X-VLA/FastWAM, each out of three), the number of wins and the p-value over all nine pairs:

SAVE w/ VFD vs.AULCPooled AULCLast-3 SRPooled Last-3 SR
Random3/3/39/9, p=0.002p = 0.0023/3/28/9, p=0.004p = 0.004
Diversity3/2/38/9, p=0.004p = 0.0043/2/38/9, p=0.004p = 0.004
AMF3/3/39/9, p=0.002p = 0.0022/3/38/9, p=0.010p = 0.010
SAVE w/ Action-L23/3/28/9, p=0.006p = 0.0063/2/38/9, p=0.006p = 0.006
SAVE w/ GU3/2/27/9, p=0.037p = 0.0373/2/38/9, p=0.006p = 0.006

SAVE w/ VFD outperforms every baseline in at least seven of the nine pairs on both metrics. All ten comparisons are significant at the 5% level and remain so after a Holm correction for multiple comparisons. We have added this analysis to Appendix C.6 (Table 14) and refer to it in Section 6.3.

W1/Q1: Count-balancing baseline.

Thank you for suggesting a count-balancing baseline. We have evaluated it in our LIBERO-Long experimental setting, where it achieves a Last-3 success rate of 55.8±3.055.8\pm3.0 on SmolVLA (SAVE w/ VFD: 63.0±2.763.0\pm2.7) and 51.0±1.551.0\pm1.5 on X-VLA (SAVE w/ VFD: 52.9±2.952.9\pm2.9). We note, though, that the pre-training task distribution is unknown for most state-of-the-art generalist policies, making the count-balancing method a privileged baseline. As you said, in real-world experiments, the Random baseline already corresponds to count balancing. Here, its final success rate is 34±734\pm7 compared to 48±448\pm4 for SAVE w/ VFD, and its AULC is 25±125\pm1 compared to 36±236\pm2 for SAVE w/ VFD. In summary, we conclude from the simulated and real-world experiments that uncertainty-guided active data acquisition with VFD outperforms uniform allocation of demonstrations across tasks.

AuthorsOpenReview ↗

Response to Reviewer 8TYa (2)

W1/Q3: Real-robot statistics and expert queries.

We appreciate your comment on the statistical significance of the performance gains in active fine-tuning. We have extended our real-world experiment by two additional independently trained seeds and updated Figures 5 and 14. For each seed, we roll out the policy 10 times per task, per method, per round (3, 6, 9, 12, 15), yielding a total of 2,250 rollouts. The final success rates and the areas under the learning curve (AULC) for all methods across three seeds are given in the table below:

MethodFinal SR [%]AULC [%]
Random34 ± 725 ± 1
Diversity28 ± 722 ± 5
AMF35 ± 427 ± 2
SAVE w/ GU28 ± 526 ± 5
SAVE w/ VFD48 ± 436 ± 2

SAVE w/ VFD achieves a final success rate that is 1313 pp higher and an AULC that is 99 pp higher than the second-best method, AMF. This highlights that in real-world tasks with diverse initial conditions, VFD provides a more informative signal for active data acquisition than existing methods. In the real world, GU is a much weaker signal than VFD for data acquisition. We attribute this to GU's use of a Gaussian approximation to the posterior predictive distribution, which can be highly inaccurate for complex, multimodal distributions of human demonstrations.

Regarding your question about expert queries: we have used the same pre-collected pool for all methods in order to isolate the effect of the data acquisition strategy from the quality of the expert demonstrations (which may vary over time, for example, because an operator gets tired). All methods can potentially select an initial configuration in different rounds. In that case, the corresponding expert demonstration is added again, increasing its relative weight in the data buffer, and we count this against the budget. We allow this to account for the fact that, in principle, active learning methods may repeatedly request very similar data. This could be, for example, due to weak epistemic uncertainty estimates or to a single episode not sufficiently reducing epistemic uncertainty in an otherwise poorly covered area.

W2/Q5: Theorem 4.1 for learned velocity fields.

Thank you for these insightful comments. As stated in Theorem 4.1, it is formulated for exact marginal velocities of Gaussian OT paths, to which learned velocity fields are an approximation. We have added a sentence below Theorem 4.1 to make this clearer. To quantify the quality of the approximation, we have computed the exact pairwise KL divergence between the learned flow-matching models in the experiment shown in Fig. 9, as you suggested. Over a 50×5050\times50 grid of inputs and five independently trained ensembles, the Spearman correlation between VFD (Ns=10N_s=10) and the pairwise KL divergence is very strong at 0.97±0.010.97\pm0.01 for the full ensemble (M=5M=5) and 0.94±0.010.94\pm0.01 for two-member ensembles (M=2M=2), which we also use in our robot experiments. In the same setting, we have also analyzed the sensitivity of VFD to the step size δs\delta s:

NsN_s23457102050
M=5M=50.81 ± 0.050.87 ± 0.020.92 ± 0.020.95 ± 0.010.97 ± 0.010.97 ± 0.010.98 ± 0.010.97 ± 0.01
M=2M=20.71 ± 0.040.78 ± 0.030.84 ± 0.020.88 ± 0.020.92 ± 0.010.94 ± 0.010.94 ± 0.010.94 ± 0.01

For Ns≥7N_s\geq7, VFD and the exact KL divergence rank the inputs consistently with ρ>0.9\rho>0.9, even when using only two-member ensembles as in our robot experiments. We have added these results to Appendix C.1 (Figure 9 and Table 8).

AuthorsOpenReview ↗

Response to Reviewer 8TYa (3)

W2: Weak approximation of the posterior.

We appreciate your comment about the nature of the posterior samples obtained through fine-tuning an ensemble from a single checkpoint. Deep ensembles trained from scratch are an established method to obtain posterior samples [1, 2], but there is typically only a single pretrained checkpoint available for large-scale generalist policies. Therefore, we propose a practical approach to creating an ensemble from it by shuffling data during mid-training. In this way, VFD achieves a negative Spearman correlation coefficient ranging from 0.480.48 to 0.730.73 in the calibration experiments and a failure-detection accuracy of 0.720.72, both of which are superior to other methods using the same ensemble.

However, we note that our ensemble training approach may be further improved in principle. We have investigated a possible strategy for doing so in an additional calibration experiment, in which the two ensemble members are trained on disjoint halves of the diverse mid-training data. This approach, also known as subsampling, slightly improves calibration for SmolVLA from −ρ=0.73±0.09-\rho=0.73\pm0.09 to −ρ=0.74±0.07-\rho=0.74 \pm0.07. Yet the gains are modest, which we believe indicates that our approach already yields policy ensembles strong enough to assess uncertainty quantification capabilities. We have expanded the "Generalist Policy Ensemble'' paragraph in Section 5 to clarify that our approach is a practically feasible way to train a generalist policy ensemble, though even stronger approaches (e.g., subsampling, bagging) may exist. We have also added the subsampling experiment to Appendix C.2.4.

References:

  • [1] Berry et al., Shedding Light on Large Generative Networks: Estimating Epistemic Uncertainty in Diffusion Models, Conference on Uncertainty in Artificial Intelligence (UAI), 2024.
  • [2] M. Jazbec et al., Generative Uncertainty in Diffusion Models, Conference on Uncertainty in Artificial Intelligence (UAI), 2025.

W2/Q6: Failure-detection baselines and AUROC.

Thank you for these suggestions. We have added ensemble-based Action-L2 and GU from the active fine-tuning experiments as baselines to the failure-detection evaluation in Table 3 and included AUROC as an additional metric:

DatasetMetricSTAClogpZORND-OEACEAction-L2GUVFD
Push-TAcc. ↑0.55 ± 0.000.53 ± 0.010.50 ± 0.020.55 ± 0.060.61 ± 0.070.60 ± 0.050.64 ± 0.05
TWA ↑0.51 ± 0.000.51 ± 0.020.50 ± 0.020.51 ± 0.030.54 ± 0.020.52 ± 0.040.56 ± 0.05
AUROC ↑0.63 ± 0.050.62 ± 0.050.55 ± 0.030.66 ± 0.030.75 ± 0.050.72 ± 0.060.75 ± 0.08
LIBERO-PlusAcc. ↑0.70 ± 0.030.68 ± 0.020.67 ± 0.010.75 ± 0.020.71 ± 0.020.78 ± 0.020.79 ± 0.00
TWA ↑0.59 ± 0.020.62 ± 0.010.62 ± 0.010.59 ± 0.020.60 ± 0.010.64 ± 0.020.65 ± 0.00
AUROC ↑0.81 ± 0.020.90 ± 0.010.92 ± 0.010.85 ± 0.020.90 ± 0.010.91 ± 0.010.90 ± 0.01
Real WorldAcc. ↑0.43 ± 0.020.70 ± 0.030.69 ± 0.040.54 ± 0.010.70 ± 0.010.58 ± 0.010.73 ± 0.01
TWA ↑0.42 ± 0.020.64 ± 0.030.61 ± 0.020.50 ± 0.000.57 ± 0.010.53 ± 0.020.58 ± 0.00
AUROC ↑0.35 ± 0.000.80 ± 0.000.80 ± 0.000.50 ± 0.010.73 ± 0.000.67 ± 0.000.80 ± 0.00
AverageAcc. ↑0.56 ± 0.020.64 ± 0.020.62 ± 0.000.61 ± 0.020.67 ± 0.020.65 ± 0.020.72 ± 0.02
TWA ↑0.51 ± 0.010.59 ± 0.020.58 ± 0.000.54 ± 0.020.57 ± 0.020.56 ± 0.020.60 ± 0.02
AUROC ↑0.60 ± 0.010.77 ± 0.020.76 ± 0.020.67 ± 0.010.80 ± 0.020.77 ± 0.020.82 ± 0.03

Compared to the dedicated failure-detection baselines (STAC, logpZO, RND-OE, ACE), VFD increases overall AUROC by 0.05-0.22. Action-L2 and GU perform well particularly on Push-T and LIBERO-Plus, but achieve significantly lower AUROC in the real world. Compared to VFD, GU performs much worse in the real-world experiments. We attribute this to its assumption of a Gaussian posterior, which is inaccurate for the complex, multimodal distributions induced by diverse human demonstrations.

Comments on the paper

1 comment

We would like to thank all reviewers for their constructive feedback. In the following, we summarize the main changes we have made in the revision. They are marked in blue in the manuscript for ease of parsing.

  • We strengthened the statistical evidence for our main results. The calibration analysis now uses five seeds per policy (Table 1) and the real-robot active fine-tuning experiment uses three seeds (Figures 5 and 14), with SAVE w/ VFD reaching a final success rate of 48% compared to 35% for the best baseline. We also added significance tests for calibration and for active fine-tuning on LIBERO-Long (Appendices C.2.1 and C.6), which confirm that the gains of VFD are consistent across policies and seeds. (8TYa, tnbr)
  • We reran the SmolVLA active fine-tuning evaluation on new seeds, separate from those used to tune the task-sampling temperature (Table 2). SAVE w/ VFD remains the best method. (8TYa, FffU)
  • We added two baselines that select tasks using privileged information, namely per-task failure rates and demonstration counts (Appendix C.4). SAVE w/ VFD is at least as effective as both, without requiring additional rollouts or knowledge of the pre-training data. (8TYa, FffU)
  • We extended the toy example (Appendix C.1) and added a condensed version to Section 4. It shows that VFD closely tracks the exact KL divergence between the learned models and that it captures epistemic, but not aleatoric, uncertainty. A new experiment in Push-T shows that VFD is also better calibrated than the strongest baseline at the level of individual initial states (Appendix C.2.5). (8TYa, FffU, tnbr)
  • We investigated our ensemble design further. Training the ensemble members on disjoint data improves calibration only marginally, and SAVE w/ VFD remains competitive with the ensemble-free baselines even when no ensemble is trained (Appendices C.2.4 and C.5). (8TYa, FffU, tnbr)
  • We added the ensemble-based baselines Action-L2 and GU, as well as AUROC, to the failure-detection evaluation in Table 3. VFD achieves the highest average accuracy and AUROC. (8TYa)
  • We added a comparison to one-shot fine-tuning on the same demonstration budget, which SAVE w/ VFD outperforms (Appendix C.7). (tnbr)

We believe that these updates have strengthened our work, and we thank the reviewers for prompting them. We look forward to further discussions in the rebuttal phase.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.