- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes VFD, an epistemic uncertainty estimation method for flow-matching-based generalist robot policies. It approximates the pairwise KL divergence between ensemble members via their weighted velocity-field disagreement along sampling trajectories, and applies it to runtime failure detection and uncertainty-guided active multitask fine-tuning (SAVE). Across three experiment types (calibration, active fine-tuning, failure detection) in simulation and the real world, it outperforms existing methods.
Critical strengths
- Important problem: the lack of uncertainty quantification in flow-matching generalist policies is a real gap for deployment safety; both failure detection and active fine-tuning are practically valuable.
- Clear derivation: from Bayesian epistemic uncertainty to a velocity-field expression of KL (Theorem 4.1) to a lightweight estimator, without modifying the training procedure.
- Comprehensive and internally controlled experiments: three policy scales with full baseline coverage per task, ranking first throughout; statistically significant on Push-T with 10 seeds (Wilcoxon); code released.
Critical weaknesses
- Application-level novelty: the velocity-field KL expression is a new increment, but the ensemble-disagreement framework itself is a general method; the paper contains no design specific to robot policies.
- Limited practicality and high ensemble cost: M=2 members are fine-tuned from the same base on shuffled versions of the same data, offering limited diversity; there is no direct verification separating epistemic from aleatoric uncertainty. Active fine-tuning requires maintaining M members across rounds and evaluating uncertainty for every candidate state, with cost scaling linearly with model size — yet the paper reports only inference-side overhead.
- Coarse-grained calibration evaluation: the task-level Spearman correlation is computed over only ~10 points and 3 seeds; the 0.05–0.17 margins between methods carry no significance test, and VFD's 0.44±0.17 on FastWAM is noisy.
- Non-standard protocol without an anchor: for LIBERO-Long, Dpre mixes all 30 tasks from LIBERO-Goal/Spatial/Object with demonstrations from 3 LIBERO-Long tasks, followed by active fine-tuning under a budget of 15 rounds × 5 demonstrations (75 in total). This protocol is not comparable to standard LIBERO training (full suites, hundreds of demonstrations per task), and the paper does not report the models' success rates under the standard protocol, leaving absolute performance unanchored.
Questions for authors
- What design choices are specific to robot policies, and in which scenarios does the method shine?
- How do you trade off the cost and diversity of M? Can the training-free alternative be validated on the active fine-tuning axis?
- Can you provide significance tests for the margins in Table 1?
- Can you report baseline success rates under the standard protocol?
6 comments
Response to Reviewer tnbr (2)
W3/Q3: Significance of calibration results.
We have extended the calibration analysis in Table 1 from three to five seeds, and the updated results are given in the table below.
| Policy | Action-L2 | ACE | DECU | GU | Entropy | Perplexity | VFD (ours) |
|---|---|---|---|---|---|---|---|
| SmolVLA | 0.43 ± 0.12 | 0.40 ± 0.14 | 0.31 ± 0.11 | 0.65 ± 0.06 | 0.08 ± 0.12 | 0.13 ± 0.13 | 0.72 ± 0.09 |
| X-VLA | 0.29 ± 0.21 | 0.10 ± 0.07 | 0.03 ± 0.06 | 0.44 ± 0.07 | N/A | N/A | 0.48 ± 0.06 |
| FastWAM | -0.07 ± 0.08 | -0.30 ± 0.09 | 0.03 ± 0.06 | 0.23 ± 0.08 | N/A | N/A | 0.49 ± 0.17 |
VFD provides superior calibration, with gains ranging from 0.04 to 0.26 relative to the strongest baseline, GU. We have also analyzed the statistical significance of the gains and calculated one-sided Wilcoxon signed-rank p-values for each VFD-baseline comparison pair. The results are shown in the table below, with each cell indicating the number of pairs in which VFD beats the baseline and its corresponding p-value.
| VFD vs. | SmolVLA | X-VLA | FastWAM | All 15 pairs together |
|---|---|---|---|---|
| Action-L2 | 5/5, | 4/5, | 5/5, | 14/15, |
| ACE | 5/5, | 5/5, | 5/5, | 15/15, |
| DECU | 5/5, | 5/5, | 5/5, | 15/15, |
| GU | 5/5, | 3/5, | 5/5, | 13/15, |
| Entropy / Perplexity | 5/5, | N/A | N/A | — |
The gains of VFD reach in 11 out of 13 per-model comparisons. When considering all three policies together, the gains of VFD over the baselines have p-values ranging from to . This statistical analysis highlights that the improved calibration of VFD is consistent and statistically significant across different generalist policies. We have added this discussion to Appendix C.2.1.
W4/Q4: Non-standard LIBERO protocol.
The standard LIBERO protocol is to train on all available data from a single suite (Goal/Spatial/Object/Long) or all suites at once. In contrast, we consider the case that data is not yet available but needs to be collected, and our objective is to achieve maximum performance under a limited demonstration budget. To evaluate the impact of iterative fine-tuning (as opposed to a single fine-tuning run), we have conducted an additional experiment, in which the policies are trained only once on 75 episodes, using the cumulative number of gradient steps of all iterative fine-tuning rounds. This setting corresponds to the common setting in which data across tasks are collected only once, uniformly at random, without uncertainty guidance. This approach yields a final success rate on LIBERO-Long of for SmolVLA (SAVE w/ VFD: ) and for X-VLA (SAVE w/ VFD: ). This experiment demonstrates that, given the same data budget, iterative active data acquisition guided by VFD uncertainty can achieve higher absolute performance than common one-shot fine-tuning. We have added this discussion as a new Section C.7 to the Appendix.
Thank you for the detailed review and the positive evaluation of our work. We believe that integrating the suggested experiments has strengthened our work, and we hope it has addressed your concerns. Please do not hesitate to ask any further questions.
I thank the authors for their response. The added experiments are solid, and my assessment has changed considerably.
W2 and W3 are withdrawn. The synthetic experiment separates epistemic from aleatoric uncertainty; the disjoint-subset ablation (0.72→0.74) shows random shuffling already suffices; computational cost is now reported; and the initial-state calibration in C.2.5 (0.48 vs 0.42, 10/10, p=0.001) is more convincing than the task-level analysis.
W1 is downgraded. SAVE does have domain specificity (task-level aggregation, temperature-based sampling, selection over candidate initial states), so I withdraw "no robot-specific design" and reframe it as "the contribution's main body is a general estimator, with SAVE as its domain instantiation". It no longer counts as a major deduction.
Three points remain.
First, the calibration significance is constrained by design. With n=5 the smallest attainable one-sided Wilcoxon p is 0.031, and 5/5 lands exactly on that floor, so the value reflects sign agreement only, not effect size; the smallest p among 13 comparisons is 0.031, above the Holm threshold 0.05/13 ≈ 0.0038, so the family is unlikely to survive multiple-comparison correction. X-VLA is also non-significant against the strongest baseline GU (3/5, p=0.16, 0.48 vs 0.44). The pooled p=6×10⁻⁴ over 15 pairs is weakened by differing effect sizes across models (+0.04 to +0.26) and by seeds nested within models. Reporting per-model effect-size intervals, or a mixed-effects model with model as a random effect, would be more robust.
Second, the ensemble-free variant should not be framed positively. In C.5 it reaches AULC 47.8 and Last-3 57.2, against 52.5/63.0 for the full model and 47.3/57.4 for AMF — it degrades to the AMF level, consistent with the +5 pp from M=2→4. The gain depends on training one additional member, which reads better as a boundary condition.
Third, absolute performance still lacks a scale reference. I accept the C.7 comparison, and I agree that full-data training is not an appropriate baseline here; but C.7 answers "is this only more gradient steps", not "where does 63.0 sit". A single cited value for SmolVLA on full LIBERO-Long would suffice — no new run needed. I would also suggest cross-checking C.7 against C.8: random-selected one-shot 58.1, VFD-selected one-shot 70.6, VFD-selected iterative 67.1, i.e. selection ≈ +12.5 and iteration ≈ −3.5. If that reading is right, the effective ingredient in this section is selection rather than iteration, and saying so explicitly would make the method's operating conditions clearer.
Response to Reviewer tnbr (1)
Thank you for your constructive feedback, which we believe has significantly strengthened our paper. We have uploaded a revision with the changes highlighted in blue and have provided detailed responses to your comments below.
W1/Q1: Novelty and robot-specific design.
Theorem 4.1 is a novel, general formulation for flow-matching models (e.g., image/video generation) that holds beyond policies, and we believe that applying our proposed VFD score to flow-matching models in other domains represents an interesting avenue for future work. However, several design choices in our work are motivated by the specific challenges of robot learning:
SAVE w/ VFD particularly shines in active fine-tuning scenarios with very diverse possible initial conditions (e.g., object layouts), such as in our real-world experiments. There, SAVE w/ VFD achieves a final success rate of 48±4 and an AULC of 36±2, indicating that guiding data acquisition based on the model's current epistemic uncertainty is significantly more efficient than collecting data uniformly at random (34±7 and 25±1).
W2/Q2: Ensemble diversity and cost.
Epistemic vs. aleatoric uncertainty: To directly verify that VFD separates the two, we have conducted a new targeted experiment on our toy example (Appendix C.1, Figure 10), in which an ensemble of M=5 flow-matching models learns a 2D conditional Gaussian mixture. To vary the aleatoric uncertainty, we have made the standard deviation of the mixture components depend on the input. Over a 50×50 grid of inputs and five independently trained ensembles, VFD is strongly correlated with the epistemic uncertainty (Spearman ρ=0.84±0.02), but not with the aleatoric uncertainty (ρ=0.00±0.08), whereas the Action-L2 baseline mainly follows the aleatoric uncertainty (ρ=0.65±0.02). These results are in line with our theoretical derivation, which establishes VFD as a method for estimating specifically the epistemic component of model uncertainty.
Ensemble cost: Excluding parallelization, the cost of training an ensemble scales linearly with the ensemble size M. We provide details about the computational cost of iterative ensemble fine-tuning in Appendix B.1. However, we find that increasing the ensemble size beyond two yields only minor (calibration, Figure 11) to modest (failure detection, Table 4) performance gains in our active fine-tuning experiments. We argue that for active fine-tuning, the additional computation of training a second ensemble member amortizes particularly well in applications where collecting expert demonstrations is very expensive, such as surgery or care. Moreover, we find that even without training an ensemble, VFD can achieve strong performance. For example, as shown in Table 4, perturbing the weights of the policy to obtain a second model and reducing the computational overhead to a minimum (using only a single ODE timestep and batch size 1 for VFD computation) reduces failure-detection accuracy in the real world by only 5 pp, from 0.73 to 0.68.
Ensemble diversity: In our experiments, random shuffling of the mid-training data is sufficient to obtain an ensemble that, with VFD as the uncertainty signal, is well-calibrated at a negative Spearman rank correlation −ρ of 0.48 to 0.72 (see Table 1). As a potential way to further improve the diversity between the models, we have mid-trained an ensemble by randomly splitting Dpre into two disjoint halves (subsampling). This additional injection of diversity improves calibration for SmolVLA only slightly from −ρ=0.72±0.09 to −ρ=0.74±0.07 (Appendix C.2.4). This ablation supports our claim that our ensemble-creation strategy already yields policy ensembles sufficiently diverse across the scenarios we consider.
Ensemble-free active fine-tuning: We have evaluated SAVE w/ VFD without an ensemble, instead perturbing weights with layer-normalized Gaussian noise, for iterative active fine-tuning on SmolVLA. Despite its simplicity, this approach achieves AULC and Last-3 success rates of 47.8±5.9 and 57.2±4.8, tied with AMF (AULC: 47.3±3.2, Last-3 success rate: 57.4±5.2) and above Random (AULC: 41.8±3.1, Last-3 success rate: 52.6±4.6), the two ensemble-free baselines.