- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3(+1)
Read review
Paper summary
This paper studies epistemic uncertainty estimation for pretrained flow matching robot policies. It establishes an variance-collapse bound for BayesDiff and proposes CoMP-EKF and CoMP-UKF, which retain state–weight cross-covariance during integration. It also augments the Bayesian last layer with existing observation embeddings. Experiments evaluate posterior covariance approximation, sensitivity to language instructions, failure detection, and demonstration selection across three manipulation benchmarks.
Critical strengths
- The paper identifies a consequential inconsistency in uncertainty propagation: refining the integration grid can drive estimated epistemic variance toward zero. The analysis clearly explains how discarding dependence between states and shared model weights causes this behavior, providing a useful methodological insight beyond the particular robotics applications.
- The proposed correction has convincing empirical support for its immediate objective. Both CoMP variants closely recover the reference posterior covariance on robomimic, while the full-covariance BayesDiff control helps distinguish the missing state–weight dependence from merely discarding off-diagonal covariance. The downstream experiments further demonstrate useful, although task-dependent, applications.
Critical weaknesses
- The computational practicality remains insufficiently established. CoMP-UKF evaluates sigma points per integration step—225 in the robomimic experiment—whereas sampling baselines generally use 16 posterior draws. No inference latency or peak working-memory measurements are reported. Consequently, lower posterior storage and fitting costs do not establish a favorable inference trade-off, particularly for runtime monitoring. A comparison against posterior sampling using the same endpoint uncertainty score and comparable computation is also missing.
- The downstream evidence is less decisive than the covariance results. Figures 4–5 compare augmented-head CoMP against plain-head baselines. The appendix provides useful controls, but augmented DECU slightly exceeds CoMP-UKF’s robomimic TWA (0.662 versus 0.656), and Figure 13 shows higher mean fine-tuning gains for some augmented baselines. Furthermore, the thresholding and selection rules in Eqs. (33) and (35) are invariant to uniform positive score scaling, so variance shrinkage alone does not explain the downstream deterioration in Table 4.
Questions for authors
- What are the inference latency and peak memory of both CoMP variants on π0.5 and SmolVLA? Under comparable computation and the same augmented posterior, how does CoMP compare with posterior sampling scored using U_endpoint?
- As integration steps increase, does BayesDiff’s relative ranking of observations or demonstrations deteriorate? Quantifying this would help distinguish loss of useful uncertainty information from changes in absolute scale, which the downstream decision rules normalize away.
Thanks for the response to my concerns. I am happy to keep my score as is.