- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
As frozen VLA models often encounter unfamiliar visual content (epistemic uncertainty) or multiple valid action choices (aleatoric uncertainty), they propose DUET to track uncertainty separately through a visual evidence branch for novelty and an action evidence branch for local action ambiguity, calibrating both into three clear responses: EXECUTE, CAUTION or HELP. They found DUET detects unfamiliar visual frames accurately (0.940 AUROC) and action ambiguities (0.963 AUROC), successfully triggering safety alerts.
Critical strengths
- Methodology-wise, DUET separates visual novelty from action dispersion, which avoids conflating failure modes into a single scalar score.
- It has formal and mathematical grounding for trace evaluations under Gaussian flow matching.
- It can be deployed on frozen pre-trained VLA models with an easy manner, without fine-tuning, architectural retraining or disrupting standard deployment flow.
Critical weaknesses
- Unclear motivations why detecting novel evidence are correlated with evidence provenance.
- Lack of comparisons with prior work in the introduction to motivate their methods. This is also correlated to an unclear writing in the introduction, which would benefit from improvement.
- Methodological details not clear enough. e.g., Although this is a minor issue, I think the author should specify that in the 0.963 \pm 0.052 of AUROC, is 0.052 the SD, SE or 95% CI? e.g., For all of the accuracies in both the Table and the main text, the authors would benefit from explaining whether the number after the \pm is SD, SE or 95% CI, or others. e.g., The authors should briefly explain H in L151. e.g., Are 4.6 a corresponding experiment? What is the setting? Are Figure 4 some illustration cases or are they realistic cases in the evaluation of 4.6? The authors need to explain these.
Questions for authors
- What motivates the choice to detect novel evidence, and why this is useful, what are the practical implications of such a choice?
- How the authors' work differed from prior papers?