acceptodds
Under review as a conference paper at ICLR 2027

DUET: Dual Uncertainty Evidence Tracking with Visual Tokens and Action Traces in VLA

Abstract

A frozen vision-language-action (VLA) policy may encounter unfamiliar visual content or multiple valid action continuations during deployment. We associate these cases with different uncertainty effects: coverage-related epistemic effects for unfamiliar visual content and action-related aleatoric effects for multiple valid continuations. Existing monitors often merge both into one score and obscure the source of a warning. DUET tracks the two evidence sources separately: a visual evidence branch measures novelty relative to successful demonstrations, while an action evidence branch measures local conditional dispersion at a fixed observation and noisy flow state. Separate references calibrate the two branches into EXECUTE, CAUTION, and HELP, and the visual branch also provides spatial evidence. The visual evidence branch detects unfamiliar observations with frame AUROC and identifies the corresponding regions. The action evidence branch detects local action ambiguity with AUROC . On a physical robot, HELP triggers in unfamiliar-object trials and familiar-object trials, with every unfamiliar alert preceding irreversible motion. DUET shows how a frozen VLA can track the provenance of uncertainty-related evidence and convert complementary evidence into calibrated deployment responses.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Uncertainty Quantification, Robot Safety

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_zAPzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

As frozen VLA models often encounter unfamiliar visual content (epistemic uncertainty) or multiple valid action choices (aleatoric uncertainty), they propose DUET to track uncertainty separately through a visual evidence branch for novelty and an action evidence branch for local action ambiguity, calibrating both into three clear responses: EXECUTE, CAUTION or HELP. They found DUET detects unfamiliar visual frames accurately (0.940 AUROC) and action ambiguities (0.963 AUROC), successfully triggering safety alerts.

Critical strengths
  1. Methodology-wise, DUET separates visual novelty from action dispersion, which avoids conflating failure modes into a single scalar score.
  2. It has formal and mathematical grounding for trace evaluations under Gaussian flow matching.
  3. It can be deployed on frozen pre-trained VLA models with an easy manner, without fine-tuning, architectural retraining or disrupting standard deployment flow.
Critical weaknesses
  1. Unclear motivations why detecting novel evidence are correlated with evidence provenance.
  2. Lack of comparisons with prior work in the introduction to motivate their methods. This is also correlated to an unclear writing in the introduction, which would benefit from improvement.
  3. Methodological details not clear enough. e.g., Although this is a minor issue, I think the author should specify that in the 0.963 \pm 0.052 of AUROC, is 0.052 the SD, SE or 95% CI? e.g., For all of the accuracies in both the Table and the main text, the authors would benefit from explaining whether the number after the \pm is SD, SE or 95% CI, or others. e.g., The authors should briefly explain H in L151. e.g., Are 4.6 a corresponding experiment? What is the setting? Are Figure 4 some illustration cases or are they realistic cases in the evaluation of 4.6? The authors need to explain these.
Questions for authors
  1. What motivates the choice to detect novel evidence, and why this is useful, what are the practical implications of such a choice?
  2. How the authors' work differed from prior papers?
Reviewer_CzeyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors use a frozen flow-matching VLA (SmolVLA), computing a visual-novelty score (PCA + GMM on pre-fusion tokens to capture scene-level novelty and spatial map) and an action-dispersion score (trace of the conditional covariance, estimated via Hutchinson/VJP on the velocity-field divergence, via a Tweedie-type identity). Each is split-conformal calibrated and mapped to EXECUTE/CAUTION/HELP. The authors use for validation a controlled route testbed (AUROC .963/.976), LIBERO leave-one-suite-out novelty detection (.940 frame AUROC), numerical audits, routing tables, and 45/50 vs 1/50 HELP triggers on hardware. The identity and audits are useful. A core question the paper presumes (whether separated evidence helps deployment decisions), is not answered, but the paper does present a useful structure towards that question. While the the derivation checks looks reasonable, it could be more detailed; The concerns are about evidence and scope -- fixable but substantial. Furthermore, the paper could benefit from additional probes of the behavior in different conditions.

Critical strengths
  • The paper addresses an important problem (uncertainty of VLAs) with a new approach,
  • Clean, efficient representation that has a plausible interpretation of the readoutprovenance-preserving readout. -- The action score has an exact identity for the optimal field, and the VJP implementation agrees with explicit Jacobians (max error ~3e-7). -- Time overhead is modest / runtime is still reasonable.
  • Reasonable validation of each channel. The matched-input control, LOSO detection with region localization, and a physical HELP study with Wilson intervals give concrete evidence that each branch responds to its intended shift.
Critical weaknesses

main comments:

  • L040, elsewhere - The use of epistemic/aleatoric uncertainty is unfounded, clarify/hedge in what sense. (the paper uses in several places the term “associated with..” - but this is not a proof.

  • related works are weak: -- Cite “ON THE POSTERIOR DISTRIBUTION IN DENOISING: APPLICATION TO UNCERTAINTY QUANTIFICATION”, “Uncertainty in a Single Pass: A Closed-Form Identity for One-Step Flow Matching“ to mark relation to second-order Tweedie inequality and covariance estimates. -- Cite more approaches for novelty detection for policies: --- Failure Prediction at Runtime for Generative Robot Policies --- Variational Autoencoder for End-to-End Control of Autonomous Driving with Novelty Detection and Training De-biasing --- Can We Detect Failures Without Failure Data? Uncertainty-Aware Runtime Failure Detection for Imitation Learning Policies

  • In general, would be good to map the assumptions used in the paper. While the paper refers to appendix B, the appendix doesn’t have all of them either. E.g.: -- Accuracy of the learned field’s derivative / optimality of the learned field (for eq. 3) -- Evaluation-state distribution used in the euler update estimates for x (perhaps show how sensitive it is to choice of the number of steps? Or variance of that over data samples?) -- Eq. 4 shows the readout drops the between-state mean term, i.e. the mode-separation component. No \tau ablation, no P ablation.

  • would be good to check how sensitive is the approximation of the Jacobian or trace based on samples, and dependency on the number of samples+interpolation states (i.e not just interpolation states).

  • The evaluation can be made better, both in terms of probing the model behavior, and stronger testbenches -- The matched-input control only shows the visual score is flat when the action head changes, which is true by construction (AUROC .500). The reverse test is missing: action score under visual shift, and visual score under multimodality. -- The action evidence rests on a toy testbed with labels set by design (1-route vs 2-route training), with saturated AUROC 1.000. Would be good to use more diverse testbeds in terms of their trajectory distribution and objects, to see how stable is the result, and test on a harder testbed that is not saturated.

  • Would be good to note the algorithm's behavior over simple changes to the data, such as illumination/image modification (e.g. affine distortion, added noise, region occlusions) to have some idea on how well the action dispersion behaves when there's a visual shift, and probe the model vs visual phenomena.

Questions for authors
  • See my questions above. My main concern is better clarification of the assumptions, better posing of the result in the paper, and limitations of the paper so as to make it have a better contribution to the community, as well as making that contribution more accurate and improve the signal-to-noise ratio. Can you state a crisp plan for making this paper more accurate along these lines?
  • What can you show in terms of better comparison to SOTA, as well as probing of this approach along the comments I made above?
Reviewer_AfmZOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces DUET, an inference-time monitoring framework for frozen flow-based vision-language-action policies. The main idea is to preserve the source of uncertainty-related evidence rather than collapsing different effects into a single score. DUET uses a visual branch to measure novelty relative to successful demonstrations and an action branch to estimate local conditional action dispersion after fixing the observation and noisy flow state. These signals are associated with coverage-related epistemic effects and action-related aleatoric effects, respectively. The two branches are calibrated separately and mapped to EXECUTE, CAUTION, and HELP responses without retraining the underlying policy.

Critical strengths

A key strength of the paper is the clear separation between visual novelty and action ambiguity, addressing the limitations of prior approaches that collapse different uncertainty sources into a single score. The theoretical grounding is also strong, with the action branch derived from a Gaussian flow-matching covariance-divergence identity and the visual branch supported through a proposition motivating a separate demonstration-based reference.

The experimental design is carefully constructed to validate this separation. In the matched-input setting, where visual inputs remain fixed and only the underlying action distribution changes, the action branch achieves an AUROC of 0.976 while the visual branch remains at chance level (0.500 AUROC). This provides strong evidence that the two channels capture distinct phenomena.

The paper also includes thorough numerical validation and reproducibility checks. The proposed readout is compared against explicit-Jacobian and finite-difference implementations, and the full 350-dimensional action audit achieves a Spearman correlation of 0.940 with the reverse-SDE operational reference. Results are further confirmed across new scenes and multiple independently trained checkpoints.

Empirical results are consistently strong. DUET outperforms several feature-space baselines for visual novelty detection, provides spatial localisation without additional supervision, and demonstrates promising hardware performance, with HELP triggered in 45/50 unfamiliar-object trials and only 1/50 familiar-object trials.

Critical weaknesses

A limitation of the study is that all experiments are conducted using SmolVLA. While multiple checkpoints and training seeds are evaluated, validation on additional flow-based or diffusion-based VLA architectures would strengthen the paper's claims of generality.

The hardware evaluation focuses primarily on the visual HELP mechanism. Although the paper presents the complete EXECUTE-CAUTION-HELP framework, it does not provide quantitative robot experiments assessing the action-driven CAUTION branch or the full three-way routing behaviour in deployment.

Finally, the full-action operational-reference audit is conducted on only 32 held-out observations. While the observed correlations are strong, a larger evaluation set would provide greater confidence in the alignment between the deployed readout and the proposed reference measure.

Questions for authors

The action readout is assessed solely at the midpoint Euler step (0.5). What is the sensitivity of route discrimination and operational-reference alignment to interpolation time steps that occur earlier or later along the flow trajectory?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.