- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a self-introspective approach in which a robot recognizes, during task execution, whether the current situation lies beyond its competence and requests assistance when it does. Unlike conventional flow-matching-based policies, action generation is reformulated as energy-based optimization. During training, the energy is modeled to increase along a prescribed pattern as the sample moves from noise toward the expert action; at deployment, the deviation from this pattern is used as an “uncertainty” signal. The method is applied to several VLAs and shown to improve control performance in both simulation and real-robot experiments. Furthermore, by using this signal to request human assistance and retrain on the collected data, the authors substantially improve real-robot success rates (96.7% and 98.3%), demonstrating the potential of metacognition-based robot learning.
Critical strengths
-
Uncertainty signal obtained directly from the generation process: The main strength is that the “uncertainty” signal arises directly from the action-generation process, without requiring a separate monitoring model. The signal considers the entire generation trajectory rather than only the final output, and the experiments show that this yields more accurate detection (Table 14). This makes the approach particularly practical for large VLAs, where running multiple models is costly, and it suggests that the generation process itself can serve as a source of information about a model’s reliability.
-
More effective intervention than human judgment: On the real robot, with the same amount of collected data, EqC-triggered intervention achieves much higher success than human-triggered intervention (96.7% vs. 58.3%). Since the two methods appear to share the same policy and differ only in how interventions are triggered (App. D.4), this is the most direct evidence that the signal identifies useful intervention points. A plausible explanation is that humans intervene only after a mistake becomes visible, whereas EqC can intervene earlier, when the model first begins to struggle, allowing it to collect data from states the model genuinely does not know how to handle.
Critical weaknesses
-
Insufficient comparison between automated signals: In simulation, the paper does include a comparison in which only the signal differs on the same policy (Table 3b), but EqC’s advantage shrinks considerably in that setting (at m = 11, roughly two-thirds of the TPR gain is attributable to the policy difference). In the real-robot lifelong experiments, however, EqC uses the EqC-trained policy, while Diff-DAgger uses the original G0.5 policy (the initial success rates on Garment Folding also differ, 30% vs. 25%). It therefore remains unclear whether the real-robot improvement over Diff-DAgger is attributable to the signal.
-
Unclear scope of failures the signal can detect: EqC treats a lack of smoothness in the action-generation process as a sign of uncertainty. As a result, it may miss “confidently wrong” cases, such as smoothly picking up the wrong block. Indeed, all failures detected by EqC in the paper (Figure 6), such as a block being pushed away or a sleeve not being grasped, involve gradual drift during execution. To claim that the robot is aware of its own competence, the paper should also show which failures the signal fails to detect.
Questions for authors
-
[Same-policy comparison] In the real-robot lifelong experiments, what happens if the Diff-DAgger signal is applied to the EqC policy? Could the authors also state explicitly whether the human baseline uses the same policy as EqC? If the EqC signal still performs better under the same policy, this would further strengthen the paper’s central claim.
-
[Detection timing] If EqC detects the onset of uncertainty earlier than humans do, can the authors show how much earlier the alarm is raised relative to the point at which the mistake becomes visible?
-
[Missed failures] How does the EqC signal respond to “confidently wrong” failures, such as selecting the wrong object? What kinds of failures does the signal tend to miss? Depending on the answer, I will reassess the scope of the claim that the robot “knows its own competence.”
5 comments
Summary
Dear Reviewer,
Thank you for your insightful feedback, particularly regarding the independent contribution and limitations of EqC's competence-awareness signal. We have strengthened the revised manuscript in three directions:
Same-policy signal attribution. We conducted new real-world closed-loop comparisons using the same initial EqC controller and matched demonstration budgets. Across three independent runs, Energy-Gated DAgger achieves 98.3% final Garment Folding success, compared with 81.7% for EqC + Diff-DAgger and 61.7% for a final-step gradient gate. All methods start at 30% success, directly isolating the contribution of the intervention signal.
Intervention timing and learning value. We clarified that our central claim concerns the informativeness of competence-guided recovery data, rather than a measured earlier detection time relative to humans. With comparable expert-labeled frame counts, EqC achieves 96.7% vs. 58.3% success on Block Stacking and 98.3% vs. 55.0% on Garment Folding, compared with human-triggered intervention using the same controller.
More failure case study. We expanded the failure-case study to discuss semantic and outcome-level errors that internal energy consistency may miss, including incomplete garment folding despite internally consistent actions. We also discuss task-level verification using VLMs as a complementary direction.
We would sincerely appreciate it if you could revisit these new experiments and clarifications and let us know whether they address your key concerns before the discussion period concludes. Thank you again for your constructive suggestions.
Best regards,
The Authors
Response to the authors' rebuttal
Thank you for the additional experiments and the detailed clarifications. My comments on each question are below.
[Same-policy comparison] This concern appears to be resolved. In the new experiment, all methods started from the same EqC checkpoint (initial success rate of 30%) and differed only in the intervention signal. Under this setting, the final success rate on Garment Folding was 98.3% for Energy-Gated DAgger and 81.7% for the Diff-DAgger signal. The improvement can therefore be attributed to the signal itself rather than to the policy. The clarification that the human baseline also used the same EqC policy further supports this conclusion. However, this comparison is based on three runs of a single task. To assess the stability of the result, I would appreciate it if the authors could also report the variability across runs.
[Detection timing] In my review, I interpreted the advantage of EqC as coming from intervening "earlier" than humans. From the authors' response, I now understand that the key factor is not the timing of intervention but the informativeness of the collected recovery data. Given that EqC achieved substantially higher success than human-triggered intervention with a comparable amount of expert data (96.7% vs. 58.3% on Block Stacking and 98.3% vs. 55.0% on Garment Folding), this explanation seems reasonable. That said, the central idea of this paper is introspection, namely that the policy itself recognizes its own competence boundary. From this perspective, it would be very helpful to show at which stages of the task EqC-triggered interventions mainly occurred. For example, if the authors could show how these stages differ from those of human-triggered ones, and whether they coincide with the points where the policy actually struggles, the claim that the signal reflects the policy's competence boundary would become much clearer.
[Missed failures] The example of a garment with one corner not fully folded confirms that the signal can miss failures in which actions are generated smoothly but the outcome is wrong. I think this is a matter of the scope of the signal rather than a flaw in the method. Still, given the focus on introspection, it is important to delineate this scope clearly. For example, if the authors could report the fraction of failed real-robot trials in which the signal never exceeded the threshold, together with the types of these failures, it would become clear which failures the signal recognizes and which it misses. In addition, stating explicitly in the paper that the signal reflects inconsistency in the action-generation process rather than task success would convey the scope of the claim that the robot "knows its own competence" more accurately to readers.
Summary My main concern (same-policy comparison) has been resolved by the new experiment, and I regard the other two points as additional details that would further clarify the authors' claims. I will keep my current score of 3.
Response to Reviewer 85q6
We thank the reviewer for the thoughtful and constructive feedback. We disentangle the issues below and address each with direct evidence and clarification.
Response to W1 Q1: Same-Policy Comparison
We thank the reviewer for this important suggestion. Our new same-controller closed-loop experiments demonstrate the effectiveness of EqC's intervention signal under matched lifelong-learning conditions.
We evaluate two additional automated gates, EqC + Diff-DAgger and EqC + Final-Step Gradient Gate, on real-world Garment Folding. All methods share the same initial EqC checkpoint, lifelong-learning protocol, and demonstration-count budget, differing in their intervention signals. Each method is evaluated over three independent runs, with 20 autonomous trials per checkpoint per run.
Table R1: Same-controller closed-loop lifelong learning on Garment Folding. Results report success rates (%) averaged over three independent runs.
Energy-Gated DAgger consistently outperforms both alternative gates, achieving improvements of 16.6 and 36.6 percentage points at R2, respectively. These multi-run results provide direct empirical evidence that path-wise energy consistency contributes to more effective closed-loop lifelong learning, beyond improvements attributable to the initial policy architecture.
We also clarify that the human-intervention baseline uses the identical initial EqC policy and same EqC controller as our method, with only the intervention trigger changed. We will explicitly clarify this setting and include the additional same-controller comparisons in the revised manuscript.
Response to Q2: Detection Timing
We thank the reviewer for raising this question. Our real-world experiments demonstrate the learning benefits of EqC-triggered intervention.
Across three independent runs, EqC and human-triggered intervention collect an average of 13,883 vs. 13,141 expert-labeled frames on Block Stacking (five rounds), and 7,848 vs. 7,384 on Garment Folding (two rounds), respectively. With comparable amounts of expert supervision, EqC achieves substantially higher final success rates on both tasks: 96.7% vs 58.3% on Block Stacking and 98.3% vs 55.0% on Garment Folding. We do not claim that EqC intervenes earlier than humans, as the primary factor driving effective lifelong learning is the informativeness of the collected recovery demonstrations, rather than intervention earliness.
Intervention timing and learning value. For intervention-driven lifelong learning, the value of an intervention lies in the learning opportunities it creates. Recovery demonstrations from challenging states can provide informative supervision beyond expert trajectories in already mastered states. EqC enables the policy to assess its own competence directly from action generation, identifying states where additional expert supervision can meaningfully improve its behavior. This model-internal assessment allows intervention and data acquisition to be guided by the policy's learned competence. Human operators typically infer the policy's competence boundaries from observable behavior, whereas EqC's intrinsic competence assessment can better align expert supervision with the policy's actual learning needs.
Together, these results highlight the value of competence-aware intervention for acquiring informative recovery demonstrations and improving lifelong policy learning with comparable expert-data costs.
Response to W2/Q3: Missed Failures
Missed-failure examples. In Garment Folding, the robot may complete the folding motion while leaving one corner improperly folded, producing subtle visual differences that are difficult to detect from the robot's camera viewpoint. Nevertheless, such failure cases have only been observed occasionally in our real-world experiments. And these wrong behaviors reflect a common limitation of current VLA policies.
To mitigate these limitations, a promising direction is to complement EqC's model-internal signal with task-level verification, such as using a VLM to check final manipulation outcomes. Importantly, EqC provides proactive, path-wise competence assessment directly from action generation, without requiring an additional uncertainty estimator or waiting for observable task failures. Task-level verification can complement this capability by identifying semantic or outcome-level errors that internal consistency alone may miss. We view this integration as a promising future extension.
Finally, we thank you again for your time and consideration, and would appreciate it if you could consider raising the score in light of these clarifications and additional results.