- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Risk-Informed Conceptors. It has three components. (1) a failure detector with rollout-level supervision to produce timestep-level failure scores. (2) The detector separates failed trajectories into alarmed and non-alarmed activations. These representations, together with successful trajectory activations, are used to construct conceptor matrices for modifying VLA hidden states. The authors additionally introduce an operator-aware layer selection criterion. (3) Steering is activated after the first failure alarm. The experiments cover LIBERO-10, LIBERO-Plus, LIBERO-Pro, and RoboCasa, using π0 and π0.5 backbones.
Critical strengths
-
The paper identifies a plausible limitation of rollout-level steering references. Separating normal execution from failure-related execution within failed trajectories is meaningful, especially for long-horizon manipulation.
-
Integration of detection and correction. Rather than using failure detection only to decide when to intervene, the proposed approach also uses detection signals to determine which representations should construct corrective operators.
Critical weaknesses
-
The proposed method combines refined failure references, detector-based gating, and new layer selection. However, existing steering baselines do not use detector gating. It is unclear whether the improvements come from better conceptors or simply fewer unnecessary interventions. An ablation separating these components is needed.
-
Failure detection does not necessarily identify incorrect actions. The detector is trained using rollout-level outcomes, so it may learn which states predict eventual failure rather than which timesteps contain actual execution errors. The paper should validate whether detected failure-related representations truly correspond to local execution mistakes.
-
Layer selection is not directly linked to better actions. The proposed criterion favors layers where steering modifies failed representations more than successful ones. However, larger representation changes do not necessarily produce better actions. The authors should demonstrate that this criterion correlates with actual improvements in policy behavior.
Questions for authors
-
Source of improvement: How much of the improvement comes from better conceptors versus detector-based gating? Can these effects be evaluated separately?
-
Does the detector identify actual execution errors, or just states associated with eventual failure?
-
Why should modifying failed representations more strongly lead to better actions? Is there empirical evidence?
-
Are the relatively small success-rate improvements statistically significant across multiple runs?
-
What additional benefit does Classic-weighted Local provide over the Classic operator?