Steering Vision-Language-Action Models with Latent Risk-Informed Conceptors for Reliable Execution

Vision-language-action models (VLAs) enable robots to generalize across diverse tasks, yet local execution deviations can accumulate into task failure, limiting deployment reliability. Inference-time steering with conceptors can mitigate such deviations by preserving success-related representations and suppressing failure-related components without updating policy parameters. However, these methods construct failure references using rollout-level labels, potentially mixing normal and failure-related representations and leading to harmful interventions. We therefore propose Risk-Informed conceptors, a latent steering framework that uses local failure evidence to refine the representations from which steering matrices are constructed. We leverage weakly supervised failure detection to obtain step-level risk estimates from rollout-level labels, using multiple-instance learning (MIL) with task-wise mixture-risk correction. These estimates identify failure-relevant representations for conceptor construction and intervention-layer selection. At inference time, the detector provides calibrated risk signals to determine when and how steering is applied, while keeping the VLA frozen. Extensive experiments across multiple benchmarks validate the performance of Risk-Informed conceptors, showing consistent improvements over frozen VLAs.