RouteRL-VLA: Learning from Failures via Factorized Routing for Vision-Language-Action Model

Vision-Language-Action (VLA) models can acquire broad manipulation skills through large-scale pretraining and supervised fine-tuning. Yet a failed rollout is reduced to a single outcome signal. Without knowing where the error arose or which part of the policy caused it, existing reinforcement-learning methods for VLA models often spread credit across semantic, visual, and execution failures, diluting informative gradients and risking interference with previously acquired capabilities. We introduce RouteRL-VLA, a failure-aware post-training framework that factorizes credit over failure type, trajectory time, and parameter group. It constructs and compares paired clean and perturbed rollouts, uses a selective paired teacher to estimate failure attribution, soft failure-type probabilities, and query-level temporal weights, and directs the resulting advantages to specialized policy adapters. For stable on-policy learning with continuous action chunks, we develop joint action-chunk likelihoods, duration-aware generalized advantage estimation, and transactional KL-constrained PPO. When applied to OpenVLA-OFT on LIBERO-Goal, RouteRL-VLA attains SOTA failure-recovery performance among five trained methods: 62.15% perturbed macro success, 2.99 points above uniform credit, with 95.83% clean success. In a separate 1,224-episode evaluation, it improves success from 45.26% to 78.35%, a 33.09-point gain. Oracle routing and route-permutation controls confirm that accurate route-failure alignment drives this improvement. The attribution model reaches 0.988 AUROC, while the selective router achieves 0.992 macro-F1 at 0.943 coverage. The experiments jointly show that paired diagnosis identifies how a rollout failed, factorized routing targets the responsible update, and alignment controls verify that informative routes drive recovery.