- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
RouteRL-VLA is an RL post-training framework for Vision-Language-Action (VLA) models that factorizes credit assignment across failure type (Semantic, Visual, Execution), trajectory time, and three LoRA adapter groups on a frozen VLA backbone. It uses paired clean and synthetically perturbed rollouts from the same task, initial state, and simulator seed to train a selective teacher that estimates failure attribution, failure-type probabilities, and temporal weights for chunk-level PPO updates. The authors validate the method on OpenVLA-OFT using the 10-task LIBERO-Goal suite under synthetic perturbations.
Critical strengths
Trajectory-level binary rewards in VLA RL post-training fail to distinguish whether a failure arose from language comprehension, visual perception, or motor execution, and that on-policy RL updates should operate at the joint action-chunk level with duration-aware GAE identifies a practically relevant credit-assignment problem in post-training VLA policies.
Critical weaknesses
-
Undefined perturbations and failure types (L141, L149–151): The entire method and evaluation rely on three synthetic perturbation types (
sem,vis,exe). Beyond a single generic sentence in L149–151, the paper never specifies how instructions are altered forsem, what visual corruptions are applied forvis, what control-channel disturbances are injected forexe, or what the "perturbation schedule" (L219) is. Consequently, "failure type" is merely the synthetic perturbation label injected by the authors rather than an intrinsic failure mode of the policy. -
Undefined "Selective Paired Teacher" and vague notation (L145, L155–161): In L145, "The shared context" is referenced without clearly defining it. In L155, and are used without subscripts or clarification of whether they represent full post-hoc trajectories or partial histories up to query . Crucially, the paper never defines what features are extracted, what the teacher's architecture or training loss is, how the ground-truth "divergence region" for temporal weights (L160, Table 8) is mathematically defined or supervised, or what the abstention thresholds are.
-
Unjustified need for a learned teacher router (L141–161, L318): At test time, the teacher router is not used at all, nor can it be used in real-world deployment since it requires a paired clean reference trajectory from the exact same initial state and simulator seed. Conversely, during simulated training, generating the pair already requires explicitly injecting a known perturbation and observing the terminal outcomes and (Eq. 1). Because both the perturbation type and attribution outcome are already known by construction in simulation and unavailable at test time, it is unclear why a learned teacher classifier is needed at all.
-
Temporal misalignment during on-policy PPO updates (Figure 1, Eq. 3): If the paired trajectories are collected upfront and their teacher outputs are reused for subsequent "fresh action-chunk rollouts" (Figure 1) during PPO training, the temporal credit assignment becomes invalid. As the policy is updated by PPO, the actions and trajectories of new rollouts change at every iteration; thus, multiplying a new rollout's advantage at step by the temporal weight computed from an earlier trajectory (Eq. 3) is misaligned in time. Conversely, if a new paired clean rollout must be executed online for every perturbed RL rollout, the simulation cost doubles and requires exact simulator state/seed resets.
-
Unfair baseline comparison in the 1,224-episode evaluation: In Table 5, the RL post-trained
RouteRL-VLA(78.35%) is compared only against the un-finetuned base modelOpenVLA-OFT, omitting the RL baselines (Vanilla PPOandUniform credit). Comparing an RL-finetuned policy against a zero-shot base model on the perturbed distribution does not isolate the benefit of the proposed routing mechanism. -
Narrow evaluation on a single model and a single sub-suite: Standard evaluation requires testing across all four LIBERO benchmark suites (
LIBERO-Spatial,LIBERO-Object,LIBERO-Goal,LIBERO-Long/LIBERO-10), as well as broader benchmarks such asRoboCasa,SimplerEnv, or public perturbation benchmarks likeLIBERO-Para(which the authors cite in L111 but do not evaluate on). Evaluating only a single base model (OpenVLA-OFT) on a single 10-task suite (LIBERO-Goal) under undisclosed custom perturbations is insufficient to support the paper's claims. -
Manuscript preparation and presentation issues: Out of the eight tables in the paper, only Table 2 is ever referenced in the main text, and Tables 1, 3, 4, 5, 6, 7, and 8 are completely uncited in the text. Furthermore, the tables are numbered out of order (Table 7 appears on page 7 before Tables 5 and 6 on page 8), Table 6 reports qualitative text ("Upper bound passed") without any numerical values, and there are multiple formatting typos.
Questions for authors
Following the weaknesses outlined above, my primary concern is that the current manuscript lacks essential details needed to fully understand and reproduce the proposed method, alongside concerns regarding the methodological motivation, experimental breadth, and overall manuscript presentation, which led to my current scores. I would appreciate the authors' clarification on the points raised above.