- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper addresses the problem of correcting small action errors that pretrained multi-task VLA policies make in local phases such as alignment and contact, using online RL. Prior residual RL methods reduce such errors by learning a small corrective policy on top of a frozen base, but they require a separate corrector for each task. The authors observe that tasks with different goals, such as grasping, placing, and inserting, often require similar local corrections, and therefore treat action correction as something that can be shared across tasks. Their goal is a single module that corrects many tasks at once and can be applied to tasks not used in its training without further updates.
The paper proposes three components. First, GAE is a shared module that takes the observation, the robot state, and the base policy's action proposal, and outputs a bounded correction. All tasks share its parameters without task information, and the base policy stays frozen. Second, motivated by the observation that the value of a correction concentrates on a few moments in a trajectory, the authors define CIV as the critic-estimated value difference between the executed corrected action and the base action at the same state. CIV drives CISA, a sampling scheme that prioritizes experiences where the correction mattered, and a retention rule that keeps such experiences longer in replay. Third, ReSOP is an asynchronous system that separates collection, scoring, and learning to run this process across many tasks.
Experiments use 50 RoboTwin 2.0 tasks with a LingBot-VLA base fine-tuned on this benchmark. A single editor trained on 46 tasks raises macro success over all 50 tasks from 90.00% to 90.84%. On the 4 held-out tasks, it moves success from 88.50% to 88.75% without any updates. These results show that one correction module can serve many tasks while improving performance, with a weak generalization gain on unseen tasks. In a 4-task comparison, CISA reaches 86.00%, above uniform sampling at 82.75% and TD-error prioritization at 81.00%.
Critical strengths
-
A clear formulation of action correction as a shared resource, with a systematic method and system to support it. Prior residual RL uses one corrector per task. This paper explicitly asks whether corrections can be shared across tasks and designs its components consistently around that question. The proposal-conditioned editor handles many tasks with one small module and never modifies the VLA. Since it uses no task information, it can be attached to unseen tasks as is. Together with ReSOP, which runs online RL over 46 tasks asynchronously, the paper offers a practical framework at a time when VLA post-training is becoming expensive.
-
Substantial gains on some tasks without hurting most others. Because the correction is tightly bounded, success on most of the 50 tasks stays close to the base, while macro success rises from 90.00% to 90.84%. On click_bell, success rises from 75% to 96% and the mean episode length drops from 70 to 48 steps. This points to a real change in behavior rather than noise. stack_bowls_three also improves from 72% to 85%. These results show that a shared editor on a strong base can fix systematic failures on specific tasks while preserving existing capabilities.
Critical weaknesses
-
The empirical gains are weak and not statistically established.
- The gains are small. Macro success over 50 tasks improves by only 0.84 pp. The 0.25 pp gain on the held-out tasks amounts to one episode out of 400, and the S10 editor actually lowers held-out success by 1.75 pp.
- The gains are concentrated in a few tasks. 21 tasks improve and 13 decline. Excluding the top 3, the remaining 47 tasks change by −0.04 pp on average.
- The room for improvement is structurally limited. The correction scale is fixed at 0.025% of the normalized action range, which narrows what can be fixed, and the base already reaches 90%.
- There are no real-robot experiments. Evaluation is limited to RoboTwin 2.0 Clean, without domain randomization, and to a single base policy. Local corrections matter most in real settings with sensor delay, calibration error, and contact mismatch. There is no evidence that corrections this small remain meaningful there.
-
No baseline isolates the effect of sharing.
- The paper's central question is whether sharing corrections across tasks helps. Yet it includes no comparison with independent per-task editors under the same budget, nor with existing per-task residual RL methods.
- It is therefore unclear whether gains such as those on click_bell come from sharing or could be obtained equally with per-task training.
- Although the editor receives no task information, it can identify the task from images. It may therefore simply store per-task corrections in one module rather than learn a shared correction structure.
- The held-out results are consistent with this. The S1, S4, and S10 editors all fall below the base on held-out tasks. Only S46 recovers to the base level, and its source set includes near-identical siblings of the held-out tasks, such as place_a2b_left for place_a2b_right.
Questions for authors
- Please see the weaknesses. Could the authors provide real-robot experiments, or results with multiple seeds, to show that the gains are robust?
- Could the authors compare the shared editor with independent per-task editors under the same budget, to isolate the effect of sharing?