- Rating
- 3(+1)
- Confidence
- 3(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes EQRL, which uses a lightweight RL adaptor to select the number of denoising steps K and the execution chunk length C for a frozen flow-based VLA model. The authors evaluate EQRL in both simulation and real-world experiments, comparing it with other RL adaptation methods like DSRL.
Critical strengths
-
The motivation is clear. The authors investigate how to adapt the number of denoising steps and the execution chunk length at inference time while keeping the VLA frozen, motivated by variation in difficulty across different stages of task execution. Fig 4 provides an illustration consistent with this motivation.
-
The evaluation demonstrate that the method can be deployed in real-world settings.
Critical weaknesses
However, I still have the following concerns:
-
The presentation of experimental results does not adequately convey uncertainty. Fig 3 shows learning curves for six simulation tasks. The first five report the run with the highest full-training AUC among five seeds, whereas the last one shows seed 1 with a trailing average over five evaluations. However, reporting the mean success rate across all five seeds at each evaluation checkpoint, with a shaded band indicating ±1 standard deviation, would better convey typical learning behavior and variability across seeds. In addition, with only 20 trials per method and task, the real-world evaluations provide limited evidence for differences in success rate between DSRL and EQRL.
-
Limited comparison with and discussion of related methods. The main RL baselines in this paper are DSRL and RLinf-SDE/Noise. However, some relevant methods are not included in the experimental comparisons. RLT[1] and the first stage of PLD [2] also freezes the VLA backbone, and training a small adaptorusing RL. AAC [3] adapts the execution chunk length at inference time in a training-free method, addressing a closely related aspect of the problem studied in this paper. The authors should clarify how EQRL differs from these methods and discuss the relevance of experimental comparisons.
[1] Charles Xu et al. RL Token: Bootstrapping Online RL with Vision-Language-Action Models. arXiv:2604.23073, 2026. https://arxiv.org/abs/2604.23073
[2] Wenli Xiao et al. Self-Improving Vision-Language-Action Models with Data Generation via Residual RL. arXiv:2511.00091, 2025. https://arxiv.org/abs/2511.00091
[3] Yuanchang Liang et al. Adaptive Action Chunking at Inference-time for Vision-Language-Action Models. CVPR, 2026. https://arxiv.org/abs/2604.04161
Questions for authors
-
The reward formulation is complex, and the real-world experiments use different training objectives from those used in simulation. This raises a question about the manual design and tuning effort required to apply the method to a new setting. Could the authors describe the manual design and tuning effort involved, and quantify it where possible?
-
How long does online training on the physical robot take?
-
Have the authors evaluated the method on tasks requiring finer manipulation precision?
6 comments
Response to Reviewer WS92 (Part 2/2): Related work, tuning, time, and precision
Response to Reviewer WS92, Part 2/2.
W2. Relation to RLT, PLD, and AAC
EQRL focuses on jointly learning for a frozen VLA: latent behavior adjustment, denoising computation, and feedback timing. Related methods overlap with different parts of this design.
RLT first adapts an RL-token representation, then freezes the VLA and that representation during online RL, using a lightweight actor–critic to adjust action chunks while regularizing toward the VLA reference. PLD first learns an action-space residual with a frozen VLA, then collects data for distillation into the base policy. AAC selects an execution prefix using action entropy and motion magnitude across candidate chunks, without additional scheduler training. EQRL instead learns latent steering, denoising steps, and execution length jointly before invoking the frozen generator.
DSRL shares EQRL's frozen generator and latent-adaptation interface and differs in whether K/C scheduling is learned, so it is the direct baseline for testing our contribution. RLT and the first stage of PLD change how actions are adapted, and AAC adjusts only the execution length; none of them learns K and C jointly, so they are complementary to EQRL's schedule learning and we did not run protocol-matched comparisons. Within our framework, the C-only variant, which learns only the execution length, needs requested NFE 0.41, whereas full EQRL uses 0.32.
Q1. Reward design and task-specific tuning
New tasks reuse the same core training hyperparameters and need no manual reward design: the task reward is 1 for success and 0 otherwise, without separate dense rewards for grasping, moving, and placing. The schedule-related settings (K/C ranges, the NFE band, and the difficulty-related weights) give similar success across their basic range; about 3–5 runs within a small range are enough to reach the best result.
Simulation uses an episode-level compute regularizer. Online robot training uses the query-local cost when the efficiency term is enabled, where L counts executed actions. Offline training lacks interaction feedback for alternative K/C choices, so it uses expert-action matching for scheduling supervision together with compute regularization. These differences reflect the data and feedback available in each setting.
The four LIBERO-90 tasks share batch size 256, discount 0.999, 10 critics, and actor/critic/temperature learning rates . The settings chosen per task are the K/C ranges, the NFE band and its weights, and the difficulty weight and prior scale (Table 7 of the paper). The additional three-seed sensitivity study supports this: when the alignment and cost weights vary between 0.5× and 2×, success stays between 78.67% and 87.33%, within one SD of the default.
Q2. Online robot training time
Online robot training takes approximately 1–2 hours, including robot interaction, model updates, and routine resets, but excluding collection of the initial 20 demonstrations. The complete online stage uses approximately 15k executed robot actions and 500k SAC gradient updates, with 1,000 updates after each collection round. This is the total online-training time, distinct from the final policy's approximately 11-second trajectory execution time.
Q3. Higher-precision tasks
We have not evaluated tight-tolerance insertion or screw assembly, or measured millimeter-level pose error.
The current robot tasks do include precision-demanding stages: bottle–cup alignment for bimanual pouring, bimanual cucumber handover, and placing a cup into a bowl before lifting the bowl; a trial succeeds only when all stages are completed. EQRL's largest gain is on Bimanual Coke Pouring, which requires bottle–cup alignment (15/20 versus 11/20), and its requested NFE peaks near alignment and grasping (Figure 6 of the paper), so elastic scheduling spends more computation and feedback where precision is needed.
Thank you for the response. It partially addresses my concerns, but I think a few points still need clarification. I have not seen an uploaded revision, so the comments below are based on your response and the original PDF.
W1 — Simulation. Thank you for providing the new five-seed results on these four tasks. My main concern, however, was about how the results were presented. The "Evaluation and metrics" paragraph in Section 4.1 already states: "Unless otherwise stated, simulation experiments use five random seeds." I was therefore asking for Figure 3 to show the mean and standard deviation across those existing runs. I'm not require new training experiments, if the evaluation logs are available.
W1 — Real-world evaluation. I may not have expressed this clearly enough. By "provide limited evidence," I meant that the number of trials makes it difficult to establish a success-rate advantage. I'm not asking for CIs. Apart from the pouring task, the differences are either zero or five percentage points, corresponding to at most one successful trial out of twenty. More trials—for example, 40–60 per method and task—would help improve the precision of the estimates. Given the remaining time, I am not asking you to add these experiments now. I am comfortable with the more cautious description in your response, "comparable or higher observed success rates," and would appreciate having this qualification reflected in the paper.
W2 — Related methods and comparisons. I would also like to clarify my point here. I agree that DSRL is the most directly matched baseline for isolating the contribution of learned scheduling. However, I do not think this removes the value of discussing or comparing other closely related approaches. In particular, I still think a comparison with AAC would be useful. It directly addresses the choice of execution length C without additional scheduler training. The C-only variant in EQRL is not equivalent to AAC.
More broadly, the introduction moves from learning a lightweight latent-space policy for a frozen VLA to jointly learning , with the observation that latent steering alone leaves the execution schedule fixed. This explains what EQRL adds, but does not by itself establish why the additional complexity is worthwhile. If a simpler adaptation or scheduling method already performs well, what additional benefit does joint learning provide?
For PLD and RLT, my concern is also about positioning the work. If these approaches are complementary to EQRL, as you suggest, it would help to explain more clearly how their action-adaptation mechanisms relate to the latent steering and scheduling used here, and whether these components could be combined. (Since these methods involve different adaptation mechanisms and training objectives, it is not immediately clear to me how they could be combined.) Most importantly, these closely related methods should be adequately discussed in the introduction or related work. PLD is cited in the original paper, but its differences from latent steering are not explained clearly. I could not find a corresponding discussion of RLT or AAC. If there is insufficient time to add experiments comparing EQRL with PLD, RLT, or other lightweight VLA adaptation methods, please at least discuss these methods in the introduction or related work and incorporate the relevant explanations into the paper.
Q1-3. These responses largely resolve my questions. Please include the training-time information and its accounting scope in the paper or appendix.
Thank you again for the clarifications and additional results. The points above are my remaining concerns. I see value in the problem addressed by the paper, and a convincing resolution of these concerns could lead me to raise my score. For now, I am keeping my current score.
Response to Reviewer WS92 (Part 1/2): New five-seed results and robot uncertainty
Thank you for your careful review. We address your comments in 2 parts.
W1. Five-seed statistics and real-robot uncertainty
We ran a new set of experiments on all four tasks, with five seeds per method. The table below reports mean success and standard deviation at 50k, 100k, 250k, and 500k SAC updates. Means are percentages and SDs are percentage points, showing the variability across seeds.
Table 1 | New four-task, five-seed checkpoint statistics, also reported in our response to rFA8 W3
At 500k updates, both methods achieve mean success above 93% on all four tasks. EQRL has higher mean success on three tasks and is about one percentage point lower on Tomato Sauce. The per-task, per-checkpoint SDs make the seed-to-seed variability explicit.
Robot evaluation covers five tasks with 20 complete trials per method and task, totaling 200 trials. Failure at any stage counts as failure of the entire trial. Nominal 95% Wilson intervals computed from the observed success counts are:
We provide the success counts and 95% Wilson confidence intervals for each task to show the estimation uncertainty with a limited number of trials. In this evaluation, EQRL's success rate is higher than DSRL's on four tasks and tied on one, while requested NFE is 31–34% lower and mean execution time is 11–23% shorter on all five tasks (Table 5 of the paper). These experiments therefore demonstrate lower computation cost and faster execution alongside comparable or higher observed success rates.