- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper talk about ChunkRL, which trains short-, medium-, and long-horizon action experts through horizon-specific AdaLN modules on a frozen VLA-Adapter, then learns an observation-conditioned selector over horizons with group-relative policy gradients. It also introduces Adaptive Layer-Pruning (ALP), which skips policy layers at a horizon-dependent rate based on VLM residual-norm importance, and reports success and speed gains on LIBERO, VLABench, and CALVIN.
Critical strengths
Treating the chunk horizon as a state-dependent feedback-scheduling decision, rather than a fixed inference hyperparameter, is a reasonable framing. The paper motivates it clearly with the contrast between free-space motion and contact-rich stages.
The probability-swap and horizon-sequence-shuffle controls in the appendix go beyond aggregate numbers. They give some evidence that the selector’s benefit depends on when each horizon is used, not only on the marginal horizon distribution.
The method is lightweight, with a frozen backbone, AdaLN-only experts, and an MLP selector. It reports consistent wall-clock speedups across three benchmarks at a 0.5B scale.
Critical weaknesses
Need a real machine experiment to test the performance.
The reward design largely hard-codes the intended behavior instead of learning it. Every decision in a failed episode is penalized for being long, and every decision in a successful one for being short, regardless of whether that decision caused the outcome.
Pruning error corrupts the first actions of a chunk equally at every horizon, so frequent replanning limits how long the error persists, not how large it is. The paper also never validates that VLM residual norms predict the importance of the aligned policy layer: there is no comparison against random or fixed-layer skipping.
The baseline is a fixed chunk of 8 rather than a tuned horizon. Fixed h = 16 has the same runtime as ChunkRL in Table 9.
Episode-level time is confounded by episode length. The paper’s own explanations conflict: Table 5 argues failures shorten episodes, yet fixed h = 64 in Table 9 is slower than h = 32.
Adaptive-chunking methods (AAC, MoH) are not compared on the same backbone.
Questions for authors
See Weakness
3 comments
Response to Reviewer ZYzn (Part 2 of 2, Q4–Q6)
Q4. Is fixed h=8 an adequate baseline when fixed h=16 has similar runtime?
R4. Our comparisons extend beyond fixed h=8. Table 10 in the revised manuscript (Table 9 in the original submission) also evaluates fixed h=16,32,64 using the same 0.5B backbone and evaluation protocol. On LIBERO-Goal, fixed h=16 and ChunkRL have similar mean inference times, but ChunkRL increases success from 97.4% to 98.6% (+1.2 percentage points). Thus, even in this configuration with nearly identical inference cost, adaptive selection achieves higher success.
This configuration-specific near-equality in cumulative inference time does not imply equal speed across other settings. Our additional component ablations compare both strategies with ALP enabled:
| Suite (both with ALP) | Fixed h=16 success (%) | ChunkRL success (%) | Fixed h=16 time (s/episode) | ChunkRL time (s/episode) |
|---|---|---|---|---|
| LIBERO-Object | 99.4 | 99.6 | 0.72 | 0.62 |
| LIBERO-Goal | 97.4 | 98.6 | 0.79 | 0.86 |
| LIBERO-Long | 94.6 | 98.2 | 2.39 | 2.86 |
On Object, ChunkRL increases success from 99.4% to 99.6% while reducing inference time from 0.72 to 0.62 s, a 13.9% reduction. On Goal and Long, it improves success by 1.2 and 3.6 percentage points, respectively, with an additional 0.07 s and 0.47 s of cumulative inference time.
These results show that the runtime relationship varies across tasks and configurations, while ChunkRL achieves higher success on all three suites above. The similar times in Table 10 are specific to that configuration and do not establish equivalence between fixed h=16 and adaptive selection; their benefits should be assessed jointly through success and cumulative inference cost.
Q5. Is episode-level inference time affected by episode length, and why is h=64 slower than h=32?
R5. Episode-level inference time is affected by episode length. We apologize for previously attributing Table 5's timing results incorrectly to early termination of failed episodes. We have revised that explanation.
Simulation inference time is the sum of all policy-call durations within each episode, averaged across episodes. Under sparse feedback, unsuccessful attempts that continue until the episode step limit accumulate additional control steps, offsetting the savings from lower call frequency. Episode inference time therefore depends on both call frequency and episode length.
This explains why fixed h=64 accumulates more inference time than h=32 in Table 10 (1.024 vs. 0.465 s). Table 5 shows the same trend: adding h=128 to {8,16,32,64} lowers success from 93.8% to 78.2% and increases mean inference time from 0.507 s to 0.582 s.
Q6. How does ChunkRL compare with AAC and MoH on the same backbone?
R6. The AAC and MoH results in Table 1 used 3B backbones. We therefore adapted both methods to the same pretrained 0.5B VLM backbone used by ChunkRL. AAC uses a flow-matching action head and selects the execution horizon based on action entropy. MoH learns to fuse predictions across horizons and uses their agreement to determine the executed prefix length. Each baseline was evaluated over 500 episodes per suite.
| Method (0.5B VLM) | Goal success (%) ↑ | Goal time (s/episode) ↓ | Object success (%) ↑ | Object time (s/episode) ↓ |
|---|---|---|---|---|
| AAC | 4.40 | 15.368 | 92.00 | 2.368 |
| MoH | 86.40 | 0.697 | 97.60 | 0.729 |
| ChunkRL (paper) | 98.60 | 0.864 | 99.60 | 0.622 |
ChunkRL achieves the highest success on both suites, exceeding MoH by 12.2 percentage points on Goal and 2.0 percentage points on Object. On Object, it also reduces inference time from 0.729 to 0.622 s/episode. On Goal, the higher success rate comes with higher inference cost: 0.864 vs. 0.697 s/episode. Thus, ChunkRL improves both success and inference time on Object and offers a different success–time trade-off on Goal.
Invitation for Further Discussion
We sincerely thank you for your careful review and constructive feedback, which helped us identify shortcomings in the original manuscript. We have carefully revised the relevant sections and provided detailed, point-by-point responses to your concerns. We hope these revisions and the additional evidence provide a clearer and more complete basis for evaluating our work.
We would be grateful if you could take some time to review our responses and the revised manuscript. If any questions or concerns remain, we welcome further discussion and will make every effort to address them thoroughly.
Thank you again for your time, effort, and valuable feedback.
Response to Reviewer ZYzn (Part 1 of 2, Q1–Q3)
Thank you for the constructive feedback. ChunkRL jointly adapts feedback frequency and policy computation with an observation-conditioned RL selector over horizon-specific experts and horizon- and input-dependent Adaptive Layer-Pruning (ALP), improving the success–efficiency trade-off. Anonymized real-robot demos
Q1. Can the method be validated on a real robot?
R1. We tested an ARX5 robot on RoboChallenge Table30 V2's
press_the_button: four buttons in sequence, with 10 valid trajectories per strategy. All used frozen DexmalDM05-Table30v2-ARX5(revision29c433e). We use the paper's method, including adaptive horizon selection and ALP. We initialize the selector with 1,020 real-robot demonstrations, then fine-tune it with real-robot GRPO under the paper's settings. Limited platform models and DM0.5's poor eight-step performance motivated {16,32,48}.Runtime spans the first observation request to the last robot request, including observation, communication, inference, execution, and waiting, averaged over all 10 trajectories (successes and failures).
Versus the best fixed configuration (32), success rises from 50% to 60% and mean runtime falls by 26.32 s (13.3%), demonstrating real-robot effectiveness with system overhead included.
Q2. Does the reward hard-code horizon preferences rather than assign credit to individual decisions?
R2. One reward is computed after each episode and shared by all its horizon decisions. It combines the episode-outcome and feedback-efficiency terms. Group normalization assigns these decisions the same advantage; “per-decision” refers to the policy-gradient terms for their observation-conditioned horizon choices. The reward defines the success–compute objective; the selector learns the observation-to-horizon mapping through complete rollouts.
Table 9 compares ChunkRL with five shuffles preserving source horizon composition, using the same experts and ALP. The results support state-dependent scheduling.
Q3. Does replanning reduce pruning error, and do VLM residual scores predict policy-layer importance?
R3. Pruning error. Frequent replanning corrects accumulated state drift through fresh observations; it does not reduce the initial action perturbation. In a 1D tracking experiment, the controller predicts 32 actions, distributes correction of the observed position error across them, and executes the first h before replanning.
For each of two Gaussian perturbation models with equal per-step variance—trial-constant bias plus independent step noise, and independent step noise alone—we ran 1,000 paired trials. Paired conditions share each trial's perturbations and the first eight executed actions. A matched-frequency control replans every eight steps using unperturbed reference positions. We report mean absolute position deviation over steps 9–32.
At h=8, actual versus reference feedback reduces deviation by 21.35% under persistent bias (95% paired bootstrap CI: 21.26%–21.43%) and 18.15% under independent noise (17.36%–18.90%). Reference feedback matches h=32. This shows reduced accumulated state deviation under identical perturbations.
Layer importance. We skipped each eligible policy layer individually and measured action deviation from the full-depth prediction on the same observation. Larger VLM residual scores corresponded to larger deviations. All six Spearman correlations exceeded depth-matched permutation controls (p≤0.010, one-sided).
We compared skipping strategies using the same layer budget per horizon and the same 500 initial states per suite.
The single-layer test supports ranking layers by skipping sensitivity. ALP exceeds the strongest tested rollout baseline by 82.0/87.8 pp on Object/Goal.