- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper revisits why action chunking improves imitation-based manipulation policies. Predicting an action sequence and executing a long prefix open-loop has driven much of the recent progress, but a long open-loop prefix reduces reactivity, and the source of the performance gain is unclear. The authors argue that long open-loop execution mainly helps short-context policies imitate non-Markovian demonstrations. Across four simulation tasks and two real-world tasks, expert non-Markovianity shapes the relationship between success and the open-loop horizon.
Critical strengths
-
The paper studies why long open-loop execution helps imitation learning, a question that bears directly on how manipulation policies are trained and executed. Each experiment is constructed with strong motivation, including non-Markovian demonstrations, compounding error, and context length, making the argument easy to follow.
-
The experiments separate expert non-Markovianity from compounding error, and compare influence of each factor on the execution horizon. Carrying that comparison through four simulation tasks and two real-world tasks gives the claim a solid basis.
-
The empirical results are convincing, and the analysis of these results provides a helpful and practical guidance for the design of observation horizon and execution horizon.
Critical weaknesses
-
Some experimental details are missing. ACT introduced Temporal Ensembling, under which the policy predicts reactively before the full action chunk has been executed. The paper does not appear to state whether this design is used. Without that information, it is difficult to judge how much reactivity the evaluated policies actually give up.
-
The reasoning in Section 4.2 needs further explanations. It assumes that if a long execution horizon could mitigate compounding errors, mitigating compounding errors (by increasing data scale) should disproportionately improve performance at short execution horizons. The step from the first statement to that prediction is not spelled out, so readers may be confused about the expected change in the curve.
Questions for authors
-
Do the ACT and Diffusion Policy experiments use temporal ensembling? Temporal ensembling allows policies to reactively predict actions without waiting until every action in the prediction horizon has been executed. Does that weaken the paper’s motivation concerning reduced reactivity?
-
In the left plot of Figure 4, increasing the data scale does not change the shape of the curve, which agrees with the paper’s analysis. However, reducing the number of demonstrations from 1000 to 400 and then to 200 reduces state coverage and would be expected to increase compounding error. At N_demo = 200 and 400, where compounding error should be larger, lengthening the execution horizon brings performance close to that of the policies trained with 1000 demos. Does this indicate that a long execution horizon could mitigate compounding error, as stated in prior works?
-
In the center plot of Figure 6, why does increasing the context length gradually reduce the success rate? What distinguishes this setting from the other two settings in that figure?
3 comments
We now comment on the feedback for figure 4 and compounding errors analysis:
- Figure 4 and Compounding Errors Analysis: We thank the reviewer for pointing out the analysis of the N=400 curve with the non-Markovian expert, which is currently missing direct commentary in the text. Overall, our analysis from this section is that compounding errors do help (as reported in prior works), but we do not see the effect as being as strong as what we see from expert Markovianity. For instance, for Figure 4 left (scripted non-Markovian), we see the optimal execution horizon as: 6 at N=100, 6 at N=200, 10 at N=400, and 8 at N=1000. If compounding errors were indeed being significantly mitigated by longer execution horizons and were the driving force behind longer execution horizons, then increasing data (akin to lowering compounding errors) should have lowered the optimal execution horizon (but here it in fact seems to raise it slightly). However, if we switch the nature of data (Figure 4 right) to a Markovian expert, the optimal execution horizons are: 3/4 at N=100, 1 at N=200, 3 at N=400, and 1 at N=1000. Thus, fixing the expert and increasing data (mitigating compounding errors) barely shifts the execution horizon to smaller values, but fixing the data and changing the expert shifts it significantly toward smaller values in all cases. This leads to our analysis that compounding errors do matter, but not as much as other factors. Our HG-DAgger experiment confirms this. That said, we will add the reviewer’s suggested comparison between the N=400 and N=1000 curve of the non-Marlovian expert at longer execution horizons. This will strengthen our claim that in our experimental setting compounding errors do have an effect as shown by limited evidence, but not as strong as that of expert Markovianity.
- Clarity of Section 4.2: We appreciate the feedback on this. We believe this is linked to the question about the analysis for Figure 4. Given the additional page for revision, we will add a summary of our discussion about Figure 4. We hope this will help address clarity concerns of Section 4.2. Overall, we are claiming that controlling for the factor that more strongly impacts execution horizon length should shift the shape of the success horizon curve more sharply. While the effect of compounding errors is only seen modestly, the effect of changing expert nature is seen strongly across experiments.
I appreciate the authors' detailed explanation and engagement, which has addressed my concerns regarding Temporal Ensembling and clarity of Section 4.2. I will therefore maintain my current score.
We thank the reviewer for their feedback and questions. In our first comment, we will discuss temporal ensembling and Figure 6. In the next one we will address concerns about Figure 4 and compounding errors section:
[1] Zhao et al. 2023, https://arxiv.org/abs/2304.13705
[2] Chi et al. 2023, https://arxiv.org/abs/2303.04137
[3] TRI LBM Team 2025, https://arxiv.org/abs/2507.05331
[4] Physical Intelligence, https://github.com/Physical-Intelligence/openpi