- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a specific mechanism for encoding task progress or in state information into a conditional policy (VLA/diffusion policy). They show results on the LIBERO benchmark.
Critical strengths
Overall this paper is pretty well written. It's very nice that the authors included variant statistics on their experiments, and the figure 5 ablations are good. Similarly figure 7 is a nice analysis to show the speed improvements.
Critical weaknesses
The motivation in this paper is slightly strange. In the introduction, the authors mentioned that if a policy goes OOD then performance degrades, which is true. Then they immediately discuss that the action generator does not know that a transition is imminent. It’s not clear how these two points are connected. Why does the action generator need to know the phase transition explicitly? Why not let the policy learn that implicitly? And how does that cause the OOD behavior to be better? Certainly even if the policy knows where the phase transition is, if it goes OOD performance will degrade.
As such, I'm not sure if these experiments are entirely fair. The LaDyP results are incrementally better than the clock results and within the margin of error for clock (and several null adapter results). The LaDyP network has access to a longer history (32 steps) than the baselines and one might ask what the performance would be like if instead of this specific structure for how that history is used, one simply gave the policy the 32 frame history (or a smaller recurrent head to compress that 32 frame history if it's intractable to consume it in the diffusion body). If LaDyP is truly a generally valuable addition presumably the authors could add it to OpenVLA-OFT to set SOTA score on this benchmark?
Finally this paper would be much stronger with some real world experiments showing that the improvements are transferable.
Questions for authors
“no shuffled, clock, discrete-option, or clustered-segment signal reaches LaDyP, and a 32-step delay removes most or all of the gain. Failure attribution locates the change at engagement, release, and sub-task switches.” This is unclear and should be re-written.
Hi everyone, please do acknowledge the authors’ rebuttal with a brief reply, even if your assessment is unchanged.