- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work addresses the challenge that scenes that continue to change during VLA inference.
It introduces PhysClock, a dynamic-task benchmark with temporal annotated data, and trains a compact prediction head to estimate the valid activation window of each chunk.
Critical strengths
The problem is highly valuable, and the method is intuitive and effective. The addition of an effective window for predicting action blocks represents only a modest modification to existing common VLA architectures , yet it is effective.
The dynamic evaluation benchmark, expert demonstrations and annotations, and annotations form a comprehensive resource that could advance research on this problem within the community.
The setup for real-world evaluation is presented in considerable detail.
Critical weaknesses
The success-probability head and the validity-window head achieve similar results, but the implications of this finding seems somewhat vague. I believe this is an interesting and valuable finding, but it requires a clearer discussion.
Questions for authors
The labeling approach is reasonable, but obtaining the labels requires multiple simulation replays, which is quite difficult. Could lighter-weight labeling method or alternative supervision signals provide similar benefits?
Data acquisition costs are important for assessing scalability of this method. I think it’s important to discuss roughly how much data is required for each new task, and what are the associated collection and annotation costs.