- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work presents an action-conditioned generative model called GWS. It addresses the error accumulation problem encountered by previous methods during long-horizon generation. The work makes three key contributions: (1) a memory mechanism combining short-term and long-term memory, (2) a keyframe selection mechanism for long-term memory, and (3) a model architecture. The results are promising and establish a SOTA.
Critical strengths
The results look good, especially the live demo. However, these examples use fixed inputs, so it is unclear how well the model generalizes to other OOD cases.
Critical weaknesses
- The manuscript appears to be at an early stage and contains numerous errors, such as “ABD datasets()” on L261, and “SSDD-M()” and “decoder ()” in the implementation details.
- The rationale behind all design choices is missing. For example, why use unit-sphere normalization, VLM-based supervision for contact prediction, Diffusion Forcing, a learned sink frame?
Questions for authors
-
Why does error accumulation occur when an action serves as a condition? Does it refer to an appearance shift or a geometric shift? An illustrative figure would help clarify this.
-
The rationale behind the unit-sphere tokenizer is unclear. Why use unit-sphere normalization? Is this normalization applied to ? I am unsure because you refer to as a latent representation but state that normalization is applied to tokens. Consistent terminology would improve readability. What does it mean to renormalize tokens and reuse them as context?
-
The sample is very confusing. What does the upright index mean? There is a missing reference on L197. Which pretrained encoder does this work use? Why not freeze it and train the flow decoder?
-
How is the VLM for contact transition prediction fine-tuned? How do you obtain the ground-truth labels? Why not simply use these data to train the additional head?
-
What does “ABD datasets()” on L261 mean?
-
Would it be possible to show an attention map for the learned sink frame? How does it support long-horizon tasks? What is the architecture of SSDD-M()? What does “decoder ()” refer to?