- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes PA-WAM: Progress-Aligned World-Action Model, for Long-Horizon Robot Manipulation. It essentially creates a progress "exemplar history" from task-specific demonstrations, and use that to guide the WAM to generate the new future representation token for action decoding. The core attention mechanism for modulating the latent action queries seems to be flawed. The design concept raises concerns on how generalisable this approach is, especially for challenging cases where unfamiliar progress structures or recovery behavior might be required, as PA-WAM's progress prior may become unreliable outside the successful demonstration distribution, and this is not sufficient address in the empirical evidence.
Critical strengths
- The motivation behind the proposed method is straightforward, and reasonable. Extracting exemplar progress + transition targets from the collective demonstration trajectory and using them to guide the World model for future generation conceptually grounds the WM's future prediction be more "state-aware", hence improving the WM's capability in assimilating "dominant behaviors under similar conditions" while generating future rep tokens for a specific task.
- The evaluation shows promising results in simulation-based benchmarks and three real-world manipulation tasks.
- The ablation study includes a broad range of useful controls.
Critical weaknesses
- The attention mechanism defined in Sec4.3 raises serious questions - with identical keys and values, it degenerates to a broadcast conditioning vector as the query gets cancelled in the derivation, meaning that the results of the attention is irrelevant to the input query, and every query just receives the same conditioning vector as long as C^p_t remains the same.
- Most of the reported improvements (over strongest baselines) are modest, and uncertainty reporting is insufficient.
- The core design principle triggers concerns over generalizability e.g. recovery after failed grasps, object displacement, etc. CALVIN ABC→D provides some evidence of environmental generalization, but does not directly resolve this concern. Would be great to have a focused experiment to validate the flexibility/generalizability of PA-WAM.
Questions for authors
- Why replicate the condition vector as both K and V for the attention?