- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work introduces LaMem-VLA, a new design in the family of memorizable robot generalists. The core is to take the representation of VLA as a nature to construct so-called memory, and reconstruct the historical experience into a latent part that carries memory and guides reasoning. The method consists 4 components that sequentially and explicitly generate and utilize these latent memory tokens in the same representation space for memory-based action generation. The experiments show the effectiveness in both simulation benchmarks and real-world tasks of different purpose (generalizable and long-horizon.)
Critical strengths
-
As a new way to explore memory utilization in robot generalists, this work seems clear in its process pipeline and interests people to research on the native representation rather than adding other external modules while some comparison may lack further analysis (see Weakness 1). The insight is rather clear and well motivated.
-
The experiments are conducted properly, and the presentation is detailed. The improvements on MIKASA-Robo and real-world long-horizon tasks are encouraging. The ablation and choice sweeps help me understand the mechanism and clarify the contribution of this framework.
Critical weaknesses
-
For method novelty, as far as I know, the method itself is not novel enough. After removing decorative expressions, the essence of LaMem-VLA is to move the injection point of memory from "after-VLM" (the work MemoryVLA by Shi et al.) to "before-VLM". The backbone, dual memory back, and specific sequence memory processing all follow MemoryVLA.
-
For method techniques and motivation. Firstly, the claimed point of pushing memory into the VLM input sequence is well explored, as the authors also classified. Recent works like ContextVLA (Jang et al.) also used similar techniques. Well the author didn't compare with this method. And the "short term" and "long term" description didn't have support in the paper, the capacity are same, two vaults are same in time scale. The labelled "milestones, progress, goals, semantics" in Figure 2 also lack explanation and analysis. The description of memory building and latency analysis confused me of how it work, is it 2 times VLM forward?
-
For the experiment. The evaluation results as claimed are selected alone, if I understand this correctly, all the simulation results of LaMem-VLA are tuned and tested while MemoryVLA lacks a fair comparison. If the training budgets are different, the gain is relatively small and not promising.
-
For writing, this paper includes many terms that distract my attention while reading, and I can't understand the proposed framework after many times of back-and-forth checking and alignment. The logic and technical framework should be better positioned and written clearly.
Questions for authors
-
Regaring the method, Section 3.4 states that the VLA backbone produces Q^con_t from X_t and I, and Eq. 8 then processes S_t with the VLM again. Is Q^con_t obtained from a full LLaMA-7B forward pass on the memory-free sequence, or from the vision encoder or input embeddings? Is the H^action written to the long-term vault taken from that pass or from VLM(S_t)? Please also give a latency breakdown for Table 11 (backbone, memory modules, DiT), and state whether N_con is the full token sequence.
-
The claimed difference from MemoryVLA and ContextVLA is the injection point. Can you provide a controlled comparison in which the same memory content (same vaults, retrieval, and condenser) is injected (a) before the VLM as in LaMem-VLA, (b) after the VLM as in MemoryVLA, and (c) as compressed past-frame tokens as in ContextVLA, all trained under the same protocol? Without this, the benefit cannot be attributed to the proposed mechanism rather than to extra training or checkpoint selection.