- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MemAlign, a post-training framework for improving memory–action alignment in memory-augmented VLA models. It combines counterfactual supervision with history-aware action modulation, and optionally uses visual rebinding for spatial grounding. Experiments on MIKASA-Robo and RoboMME show improved performance on memory-dependent manipulation tasks.
Critical strengths
1.The paper identifies a clear and meaningful problem. Memory–action misalignment is well motivated, and the original/replaced/erased history evaluation gives a useful way to test whether the policy is actually using memory to guide its actions.
2.The improvement on the controlled MIKASA setting is substantial. MemAlign significantly improves over the locally evaluated MemoryVLA baseline, showing that the proposed approach can effectively redirect actions based on historical information.
Critical weaknesses
1.The comparison with existing methods is not fully fair. The strongest MIKASA results use a controlled evaluation protocol that differs from the protocols used by the published baselines, so the claimed advantage over prior methods is hard to judge directly.
2.It is unclear how much of the gain comes from the core MemAlign design. For example, the RememberColor results also rely on a task-specific visual rebinding and metric grounding pipeline, while the main Shell result uses a different warm-start and recovery-data setup from the ablation study.
3.The evidence for general memory–action alignment is still limited. The strongest results mainly come from controlled target-selection tasks with fixed cue timing, while the gains on the broader RoboMME benchmark are relatively small and uneven across task categories.
4.There is no real-robot evaluation. All experiments are conducted in simulation, leaving the effectiveness and practicality of MemAlign in real-world deployment unclear.
Questions for authors
1.Why are the strongest memory baselines not re-evaluated under the same controlled MIKASA protocol as MemAlign? This would make the comparison with prior methods much more convincing.
2.The main Shell result reaches around 90%, while the full model in the matched ablation achieves only about 52–56% and uses a different training setup. Can the authors provide ablations under the same setup as the main Shell result?
3.How exactly is MME-VLA + MemAlign implemented and trained in Table 3? The paper reports an improvement from 44.51% to 45.62%, but the corresponding adaptation and training details are unclear.
7 comments
W3. Scope of the improvement and broader memory capabilities
We agree that MemAlign achieves its strongest improvements on the target-selection tasks in MIKASA-Robo, while its gains on RoboMME are more modest and uneven across task categories.
Why MIKASA-Robo is the primary setting. These tasks allow a direct test of memory–action alignment. We hold the current observation and instruction fixed and change only the historical cue. We can then check whether the policy selects the target that this history specifies. This is an interpretable test of exactly the behavior MemAlign is designed to improve.
Evidence beyond controlled cue timing. RoboMME does not use our controlled handover: cues appear at their native times and are followed by unconstrained robot motion. Even so, the gains follow the pattern we would expect.
- They are largest on Reference tasks (+8.5 pp), where history determines which target to act on.
- They are smaller on categories that depend on counting or imitation, such as Counting (−0.5 pp).
- Average success improves for all three memory-augmented VLAs we evaluated (Table 3).
Counterfactual supervision currently covers all four Permanence tasks and three Reference tasks (PickHighlight, VideoPlaceButton, and VideoPlaceOrder). VideoRepick and all Counting and Imitation tasks are trained without the auxiliary counterfactual objective (see our response to Reviewer 1Cz8, W2).
Revised claims and limitations. We have revised the abstract and introduction to state that MemAlign primarily improves history-dependent target selection, rather than memory capabilities in general. App. G.2 now explicitly states that the MIKASA results rely on controlled cue timing and do not establish benefits across all memory types. In future work, we plan to construct counterfactual supervision for more diverse memory requirements and to evaluate transfer on further memory benchmarks.
Q3. Implementation and training details for MME-VLA
We thank the reviewer for this question. We have expanded App. D, and summarize the key details below.
Architecture. We use the released MME-VLA (perceptual Frame Sampling + Modulator) and keep all of its parameters frozen: vision, language, memory, and action expert. MemAlign adds two modules:
- History controller. A two-layer, width-256 Transformer reads MME-VLA's native per-frame perceptual and memory tokens. The current frame is excluded, so the controller sees only past frames. It outputs coefficients based on how the action response under the actual history differs from that under a fully masked history. With no history, the coefficients are zero.
- History-aware low-rank modulation. Rank-8 adapters are applied to the 18 memory-modulation projections and the 18 action FFN layers. Their strength is scaled by the controller's coefficients.
The adaptation adds 3.1M parameters. It does not use visual rebinding.
Counterfactual supervision. Each training pair shares the same current observation, robot state, and instruction, but has two different valid histories, each with its own expert continuation. The loss combines MME-VLA's native flow-matching loss with a counterfactual term. This term trains the policy to match each history's expert action and to change its action when the history changes. An erased-history teacher keeps behavior unchanged when no history is available.
Training. We train for 224 AdamW updates (learning rate 1e-4). Each update uses one counterfactual pair and two ordinary samples, weighted 0.25 / 0.75. Checkpoints are selected on development data only.
Evaluation and the updated numbers. The original submission compared our result (45.62%) with the published MME-VLA score (44.51%). In the revision, we rerun the released MME-VLA checkpoint locally under the same protocol as our adapted model: all 16 tasks, 50 episodes each. Under this matched comparison, MemAlign improves MME-VLA from 45.38% to 46.50% (Table 3).
W4. Real-robot evaluation
We thank the reviewer for emphasizing real-world validation. We have added a real-robot evaluation to the revised manuscript (Sec. 5.4, App. F).
Setup. We use a PiPER-X arm with a fixed RealSense D435 camera and a wrist-mounted D405 camera, both recording 640 × 480 RGB through ROS 2. The policy takes both camera views, proprioception, the instruction, and visual history as input, and predicts 16 future joint targets on a 10 Hz control grid. Training uses 108 demonstrations per task and has two stages:
- Stage 1 adapts the native diffusion action expert, conditional LoRA, and the wrist/embodiment adapters for 20,000 updates, with an effective batch size of eight.
- Stage 2 freezes the native action expert and wrist/embodiment adapters, and trains conditional LoRA, the history controller, and the coefficient generator for up to 3,000 updates with action fitting and counterfactual supervision.
Vision and memory remain frozen in both stages. Training uses four L40S GPUs with DDP; the native action expert is updated in Stage 1 and frozen in Stage 2 (App. F.3, Tables 20–21).
Tasks.
- RememberColor3: the robot sees a transient color cue, then touches the block of the matching color among three candidates.
- ShellGameTouch: the robot observes which cup hides the ball, waits, and then touches that cup.
We run 10 trials per task and history condition, and all methods use the same held-out layouts.
Table R3.3. Real-robot exclusive target completion (successes / 10 trials). The MemoryVLA baseline is adapted to the same embodiment with the same 108 demonstrations per task, without history-aware modulation or counterfactual supervision. Erased removes the cue and is scored against the original target; it is a diagnostic, not a success metric.
| Task | Method | Original ↑ | Replaced ↑ | Erased |
|---|---|---|---|---|
| ShellGameTouch | MemoryVLA | 4/10 | 2/10 | 3/10 |
| ShellGameTouch | MemAlign | 6/10 | 4/10 | 2/10 |
| RememberColor3 | MemoryVLA | 1/10 | 2/10 | 1/10 |
| RememberColor3 | MemAlign | 5/10 | 3/10 | 1/10 |
Results. On ShellGameTouch, MemAlign improves Original completion over MemoryVLA from 4/10 to 6/10. Under replaced histories, it redirects 4/10 trials to the new target, compared with 2/10 for MemoryVLA. Erasing the cue reduces MemAlign's original-target completion from 6/10 to 2/10, consistent with sensitivity to the cue history. On RememberColor3, MemAlign improves Original completion from 1/10 to 5/10 and Replaced completion from 2/10 to 3/10; its original-target completion under erased histories is 1/10. Failures generally fall into two categories: the gripper is not correctly oriented toward the target object, or it is oriented toward the target but fails to make proper contact. This can be observed in the videos provided in the supplementary materials.
Practicality. The 16-step profiling results in App. F.2, Table 19 show about 11% additional query latency and 0.8 GiB of peak GPU memory relative to MemoryVLA. The measured RC5/Shell query times are summarized in our response to Reviewer Zzha, W1 & Q2. On the real robot, RC3 query latency is 497.65 ms. The policy predicts 16 joint targets on a 10 Hz temporal grid, and RTC overlaps inference with buffered action execution. Local interpolation provides joint-target commands at 30 Hz, separating the command publication rate from the approximately 2 Hz compute-limited policy query rate.
We present these results as a feasibility study rather than a full real-world benchmark, and we note this in the limitations. Larger-scale real-world evaluation is an important direction for future work.
Response to Reviewer RKCn
We thank the reviewer for the insightful and constructive comments! In the following, we provide our point-by-point response and hope our response helps address your concerns. We also look forward to the subsequent discussion which may further help to solve the current issues.
W1 & Q1. Evaluation under the original and controlled protocols
We thank the reviewer for raising this point. We agree that the original manuscript did not clearly separate the two evaluation protocols. We have now evaluated both MemoryVLA and MemAlign under both protocols.
Why a controlled protocol. In MemoryVLA's original setting, the policy can already act while the cue is still visible. Actions planned during this period can carry over after the cue disappears, and the robot's cue-driven movement can itself encode the target. Success therefore mixes two effects: using memory, and continuing actions started while the cue was visible. Our controlled protocol adds a stationary cue-observation phase: the robot observes the cue without moving and only acts after the cue disappears. This isolates whether the target choice is driven by memory. Both policies are evaluated under the same protocol.
Table R3.1. Success rate (%) with and without the stationary cue-observation phase (mean ± std over three evaluation blocks).
Experiments under both controlled and original protocols. To make the comparison explicit, we evaluated MemoryVLA and MemAlign under both protocols (Table R3.1). Under the controlled protocol, MemAlign raises the average success rate from 15.00% to 87.83% and improves every task. Under the original protocol, MemAlign still improves the average from 39.67% to 50.50%, with consistent gains on all three RememberColor tasks.
The exception is ShellGameTouch under the original protocol, where MemoryVLA scores higher than MemAlign (86.67% vs. 77.33%). This result also helps explain why the protocols differ. MemoryVLA's 86.67% under the original protocol is consistent with its published 88%, yet the same model drops to 16.67% under the controlled protocol. This suggests that its original performance mainly comes from actions started while the cue is still visible, rather than from using memory after the cue disappears. The controlled protocol removes this shortcut and therefore isolates the memory-dependent target selection that MemAlign is designed to improve. This also accounts for the gap between the published MemoryVLA score in Table 1 and our controlled result in Fig. 3.
In the revision, we report both protocols side by side and clearly separate matched local comparisons from published baseline results obtained under their source protocols.
W2 & Q2. Ablations matched to the main experiment
We thank the reviewer for identifying this issue. The original ablation study and the main ShellGameTouch experiment started from different checkpoints. We acknowledge that this mismatch limited their direct comparability and prevented a fully controlled assessment of component contributions under the main configuration. To address this concern, we have rerun the ablation study using the same initial checkpoint, training data, training budget, and action-sampling settings as the main experiment. The updated results and detailed experimental setup are provided in the appendix. We have added this experiment to the manuscript in Line 471, Table 4.
Table R3.2. Matched component ablation on ShellGameTouch. Exclusive completion (%) ↑.