- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes FlowMem, a non-parametric memory mechanism for latent-reasoning VLA models (built on LaST0) that segments latent trajectories from successful training episodes into fragments, retrieves and recomposes trajectory suffixes via constrained search, refines them with cross-attention, and injects them into the flow-matching action expert. Experiments on RoboMME and LIBERO-Plus report improvements in success rate and lower per-episode inference time over LaST0.
Critical strengths
- Storing and recomposing latent reasoning trajectories rather than raw observations, action chunks, or text subgoals offers a clean and well-motivated perspective on non-parametric memory for VLA policies.
- The counterfactual memory protocol (testing wrong-task, shuffled-order, and mismatched-progress memories) is a sensible experimental design, and the 4.1-point gain on LIBERO-Plus is evaluated over a large sample (10,030 instances).
Critical weaknesses
- On RoboMME, both the overall gain over LaST0 (48.0% vs. 46.3% on 800 episodes) and all mechanism-level ablations in Tables 3–5 lie well within single-run binomial evaluation noise (≈1.8% standard error per method), and the benchmark itself does not match the proposed mechanism. Every counterfactual and ablation variant falls between 44.0% and 46.3%, with wrong-task memory (46.1%) and no-refinement (46.3%) performing virtually identically to the no-memory LaST0 baseline (46.3%). Moreover, RoboMME tests within-episode working memory (e.g., counting or object permanence in the current rollout), whereas FlowMem retrieves only from past training episodes based on the current observation and state. Consequently, the RoboMME experiments cannot substantiate the paper's core claims that trajectory ordering, task identity, or progress alignment drive policy performance.
- On LIBERO-Plus, the 4.1-point gain is not isolated from extra training and added parameters, and key algorithmic details are missing. The counterfactual memory controls are not run on LIBERO-Plus, making it unclear whether the gain over LaST0 stems from retrieved latent memories or simply from the extra cross-attention module and additional finetuning stage. In fact, the largest gains occur under camera (+8.2) and noise (+10.8) shifts—where visual retrieval should be least reliable—while almost no gain appears under robot initial-state shifts (+0.3), where trajectory recomposition should help most. It is also unclear why training refined memory tokens to reconstruct LaST0's frozen latent enables FlowMem to outperform LaST0, and core details (fragment segmentation, constrained search, and memory bank composition) are omitted, preventing reproduction.
Questions for authors
- Could the authors run the counterfactual memory controls (wrong-task memory, shuffled order, and a null/random-memory baseline with the same extra training steps and parameters) on LIBERO-Plus to verify whether the 4.1-point improvement comes from memory reuse?
- Since the alignment loss trains the refined memory tokens to match LaST0's native latent , why does FlowMem outperform LaST0 rather than merely approximating it, and why do the largest gains on LIBERO-Plus occur under visual noise and camera shifts rather than initial-state shifts?
- Could the authors provide multi-seed confidence intervals on RoboMME, per-step query latency (unconfounded by episode termination length), and the missing specifications for fragment segmentation, constrained search, and memory bank construction?
3 comments
Why larger gains under camera/noise shifts (W2 & Q2)? We thank the reviewer for raising this question. Under camera/noise shifts, FlowMem can reuse successful latent reasoning to guide current predictions when visual evidence is unreliable. It first filters memories by task, then retrieves relevant fragments based on the current observation and robot state. Recomposition and refinement improve over nearest-state retrieval by 3.90% under camera shifts and 2.54% under noise, versus 1.20% under initial-state shifts (Table 4). These results show larger observed benefits from the complete reuse pathway under visual perturbations. Initial-state shifts additionally require action adaptation to a new robot configuration, which latent reasoning recomposition does not directly resolve.
Table 4. Success rates by perturbation (%).
| Method | Camera | Noise | Initial-state |
|---|---|---|---|
| Budget-matched LaST₀ | 81.30 | 78.93 | 36.54 |
| Nearest-state | 84.8 | 84.6 | 35.9 |
| FlowMem | 88.70 | 87.17 | 37.14 |
Segmentation, search, and memory-bank construction (W2 & Q3). We thank the reviewer for requesting these details. We build a fixed memory bank from successful demonstrations, organize latent reasoning entries into fragments, and retrieve compatible fragments for recomposition.
- Fragment construction. RoboMME provides subgoal annotations, so we group consecutive records with the same subgoal and store one latent reasoning output per stage as a memory entry. LIBERO-Plus lacks these annotations, so we construct fragments during retrieval using windows of up to eight consecutive memory entries. Progress records each fragment’s normalized position in its source trajectory.
- Search. We filter candidates by task, score them using visual similarity and state distance, and retain the 16 highest-scoring partial sequences at each step. Annotated stages must satisfy preconditions and add new stage effects, without repeated fragments or backward progress within a source trajectory. Temporal windows preserve internal order, and cross-source connections require compatible boundary states.
- Memory-bank construction. A frozen native policy processes successful training demonstrations to extract latent reasoning. We store the selected outputs with task labels, visual descriptors, robot states, progress, and source identity and order for retrieval and composition. The banks contain 1600 RoboMME trajectories and 432, 454, 428, and 379 trajectories across the four LIBERO suites. Banks remain fixed during second-stage training and evaluation. Evaluation trajectories are excluded, and training retrieval excludes the current source trajectory.
Per-step query latency (Q3). We thank the reviewer for pointing this out. Our previous timing could be affected by episode length; remeasuring latency per query shows reductions of 49.99% on RoboMME and 43.55% on LIBERO-Plus. Each query includes retrieval, recomposition, refinement, and action prediction, excluding model loading, warm-up, and simulation. We compute mean latency per task, then average tasks equally. LIBERO-Plus uses more queries because its evaluation includes more simulation episodes (Table 5).
Table 5. Per-query latency (ms/query).
| Benchmark | Type / Suite | Queries per method | LaST₀ | FlowMem | Reduction |
|---|---|---|---|---|---|
| RoboMME | Counting | 418 | 258.4 | 124.3 | ↓51.91% |
| Permanence | 246 | 267.3 | 130.9 | ↓51.04% | |
| Reference | 319 | 425.7 | 211.1 | ↓50.41% | |
| Imitation | 291 | 281.4 | 150.3 | ↓46.60% | |
| Overall | 1274 | 308.19 | 154.13 | ↓49.99% | |
| LIBERO-Plus | Spatial | 1192 | 418.0 | 226.8 | ↓45.75% |
| Object | 1202 | 424.5 | 240.4 | ↓43.36% | |
| Goal | 1200 | 419.3 | 234.6 | ↓44.06% | |
| Long | 1222 | 417.5 | 246.2 | ↓41.04% | |
| Overall | 4816 | 419.81 | 236.97 | ↓43.55% |
Official Comment by Reviewer SDFi
Thank you for your thoughtful response and for the work you put into addressing my questions during the rebuttal period. After considering your response, I’ve decided to keep my original score.
Statistical reliability and evidence for memory reuse (W1, W2 & Q1, Q3). We thank the reviewer for raising these concerns. Multi-seed results show consistent gains, and new LIBERO-Plus ablations support the effectiveness of memory reuse. FlowMem exceeds budget-matched LaST₀ by +2.92% on RoboMME (95% CI [+0.99%, +4.85%]) and +3.43% on LIBERO-Plus (95% CI [+2.70%, +4.16%]) (Table 1). Following the RoboMME counterfactual protocol, wrong-task, out-of-order, and wrong-progress memory yield 74.8%, 75.1%, and 76.2%, versus 77.3% with correct memory (Table 2). Single-source reuse and removing refinement also reduce success (Table 3), supporting the contribution of memory organization and current-context refinement.
Table 1. Three independent training runs (success rate, %; mean ± standard).
The 95% CIs for mean differences use two-sided, unpaired Welch t intervals across independent training runs.
Table 2. Newly added memory counterfactuals on LIBERO-Plus (success rate, %).
✓ indicates a retained property; blank cells indicate corrupted properties. LaST₀ is the original memory-free baseline.
Table 3. Retrieval and composition ablations on LIBERO-Plus (success rate, %).
Valid: valid-route coverage (%); Multi: multi-source query proportion (%).
RoboMME’s relevance to cross-episode memory reuse (W1). We thank the reviewer for raising this concern. RoboMME is relevant because cross-episode latent reasoning reuse can help a robot use its current-episode history to complete the task. In PatternLock, the robot must remember the target sequence shown in a demonstration video and retrace it. FlowMem can refine latent reasoning from past successful episodes using the current context to guide execution of this sequence. On RoboMME, FlowMem achieves 48.54%, versus 45.63% for budget-matched LaST₀ (Table 1).
Why outperform LaST₀ when matching its latents (W2 & Q2)? We thank the reviewer for raising this question. FlowMem reuses successful latent reasoning from multiple past trajectories to construct current reasoning, going beyond LaST₀’s prediction from the current context alone. LaST₀ provides the source fragments, but FlowMem recomposes and adapts them to the current situation rather than reproducing a single LaST₀ prediction. Restricting reuse to a single source or removing refinement reduces RoboMME success from 48.0% to 44.0% and 46.3%, respectively (paper, Table 3). The same trend holds on LIBERO-Plus (Table 3), supporting the contributions of multi-source reuse and current-context refinement across both benchmarks.