acceptodds
Withdrawn from ICLR 2027

Controlled Predictive Memory for Vision-Language-Action Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language-action (VLA) models need compact memories to distinguish execution histories that require different decisions despite similar current observations. Predicting future observations can guide this compression, but their dependence on subsequent behavior complicates comparisons between histories. We introduce Controlled Predictive Memory (CPM), which learns memory by predicting expected joint outcome features under a fixed set of short closed-loop interaction programs. The fixed programs provide a common behavioral reference for comparing histories, while joint features expose selected cross-step dependencies that separate endpoint means can miss. Response prediction and action imitation jointly train a fixed-size recurrent memory from additional program interactions and expert demonstrations; the programs and response decoder are used only during training. A prediction-risk decomposition characterizes the response information lost through history compression. CPM improves average success on each of five simulation benchmarks over two history-based baselines with the same VLA backbone and matched action-update budgets. On five single-arm and two bimanual real-world tasks, CPM achieves average success rates of 73.3% and 63.3%, respectively, exceeding the stronger history-based baseline on each platform by 6.7 and 8.3 percentage points.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robotic manipulation, recurrent memory, controlled prediction, imitation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LA69OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Controlled Predictive Memory (CPM) for more robust memory compression in vision-language-action models. CPM trains a recurrent latent memory by predicting joint outcomes under fixed closed-loop programs, jointly with action imitation. Empirically, it outperforms history-based baselines on LIBERO, SimplerEnv-Bridge, RoboMME, RoboTwin 2.0, and LIBERO-Plus, as well as on real-world single-arm and bimanual tasks.

Critical strengths
  • S1. The problem is well-motivated and clearly defined. In partially observable manipulation, identical current observations can correspond to different histories, and action imitation alone does not explicitly encourage a recurrent memory to retain history-dependent information. This paper addresses it via Controlled Predictive Memory (CPM), which trains a recurrent memory by predicting joint outcomes under fixed closed-loop programs, jointly with action imitation.

  • S2. Consistent empirical gains. CPM outperforms strong history-based baselines on multiple simulation benchmarks and on real-world single-arm and bimanual tasks.

Critical weaknesses
  • W1. The advantage of the proposed predictive objective over simpler alternatives is unclear. CPM uses program-conditioned prediction of joint future observations to supervise history compression. However, the paper does not compare this objective against alternative approaches, such as action-conditioned outcome prediction or contrastive learning that distinguishes histories with similar current observations.

  • W2. The design of the control programs is insufficiently justified. The programs are constructed by fixing the flow-sampling noise of a frozen policy, yielding reproducible closed-loop behaviors. However, the paper does not investigate whether these programs induce sufficiently diverse behaviors or justify the choice of eight programs. It is also unclear whether fixing the sampling noise is important. Ablations on program construction, program diversity, and the number of programs would help clarify the necessity and robustness of this design.

Questions for authors
  • Q1. The paper compares CPM mainly against history-based baselines, but not against other forms of predictive supervision. How would it compare to action-conditioned future prediction or contrastive objectives that distinguish histories with similar observations? Without such comparisons, it is hard to tell whether the program-based formulation is necessary or just one of several viable designs.

  • Q2. The programs are constructed by fixing the flow-sampling noise of a frozen policy. How sensitive is CPM to the number and construction of these programs? Do the eight fixed-noise programs actually produce meaningfully different closed-loop behaviors? How much would performance drop with fewer programs or with a different construction strategy?

  • Q3. Fixed flow-sampling noise has no clear semantic meaning. Are there alternative ways to define the control programs that would work just as well?

  • Q4. The program library contains only eight arbitrarily chosen programs, so its behavioral diversity may be limited. Could this bias the prediction target and the learned memory? Action-conditioned prediction covers a much broader range of behaviors and avoids this selection bias. Can the authors explain why the fixed-program formulation is still advantageous?

  • Q5. Why exactly two steps? It is unclear whether two steps is a principled choice or just a convenient one.

  • Q6. The memory uses four slots, but this choice is not ablated. How does performance change with one, two, eight, or more slots? Does the number of slots interact with the number of programs?

Reviewer_XZxKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adds a small recurrent memory (four slots) to pi0.5 for tasks where the right action depends on what happened earlier. The memory is updated after every decision from the issued commands and the observations before and after execution. It is trained with an extra loss on extra "program records". A program is a short rollout of a frozen, memoryless base policy with fixed sampling noise, run for two decisions; eight such programs are used. At randomly chosen anchors in an episode, one program is assigned at random and executed, and the two resulting observations are encoded jointly by fixed random Fourier features. A decoder predicts the expectation of this feature from memory. The decoder and programs are discarded at deployment. CPM outperforms two history baselines re-implemented on the same backbone (Triplet History and FrameSamp+Modul) on multiple tasks across simulation and the real world.

Critical strengths

I see the following strengths of this paper:

  1. The paper separates two things that are often mixed in future-prediction objectives: what the history reveals, and how the policy happens to continue. Fixing the continuation rule is a reasonable way to make predictions comparable across histories, and the design cost seems satisfying.
  2. The evaluation is broad and handled with care. There are five simulation benchmarks and two real platforms. The baselines share the backbone, inputs, update budget, and evaluation initializations, and the results illustration is clear.
  3. The ablations are conducted in the right structure, and the history ablation is clearly informative.
  4. The method, theory, and training are specified well enough, the overall demonstration is good to follow
Critical weaknesses

Below is a weakness conclusion of CPM:

  1. The comparison with the baselines is matched in action updates but not in data. CPM uses extra environment interactions, with 200 training anchors and 50 held-out anchors per task in each of two rounds. The baselines see only demonstrations, and the appendix states that they "do not use this auxiliary supervision". On the real platforms this is about 400 training anchors per task against 100 to 200 demonstrations. I think the ablations do not close this gap. Removing the prediction loss also removes the data, and the shared-query variant keeps the data but still predicts a program-randomized outcome. Maybe a baseline is missing, which uses the same extra records, or same robot time.
  2. The central idea that a fixed behavioral reference matters makes sense, but is supported by small differences. The paper's own test of fixed rule against realized commands is 53.6 against 51.6 on RoboMME. What's more, I'm not sure if many of the reported differences are within the noise of the evaluation. In implementation, the eight "programs" are the same frozen memoryless policy with different noise seeds, so it is unclear how far the language of interaction programs and interventions goes beyond a variance-reduced auxiliary regression.
  3. The real-world gains are not firm. Almost the whole single-arm gain comes from two tasks S1 and S2. CPM is tied on S4 and S5. CPM also receives extra robot data (weakness 1). The task-structure explanations in the text are plausible but not tested, since nothing shows what the memory stores.
  4. The theory and the mechanism do not connect to the results. The risk decomposition is the usual split of squared error into irreducible noise, information lost by compressing the history, and decoder error. It holds for any regression from a compressed state and says nothing about which distinctions matter for action, as the appendix itself admits. Empirically, memory trained by prediction alone (A6: 66.7 and 44.6) is no better than memory trained by action alone (A3: 67.0 and 47.5), so the benefit appears only with joint training, which the analysis does not address.
Questions for authors

I have following question for authors:

  1. Can the authors add baselines that use the same extra program records, or the same robot time, in a conventional way, for example future-feature prediction on demonstrations only and future-feature prediction on the program records without program conditioning and joint features?
  2. How do results change with the number of programs (one, two, four, sixteen), their length and genuinely different behaviors in place of noise seeds?
  3. What does the memory encode? For example, do linear probes for task-state variables (buttons pressed, drawer state, nesting order, number of parts moved) separate CPM from action-only memory and from the two history baselines?
  4. Why is prediction-only memory (A6) no better than action-only memory (A3)? Also, how much of the gain comes from the second collection round? I think addressing these questions will help me better recognize the contribution of this work.
Reviewer_JdANOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes that memory compression should be performed based on future behavior prediction. Meanwhile, to address the most critical issue in training memory with future prediction, this paper introduces a fixed procedure that provides a fair frame of reference for all histories, and incorporates the predicted historical information into both memory compression and action generation guidance through joint learning, enabling the VLA to possess action capabilities while also compressing key historical information. In addition, the method was tested on five benchmarks, and real-robot experiments also achieved performance improvements on five single-arm and two dual-arm platforms.

Critical strengths

This paper proposes that memory compression should be performed based on future behavior prediction. Meanwhile, to address the most critical issue in training memory with future prediction, namely that future historical information is highly dependent on subsequent action choices, this paper introduces a fixed procedure that provides a fair frame of reference for all histories, which is innovative to a certain extent. In addition, the method was tested on five benchmarks, and real-robot experiments also achieved performance improvements on five single-arm and two dual-arm platforms.

Critical weaknesses
  1. The Problem Setting Is Not New, and the Introduced Predictive-State Method Lacks In-Depth Mechanistic Analysis Using future observation information to supervise historical content has been done in many prior works [1] [2] [3] [4], so the problem setting itself is not novel. The authors claim innovation in applying the predictive-state method to VLA memory, but this does not address the most critical issues of that method, such as how to select an appropriate test and when that test is sufficient. The paper also does not verify that the eight procedures can cover the history differences relevant to decision-making. It does not explain why the responses defined by these eight procedures suffice to capture the differences in historical information required for subsequent control. The lack of in-depth analysis and experimental validation of this mechanism makes it difficult for the CPM method to establish a principled predictive-state memory design.

  2. No comparison is made with methods that likewise use future information to guide memory The main method of this paper uses future information to guide the compression of current information. The paper has discussed many prior works that use future information to compress current memory, such as RPSP, FLARE, and AEM. However, the current experimental baselines do not cover these works, and therefore cannot demonstrate that compressing memory via the eight fixed procedures has unique advantages over other methods that use future information to describe current memory. The authors should include comparable results for related methods such as FLARE [1], AEM [2], and MemoryVLA++ [3], which would clarify whether the CPM method offers any advantages.

[1] Zheng, Ruijie et al. “FLARE: Robot Learning with Implicit World Modeling.” ArXiv abs/2505.15659 (2025). [2] Zhou, Yijing et al. “Action-Effect Memory Pretraining for Robot Manipulation.” ArXiv abs/2606.12499 (2026). [3] Shi, Haochen et al. “MemoryVLA++: Temporal Modeling via Memory and Imagination in Vision-Language-Action Models.” ArXiv abs/2606.09827 (2026). [4] Li, Wei et al. “FoMoVLA: Bridging Visual Foresight and Motion Guidance for Vision-Language-Action Models.” ArXiv abs/2607.14739 (2026).

Questions for authors
  1. The paper claims novelty in applying the predictive-state method to VLA memory compression, but what exactly is new compared with predictive-state itself remains unclear. It is also unclear why the predictive-state method can be applied to VLA memory compression, why eight procedures are used to address behavioral diversity and whether this coverage is sufficient, and why memory is compressed at four shots. Please clarify the specific novelty of CPM over predictive-state and justify each of these design choices.

  2. The main innovation of this paper lies in using the predictive-state method to compress memory based on future prediction information. The method that should most be compared against is other approaches that also use future information for memory, such as MemoryVLA++. However, the baselines do not include a comparison with such methods, making it impossible to distinguish the unique advantages of this method. Please add comparisons with relevant methods (>2) to confirm the effectiveness of the proposed method.

  3. The innovation of this paper is claimed to lie in using closed-loop procedures generated by a frozen VLA as predictive tests. However, the other modules—including the fixed RFF result representation, recurrent memory slots, the procedure-conditioned decoder, joint training of prediction and action, and two-stage data collection—are merely combinations of existing techniques. Please explain where the innovation of these designs lies and what problems they were designed to solve.

  4. The authors adopt a two-stage training scheme. Could they report the performance of CPM in the first stage (trained for 30,000 iterations using only the data from the first round) and the performance after the second stage, where the model is retrained for another 30,000 iterations still using the first-round data? This would help distinguish whether the final performance improvement stems from the increase in data scale or from the newly added data in the second round.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.