acceptodds
Withdrawn from ICLR 2027

LoopVLA: CROSS-SUBTASK EVENT MEMORY FOR LOOPED TASK EXECUTION

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Repetitive tasks are common in real-world manipulation, yet current vision-language-action (VLA) policies remain unreliable for repeated actions. We present LoopVLA, a cross-subtask event-memory interface for looped task execution. Its central idea is to compress a just-completed subtask into memory for the next subtask's action expert, making completed execution available for subsequent repetition and stopping decisions. Learnable event latents query the closed segment's tokens, while action features query history and joint history–event context; the two action-side readouts are residually fused and injected directly into the action head. On a memory benchmark, LoopVLA achieves 80.22% success on the repetition task group with online subgoal prediction, bringing evaluation closer to realistic deployment while improving overall success over strong memory baselines.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
VLA, Event, Memory, Subtask, Embodied, Robot

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FGWROpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper targets looped (repetitive) manipulation tasks, where a vision-language-action (VLA) policy must repeat a subtask a prescribed number of times and then stop. The authors propose LoopVLA, a cross-subtask event-memory interface built on top of π0.5. At each subgoal transition, learnable event latents attend to the tokens of the just-closed subtask segment and compress it into an event memory. On the action side, action features separately read the perceptual history and the joint history–event context; the two readouts are residually fused and injected into the action head through a FiLM-like MemoryRMS module. Training combines a flow-matching action objective with a next-subgoal cosine-alignment loss. Evaluation uses 16 RoboMME tasks, with emphasis on the repetition group (PickXtimes, SwingXtimes, StopCube): under an online Qwen-VL subgoal protocol the method reaches 80.22% on the repetition group versus 22.67% for a memory-off configuration, and 47.17% across all 16 tasks versus 44.51% for the strongest non-SG RoboMME variant. Three additional real-robot tasks are reported with 39/60 successes versus 21/60 for π0.5.

Critical strengths
  1. The problem is well chosen and under-explored. Historically conditioned looped execution and the associated stopping decision (continue / repeat / stop) receive little dedicated treatment in the VLA memory literature, and the online Qwen-VL subgoal protocol is a genuine step toward realistic deployment.
  2. The division of labor between event triggering and memory weaving is plausible a priori, and the repetition group improves substantially over the memory-off configuration (80.22% vs 22.67%), so the general interface shape of "where to inject information about a completed segment" is informative for follow-up work.
Critical weaknesses

W1. Attribution of the gain is confounded, and the controlling baselines are missing. LoopVLA introduces two things that no baseline receives: the Qwen-VL subgoal labels consumed by both the policy and the event trigger, and 64 additional visual tokens (32 history observations plus 32 event latents); every baseline in Table 1 is evaluated without subgoal input, and the RoboMME SG variant is not reported. Two controls decide whether the claim holds: π0.5 with the same Qwen-VL subgoal labels but without the LoopVLA memory interface, and direct concatenation of raw frames from the closed segment at an identical token budget without learnable latents. The internal ablations already point away from the paper's causal story: raising H from 10 to 32 takes the repetition group from 60.00% to 81.33%, whereas event-aligned windows improve over fixed recent windows only from 46.38% to 49.25% across all tasks. Most of the measured gain therefore comes from additional history tokens rather than from the event memory structure, and the causal claim in the abstract and conclusion that memory of the completed subtask supports repeated execution is not supported.

W2. No identifiable mechanism, and no qualitative analysis whatsoever. Because the task is fundamentally a counting problem, the most direct falsifiable test is whether the event latents encode a decodable repetition count; the authors explicitly disclaim this test while supplying no substitute evidence. There is no similarity matrix or dimensionality-reduction plot of event memories, no visualization of Weaver attention over segment tokens, no side-by-side rollout sequences for successful and failed episodes, and no breakdown of failures into over-counting, under-counting, and premature stopping. Not one of the five figures shows a qualitative result, and the real-robot study (Section 4.7) reports six numbers with no image of the platform, the scene, or a single executed trajectory. A reader therefore cannot distinguish whether the 22.67% to 80.22% improvement reflects count information carried by the memory or merely the added parameters and subgoal supervision.

W3. The evaluation is neither comparison-fair nor statistically convincing. Baselines are quoted with the aggregation protocols of their own papers while LoopVLA is averaged over three evaluation seeds, and the authors themselves describe these comparisons as descriptive, so the headline 47.17% versus 44.51% (+2.66 points, baselines carrying no error bars) cannot support a claim of improvement. Hyperparameters (H=32, M=32) are selected on the same episodes and the same seed (Oracle, seed 7) used for reporting, and Table 2, Figure 3 and Figure 4 are single-seed without error bars, leaving the 49.25 versus 46.75 gap possibly inside seed noise. On non-repetition memory tasks LoopVLA trails MemER badly (VideoUnmask 43.33 vs 81.33, ButtonUnmask 26.67 vs 72.00, PickHighlight 24.00 vs 70.67) without any comment, so "overall success improves" conceals very large per-task variance, and the real-robot study uses only three self-designed tasks with 20 trials each against a single π0.5 baseline and no memory-based baseline, which makes 11/20 versus 5/20 weak evidence at that sample size.

W4. Presentation is below the standard. There are no qualitative figures anywhere in the paper: no rollout frame sequences, no visualization of Weaver attention over segment tokens, no illustration of what the event memory actually encodes, and even the real-robot study (Section 4.7) offers no image of the platform or the scene, so no intuition for how the method behaves during execution is ever conveyed. Figure 2 is barely legible at print scale, its legend glyphs are crowded, and the mapping between its modules and Eqs. (2)–(4) is never made clear. The data visualization is equally crude: Figure 3 renders twenty cells as a heatmap while reprinting the same numbers inside every cell and changes the reported unit (PP_IT%, %, AT%) across tables, single-seed results in Figures 3–5 carry no error bars, and all figures rely on a single red/gray palette with no regard for colorblind readers.

Questions for authors
  1. Please report the two controls that decide the attribution: (i) π0.5 with Qwen-VL subgoal labels but without the LoopVLA memory interface, and (ii) raw frames of the closed segment concatenated directly into the context at the same token budget, without learnable latents; please also provide multi-seed error bars and state whether hyperparameter selection used a held-out split.
  2. Do the event latents carry a decodable repetition count, for instance under a linear probe or in an event-memory similarity matrix? If not, please explain the mechanism behind the 22.67% to 80.22% improvement, and clarify whether the large regression relative to MemER on non-repetition tasks (ButtonUnmask 26.67 vs 72.00) follows from using subgoal transitions as the only trigger.
  3. Please report results for the RoboMME SG variant, the end-to-end latency and control frequency of each policy query including the Qwen3-VL-4B call issued every 16 actions, and the real-robot platform setup, success criterion, and comparison against at least one memory-based baseline.
4 comments
AuthorsOpenReview ↗

Factual errors requiring correction

Factual errors requiring correction

  1. Unexplained ethics flag. No reason is visible for “Flag For Ethics Review: Yes”; please state its basis.
  2. W4 — Units absent from the paper. Neither “PP_IT%” nor “AT%” occurs in the PDF. Before reconciling these units, we would be grateful for a reference to where either appears.
  3. W2/W4; Q3 — Existing qualitative evidence and platform documentation. “no qualitative figures anywhere” and “no image of the platform or the scene” overlook Figure 1(a), p. 2, with time-stamped execution frames, and Figures 7–8, p. 17, with rollouts and the dual-arm ALOHA platform. The supplementary material also includes real-robot execution videos, signposted in the Reproducibility Statement on p. 10. The platform requested in Q3 is already documented in Appendix D and Figure 8. Please reassess W2/W4 against both the figures and the videos.
  4. W4 — Existing error bars described as absent. “single-seed results in Figures 3–5 carry no error bars” is incorrect for Figure 5: it reports three-seed mean ± sample SD, with error bars on all four bars.
  5. W4 — Incorrect figure descriptions and colour-accessibility claim. Figure 3 contains 32 cells, not twenty; Figures 1–2 use multiple colours. The assertion of “no regard for colorblind readers” overlooks deliberate accessibility measures: one of our authors has red–green colour blindness, and numerical labels make the results readable without colour discrimination. We checked Figures 3–5 using Brettel-model and Color Oracle simulations of protanopia, deuteranopia, and tritanopia, alongside greyscale views. Do you request further palette changes? If so, please identify the specific figure elements that remain difficult to distinguish.
  6. W1/Q1(i) — Incorrect policy-input premise. The labels are not consumed by “both the policy and the event trigger” as additional text. They trigger segment closure and event-memory refresh; the policy's textual input remains the task instruction (Eq. (1), p. 3; Section 3.3 and Figure 2, p. 4). Q1(i)'s direct subgoal-text input is therefore a different interface from the one shown in the paper. Without LoopVLA memory interface, there is no need for using Qwen-VL to predict subgoals. We would recommend reviewer to go through relevant equations and figures for the purposes of predicting subgoals.
  7. W1 — Incompatible counts added together. “64 additional visual tokens” adds 32 observations to 32 latent slots. Both happen to be 32; that does not make them counts of the same thing.
  8. W2/Q2 — The distinction between Event memory and history is already explicit. Section 3.3 (p. 4) states that each summary represents the latest closed segment, while earlier execution remains accessible through perceptual history. Section 4.6 (p. 8) explains that history continues to update between event transitions. This distinction matters to both Q2's cumulative-count criterion and its “only trigger” interpretation: transitions govern Event replacement, not all access to past observations.

We respectfully ask you to revisit these points against the cited locations and supplementary videos. Does this evidence address your concerns about the policy inputs, memory mechanism, and presentation? If any discrepancy remains, we would appreciate the specific passage or figure so that we can address it precisely.

AuthorsOpenReview ↗

Responses to the weaknesses

Many thanks for reviewer's effort in review. We further address W1–W4 and Q1–Q3 separately.

W1 — Attribution from the history sweep

“Most of the measured gain therefore comes from additional history tokens rather than from the event memory structure”

The review compares a gain on three repetition tasks (60.00% → 81.33%) with a gain on all 16 tasks (46.38% → 49.25%). These different aggregates cannot decompose the contribution.

We have now extended the full history sweep to three evaluation seeds (7, 21, 42), with 50 episodes per task per seed. The history sweep and Event perturbation use Oracle and report mean ± sample SD; Repetition averages PickXtimes, SwingXtimes, and StopCube. With M = 32 throughout:

History framesAll 16 (%)Repetition (%)
H = 1039.58 ± 0.6260.44 ± 4.68
H = 2042.00 ± 0.2270.22 ± 3.36
H = 3247.71 ± 1.3779.78 ± 1.39
H = 4045.54 ± 1.0676.22 ± 1.92

H = 32 exceeds H = 40 on both metrics in every seed. The sweep identifies a suitable history budget for this setting: adding more history can reduce performance. Comparing different task aggregates while omitting this decline does not establish that history accounts for most of the gain. The Event perturbation below directly tests whether the learned Event representation affects execution.

W2 — The proposed counting criterion

“the most direct falsifiable test is whether the event latents encode a decodable repetition count”

This conflates the task's global requirement with one module's role. Weaver stores the just-completed segment and replaces it at the next transition; it does not accumulate multiple event memories. The policy jointly reads that segment, perceptual history, and current observations. A global counting task does not require every intermediate representation to independently encode the global count. Which claimed function of this single-segment memory would the proposed cumulative-count test actually assess?

We instead test its role in control: at the same checkpoint, we negate only the 32 normalised Event-token vectors before their K/V projections (E → −E), preserving the history-input pathway, latent budget, fusion, and MemoryRMS.

Event representationAll 16 (%)Repetition (%)
Original47.71 ± 1.3779.78 ± 1.39
E → −E39.92 ± 1.2561.33 ± 3.06

Repetition success decreases by 18.44 percentage points, with a decrease on every repetition task in every seed. The policy uses the learned Event representation for subsequent control. This supplies the requested functional evidence while keeping parameters and latent capacity fixed.

W3 — Evaluation and reporting

“The evaluation is neither comparison-fair nor statistically convincing.”

The main Qwen-VL results are 47.17 ± 1.42% overall / 80.22 ± 2.04% repetition, over evaluation seeds 7, 21, and 42.

We also evaluated history-only, H = 32 under Oracle, using the same tasks, evaluation seeds, and episode budget: 44.92 ± 0.58% overall / 72.89 ± 1.68% repetition, compared with 47.71 ± 1.37% / 79.78 ± 1.39% for full LoopVLA. This supplies a history-only comparison with seed variation reported for both configurations. Answer to Q2 addresses non-repetition performance; answer to Q3 addresses the robot study.

W4 — Presentation

“Presentation is below the standard.”

We will improve Figure 2's legibility and equation mapping. We respectfully ask that the presentation assessment also take account of the existing figures, error bars, and real-robot execution videos identified above.

AuthorsOpenReview ↗

Responses to the questions Q1 and Q2

Q1 — Controls and configuration selection

“π0.5 with Qwen-VL subgoal labels but without the LoopVLA memory interface”

“raw frames of the closed segment concatenated directly into the context at the same token budget, without learnable latents”

Q1(i). SimpleSG+QwenVL and GroundSG+QwenVL implement the requested class of direct subgoal-text input to π0.5, scoring 29.00% and 32.70% under RoboMME's published training protocol. As stated in factual error 'W1/Q1(i) — Incorrect policy-input premise', our labels instead trigger event-memory updates. The full SG table is below. RoboMME, Sections 4.2–5.2

Q1(ii). Our completed attribution controls are the history-only comparison (response to W3) and the same-checkpoint Event-feature intervention (response to W2). The latter changes only the Event features, preserves model weights and latent capacity, and reduces repetition success by 18.44 percentage points. It directly tests whether the learned Event representation influences subsequent control.

Selection and error bars. W1–W3 report the three-seed results and sample SDs.

Q2 — Decoding, improvement, and ButtonUnmask

“Do the event latents carry a decodable repetition count”

“please explain the mechanism behind the 22.67% to 80.22% improvement”

“clarify whether the large regression relative to MemER on non-repetition tasks (ButtonUnmask 26.67 vs 72.00) follows from using subgoal transitions as the only trigger.”

Count probe. We do not adopt cumulative-count decodability as the criterion for this single-segment memory. As explained under W2, the representation contains one completed segment, not an accumulated event record. The model has no count-supervision objective for event latents; a probe would require additional aligned count targets and a separate readout-training protocol. Feature similarity alone does not establish repetition count. If the concern is accumulated progress, why exclude the perceptual history with which the segment is jointly read?

Mechanism. Multiple layers of cross-attention let the action expert use the latest completed segment alongside earlier perceptual history and current observations. This conditions subsequent execution on what has happened. The 22.67% → 80.22% comparison concerns the memory-disabled result and the full Qwen-VL interface. The additional Oracle intervention in W2 directly establishes dependence on the Event representation, with repetition success falling from 79.78% to 61.33% while model parameters and latent capacity remain fixed.

ButtonUnmask. Our claim concerns repeated execution and correct stopping, as your own summary recognises. The reported ButtonUnmask gap does not contradict that claim. Transitions govern event-memory replacement, while perceptual history remains available; Weaver compresses the entire completed segment, not just its boundary frame. The scores therefore do not establish the trigger as the cause.

AuthorsOpenReview ↗

Responses to the questions Q3

Q3 — SG results, runtime, and real-robot evaluation

“Please report results for the RoboMME SG variant”

MethodAll 16 (%)Repetition (%)
SimpleSG + QwenVL29.0033.63
GroundSG + QwenVL32.7033.33
SimpleSG + Gemini23.2536.67
GroundSG + Gemini11.568.33
SimpleSG + Oracle49.5881.48
GroundSG + Oracle84.0883.22

Repetition averages PickXtimes, SwingXtimes, and StopCube. Baseline entries are calculated from the published task means; no uncertainty for this three-task aggregate is reported (Table 15, p. 38).

RoboMME's authors themselves state that “FrameSamp+Modul achieves the highest overall success rate (44.51%)” (Dai et al., 2026, §5.2, Q1, p. 8), and describe GroundSG+Oracle as “an upper bound of symbolic memory” (§5.2, Q2, p. 9). Their Table 3 (p. 8) explicitly distinguishes the best non-oracle result. Our choice of 44.51% follows that distinction: all non-oracle SG variants are lower, with the strongest at 32.70%, so they cannot represent the best non-oracle baseline.

Oracle uses privileged simulator-state checks—for example, whether an object has reached a target—which are unavailable directly in real deployment. This states SG variants with Oracle settings' scores are not comparable with realistic deployment setting's.

“the real-robot platform setup, success criterion, and comparison against at least one memory-based baseline.”

Figure 8 and Appendix D (p. 17) document the dual-arm ALOHA setup, and the supplementary videos show real-robot execution. Section 4.7 (p. 9) specifies the task objectives: three pull–push cycles for Drawer Push–Pull, four round trips for Bowl Transfer, and three round trips followed by placement in a cup for Battery Transfer, with stopping or switching at the requested count. The results are 39/60 versus 21/60 against π0.5, with 20 trials per task. The physical study evaluates repeated execution with π0.5 as its comparator; the memory-baseline comparisons are reported on RoboMME.

Do these clarifications and additional results address your concerns? If a substantive concern about the repetition claim remains, could you identify the specific experiment needed to resolve it and the alternative explanation it would distinguish?

Reviewer_mCcWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LoopVLA, a memory interface for history-dependent robotic manipulation, particularly repeated execution and stopping at a requested repetition count. At a subgoal transition, learnable event queries compress the most recently completed execution segment into a fixed-size representation. The action expert reads both sampled perceptual history and the joint history–event context, and uses the fused features to modulate action generation. The method is instantiated on π0.5 and trained with action prediction and an auxiliary next-subgoal alignment objective. Evaluation covers 16 RoboMME tasks, oracle and online predicted subgoals, and three real-robot tasks. With online subgoals, LoopVLA reports 80.22% success on the three-task repetition group and 47.17% overall success.

Critical strengths
  1. The paper addresses an important control problem with a well-motivated use of task structure. Repeated manipulation requires distinguishing visually similar states associated with different execution histories. Organizing memory around subtask transitions is a plausible way to preserve information relevant to subsequent decisions. Combining a summary of the latest completed segment with global perceptual history also provides a sensible division between recent execution details and earlier context.

  2. The empirical results are encouraging, including evaluation with predicted subgoals and physical robots. Relative to the reported FrameSamp+Modul baseline, the online version improves repetition-task success from 73.78% to 80.22%, with improvements on all three constituent tasks. Its performance is close to that of the oracle-subgoal protocol for the evaluated checkpoint, suggesting that the system can tolerate the observed prediction errors. The real-robot results also improve from 21/60 successful trials for π0.5 to 39/60 for LoopVLA, with gains on each task. These results support the practical potential of the approach.

Critical weaknesses
  1. The advantage of latent event memory over explicit high-level progress tracking remains unclear. LoopVLA already relies on oracle or Qwen-VL subgoal transitions to organize execution. A natural alternative is to use the same observations and transition signals to maintain an explicit repetition counter or compact language-based progress state, while a subgoal-conditioned policy executes individual operations with the short-term visual context needed for control. The comparison with MemER provides relevant evidence against one hierarchical memory approach, but it does not isolate this design choice under matched conditions. In particular, the paper does not establish whether the event representation contributes information beyond what an explicit progress state and perceptual history could provide. This is consequential because the central contribution concerns how completed execution should be represented and supplied to subsequent control. A matched hierarchical baseline would help determine whether the improvement comes from better progress tracking, richer information for continuous execution, or the combination.

  2. The experimental comparison and discussion do not sufficiently position the method against closely related VLA/WAM memory architectures. The paper cites MEM and ReMem-VLA, which respectively investigate hierarchical language/visual memory and recurrent-query memory, but does not compare against them experimentally. EventVLA and HiMem-WAM are also particularly relevant: they investigate selective visual-event storage and skill-boundary-triggered memory updates. The paper should explain more precisely what completed-segment compression and action-side consumption contribute relative to these designs. An exhaustive comparison against every recent VLA/WAM is unnecessary, especially when training resources and embodiments differ. However, a representative comparison with a feasible, closely related method, together with clearer discussion of architectural differences and comparability constraints, is important for assessing the contribution beyond the selected RoboMME baselines.

Questions for authors

Could you provide a history-only baseline under the same subgoal, training, and evaluation settings to clarify how much the event memory contributes beyond perceptual history?

Reviewer_XZbHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LoopVLA, a cross-subtask event-memory framework for repetitive robotic manipulation. It compresses completed subtasks into event memory and integrates this memory into the action expert. The method improves performance on RoboMME repetition tasks and shows promising results in real-world experiments.

Critical strengths
  1. The paper addresses a practical problem in VLA models: tracking completed actions during repetitive tasks. Using subtask transitions to update memory is a sensible design choice.

  2. The method achieves clear improvements on repetition tasks, with additional validation on real robots.

Critical weaknesses
  1. Limited component-level ablations. Although the paper studies history length and event windows, it does not sufficiently examine the individual contributions of event memory, residual fusion, and MemoryRMS.

  2. Baseline comparisons are not fully controlled. Most baseline results are taken from previous work, making it difficult to assess the improvements under identical settings.

Questions for authors
  1. Limited component-level ablations. Although the paper studies history length and event windows, it does not sufficiently examine the individual contributions of event memory, residual fusion, and MemoryRMS.

  2. Baseline comparisons are not fully controlled. Most baseline results are taken from previous work, making it difficult to assess the improvements under identical settings. Several ablations also rely on a single evaluation seed.

Comments on the paper

1 comment

Dear Reviewers, With the rebuttal period ending on November 18, please take a moment to check the authors' latest responses, post your feedback, and join the discussion. We greatly appreciate your hard work and dedication.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.