acceptodds
Under review as a conference paper at ICLR 2027

PACER: Training-Free Phase-Scheduled Memory for Long-Horizon Vision-Language-Action Manipulation

Abstract

Frozen vision-language-action (VLA) policies re-encode every observation from scratch, so across a chain of subgoals they can lose track of the object they are acting on, yet fine-tuning such a policy for every deployment costs data, compute, and validation. We show that a frozen policy's own prefix key–value (KV) state can serve as memory. PACER is a training-free framework that maintains this state across replans as working memory and a capacity-one event snapshot, coordinated by a proprioceptive phase controller and periodic refresh. On LIBERO-Long with the frozen policy, PACER raises three-seed direct success from 92.3% to 95.5% (97.3% with an optional instruction-level router), including 91/150 to 149/150 on a repeated-object task, while median inference latency rises by only 0.6%. Both memories contribute to the success gain. On five physical-robot tasks, direct deployment improves aggregate success from 60% to 86% over 250 trials per condition.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action models, robot manipulation, long-horizon manipulation, training-free, inference-time memory, KV cache

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_E9XFOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

PACER is a training-free intervention that maintains a frozen VLA policy’s prefix KV state across replans. It combines selective cache updates, phase-dependent readout, and a capacity-one snapshot, coordinated by proprioceptive signals and periodic refresh. Direct deployment improves LIBERO-Long success from 92.3% to 95.5%, while optional instruction routing reaches 97.3%. Five physical tasks improve from 60% to 86% aggregate success.

Critical strengths
  1. PACER improves an already frozen policy without additional policy training or a learned event detector.
  2. The repeated-object task improves from 91/150 to 149/150, and all five physical tasks improve.
Critical weaknesses

The manuscript has substantial room for improvement in its overall presentation, writing, figures and experimental evaluation.

  1. The abstract is overly brief. Figure 4 is placed in an inappropriate location, while Fig. 3 confusingly combines a figure and a table within a single presentation. Moreover, the motivation in the Introduction and the description of the proposed approach in the Method section are difficult to follow.
  2. Figures 1 and 2 overcomplicate relatively simple concepts, making the proposed framework unnecessarily difficult to understand.
  3. The simulation experiments are conducted only on LIBERO-Long, which provides insufficient evidence to substantiate the effectiveness and generalizability of the proposed method.
  4. The proposed method appears relatively simple and straightforward, and the manuscript does not clearly articulate its methodological novelty or distinct contributions beyond existing approaches.
  5. Essential details of the real-world experimental setup and qualitative visualizations of real-world task executions are missing.
Questions for authors

The authors should thoroughly address the concerns raised in the Weaknesses section for the manuscript to meet the standards of a top-tier publication.

Reviewer_NN3POpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

PACER introduces selective KV-state maintenance, a capacity-one snapshot, and proprioception-based scheduling for frozen VLA policies without additional training. Although the paper reports improvements on LIBERO-Long and physical manipulation tasks, the evidence does not convincingly establish that these gains arise from retaining historical information or resolving long-horizon dependencies.

Critical strengths
  • Improving frozen policies without additional training is a practically relevant deployment objective.
  • The historical-content replacement and memory-reset experiments provide useful diagnostic evidence, although their findings weaken the central memory interpretation.
Critical weaknesses
  • Insufficient positioning against memory-related work. The related-work discussion emphasizes training requirements but inadequately compares memory representations, update and retrieval mechanisms, and the historical dependencies addressed. Consequently, the contribution beyond KV reuse and inference-time representation modification remains unclear.

  • The experiments do not adequately establish memory-dependent capability. Sequential manipulation alone does not demonstrate that past information is necessary. Historical and current-frame snapshots perform similarly, while clearing or swapping memory midway has little observed effect. Moreover, Task 8 contributes more additional successes than the net suite-level improvement; excluding it, direct PACER underperforms the baseline. The limited LIBERO-Mem evaluation does not resolve these concerns.

  • The ablations do not cleanly identify the source of improvement. Shallow and deep configurations differ in scheduling and initialization as well as conditioning depth. The authors should independently vary state persistence, token selection, partial updates, and snapshot conditioning. Historical, current-frame, irrelevant-history, and no-snapshot controls should match token budgets and computation; phase-based, periodic, and randomized schedules should match operation budgets. These tests should cover multiple tasks with controlled observation ambiguity and varying memory delays.

  • The experimental section lacks a coherent hypothesis-driven argument. It reads as a configuration report rather than a systematic evaluation of the claimed mechanism. Each experiment should identify its hypothesis, controlled variables, primary metric, uncertainty, and supported conclusion. Overall effectiveness, historical necessity, scheduling, generalization, and efficiency should be evaluated separately, with direct deployment distinguished from routing-based fallback.

  • The methodological presentation and supplementary evidence are inadequate. Algorithm 1 is a prose checklist rather than self-contained pseudocode specifying state transitions and update conditions. Figure 3 combines an illustrative trace with an aggregate-results table, but provides no numerical time axis or signal scales, making the claimed alignment difficult to assess quantitatively. The trace and results should be presented separately with clear measurements and uncertainty. The reported incomplete video pacer_media_supplement/real_robot/pick_pen_pacer_success_2_clean.mp4 also requires correction.

Questions for authors
  • Which results demonstrate the use of information unavailable in the current observation?
Reviewer_MFAXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PACER, an inference-time method for adding state to a frozen vision-language-action policy without additional training. It maintains selected prefix KV states across replanning steps and a single event snapshot. A controller based on proprioceptive signals, together with periodic refreshes, determines how this state is updated and used to condition action generation.

With a frozen π0.5 policy, direct application improves LIBERO-Long success from 92.3% to 95.5% across three seeds; an optional instruction-based router raises this to 97.3%. On five physical-robot tasks, aggregate success increases from 60% to 86% using the same frozen, robot-adapted checkpoint. The paper also studies token selection, snapshot conditioning, refresh timing, and memory-content interventions to examine where the gains come from.

Critical strengths
  1. The method offers a useful improvement without further policy training. Direct PACER improves LIBERO-Long success by 3.2 percentage points, with a measured latency increase of 0.6% in the median and 4.3% in the mean. The physical evaluation is an important part of the evidence: success improves on all five evaluated tasks, from 150/250 to 215/250 overall, while keeping the underlying policy fixed. This makes the contribution practically relevant beyond the simulation result.

  2. The paper makes a serious effort to examine its own explanation. The historical-versus-current snapshot comparison, matched-quota timing studies, and mid-episode memory interventions are more informative than module-removal ablations alone. The authors report cases where historical content or precise timing makes little difference, as well as tasks where PACER reduces performance. These results help establish the limits of the method and make the empirical findings easier to assess.

Critical weaknesses
  1. My main concern is that some ablations change several factors at once. The p1/p2 comparison changes both KV reuse in the working bank and the action-head readout. The “selection-only” control in Table 8 already reaches 141/150 on Task 8, but its exact computation is unclear. Meanwhile, the snapshot-content and quota-matched timing studies do not show a clear advantage for historical snapshots or precise phase alignment in the tested settings. These findings are useful, but it remains difficult to tell which parts of the proposed memory and phase design are essential to the improvement.

  2. The simulation gain depends heavily on Task 8. It contributes 58 additional successes, while the net gain across all ten tasks is 48. Across the other nine tasks, success decreases from 1294/1350 to 1284/1350. The authors report these regressions, and the physical results are encouraging. Still, it is unclear when PACER should be expected to help a new task. The router reduces regressions on the evaluated suite, but the instruction audit and paraphrase tests do not answer that question.

Questions for authors
  1. Could you clarify whether the “selection-only” control in Table 8 uses historical KV, and how its readout and refresh rules differ from PACER? This would help distinguish the benefit of retaining KV from that of token selection.

  2. Appendix C states that partial forwards omit the bank padding mask. If padded entries are present in the reported experiments, do the main gains remain when these entries are masked during partial updates?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.