acceptodds
Under review as a conference paper at ICLR 2027

Between Glances: Geometric Memory for Adaptive Action Chunking

Abstract

For a robot acting between observations, each prediction must continue a movement and determine how far it should proceed before feedback returns. We introduce Geometric Replanning with Adaptive Chunk Execution (GRACE), a controller that connects these decisions through the geometry and stability of action generation. Curvature from the preceding denoising trajectory provides a geometric memory that weights directional guidance from the previous plan's remaining actions. Within the current guided generation, agreement among late action estimates determines the prefix to execute. Actual execution then positions the reference for the next observation, while the completed generation supplies its geometric memory. Direction, execution extent, and history thus evolve through one continuous control loop. Across four LIBERO suites, GRACE achieves the highest overall success at 97.60% while reducing replans by 36.6% relative to π0.5. Component comparisons support the contributions of adaptive execution, geometric guidance, and directional weighting. An anonymous demonstration is available at https://anonymous.4open.science/w/grace-demo-F4F2/.

Primary area:
applications to robotics, autonomy, planning
Keywords:
robot learning, vision-language-action models, action chunking, geometric memory, adaptive execution, flow matching

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_5bQyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work addresses the problem of adaptive action chunking. From my understanding, the method applies guidance or correction to the diffusion process to make the diffused action chunk more consistent with the unexecuted portion of the previous chunk. The consistency is weighted so that the guidance is stronger in directions that have larger second differences (along the diffusion path). The setup also provides a mechanism for deciding how long to execute the chunk before observing.

Critical strengths

The proposed method appears to work and (if I understanding things correctly) and the guidance framework would enable the model to diffuse more consistent trajectories.

Critical weaknesses

Writing could be clearer. Please provide a clear problem statement and setup in Section 3.

The overall denoising process could be better explained. The action chunk is denoted as unu_n in 3.1. But in 3.2, this changes to xx and then the updated is denoted as candidate cc?

How is c adapted to obtain the final denoised action chunk? From what I can infer, the final correction term is \Delta p_hist where the potential is scaled with W_n. I'm having trouble understanding why the matrix W_n (which captures curvature) would be useful in this setup. Why are the historical second differences across the diffusion process meaningful? What is the intuition? It is capturing "uncertainty" in the diffusion process.

Also, I think the nature of the action chunk being denoised needs some clarification. For this to work out, I think the chunk c (or x) should represent velocities or position differences / increments? Absolute position targets don't seem to make sense given the accumulation. Perhaps I am misunderstanding here.

It is unclear to me what the "variation energy" EnE_n is in Eq (4) or how it is obtained or set? It appears to be some sort of regularization given it is part of the solution W_n.

The experimental results in Table 1 suggest the method does perform better in some settings compared to the baselines but the performance difference is very small. Are these differences meaningful and statistically significant?

minor:

  • There has been recent work on streaming flow policies [Jiang et al, CoRL'25] and their stochastic counterparts that permit guidance/conditioning [Jiang et al, R:SS'26]. Are they related?
  • "Collaboration et al., 2023"" is not the right citation for the OpenX Embodiment paper
  • What is the "resulting metric" or the "stability readout"? These are mentioned without being first defined or explained. It is only defined later in (5) and (7).
Questions for authors

Please see the questions in the weaknesses section above.

Reviewer_fY3rOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This work proposes a training-free inference-time controller for flow-based action-chunking VLAs that links how each new chunk continues the previous plan with how much of it gets executed before replanning. It contributes (1) a geometric memory that turns the curvature of the previous denoising trajectory into a directional metric for guiding generation toward the previous plan's unexecuted actions, and (2) an adaptive execution rule that cuts the chunk where late-denoising estimates stop agreeing. On four LIBERO suites with π0.5, it reports 97.60% overall success and 36.6% fewer replans than the base policy.

Critical strengths
  1. The method is specified rigorously, with a complete mathematical construction in App. A.
Critical weaknesses
  1. Limited experimental scope. All experiments are on LIBERO, which is known to be a saturated simulation benchmark: the base π0.5 already reaches 96.47%, and all eight controllers fall within about 1.1 points. GRACE improves over π0.5 by 1.13 points, and over the concurrent DVAC by only 0.20 points (97.60 vs 97.40, Table 1). Such marginal gains are not convincing evidence that the proposed method really works, and we can't tell whether the design transfers to different types of embodied tasks.
  2. Only one base policy. The paper frames the method for "diffusion and flow policies", but it is only tested on π0.5-LIBERO. Comprehensive experiments across different policies (e.g., π0 or a diffusion policy), different tasks, or the same policy at different performance stages (e.g., earlier checkpoints) would help us understand how well the method generalizes.
  3. Limited contribution. Adaptive action-chunk length is not new: the concurrent DVAC also selects the prefix from the variance of late clean-action estimates with a rolling threshold (Sec. 2). The paper's own ablation shows the replan reduction comes entirely from this part (32.30 replans with fixed execution vs. 20.62 without geometric memory, Table 3). That leaves the curvature-based directional weighting as the main new component, and its design rationale remains unclear (see Question 1).
  4. The headline claim isn't statistically supported. The abstract claims GRACE "achieves the highest overall success," but its lead over the concurrent adaptive-chunking baselines is within noise. GRACE gets 9,760 successes out of 10,000 episodes vs. 9,740 for DVAC and 9,734 for PACE (Table 3), a 0.20 and 0.26 point difference. Using the paired outcomes in Fig. 3b (172 vs. 152 discordant episodes against DVAC, 192 vs. 166 against PACE), a McNemar test gives roughly p ≈ 0.27 and p ≈ 0.17, so neither difference is significant. GRACE is also not the best method on two of the four suites: BID is higher on Spatial (98.84 vs. 98.52), and RTC/PACE are higher on Goal (98.08 vs. 97.88). The evidence supports "on par with the best adaptive-chunking controllers," not "highest." Please report paired significance tests or bootstrap CIs for GRACE vs. each baseline and each ablation in Table 2.
Questions for authors
  1. The rationale behind using curvature as directional weights is unclear. During denoising, a direction that bends a lot across steps could mean either "the model cares a lot about this direction" or "the model is uncertain about this direction." In the latter case, we should not increase the weight on that direction, yet Eq. 5 assigns larger weights to exactly these directions. Could you address this concern, e.g., by showing whether high-curvature directions align with the direction of the next observation-conditioned chunk more often than low-curvature ones?

Others please see weaknesses.

Reviewer_9xQqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new controlled for action chunk generation named GRACE. The goal is to improve the continuation of action chunking for vision-language-action policies. It uses the geometry of previous denoising trajectories to guide new actions. In addition, it also estimates prediction stability to decide the number of actions to be executed before requesting a new observation. Experiments are conducted on the benchmark of LIBERO and it reported promissing improvement in terms of success rate.

Critical strengths

Below I would summarize the strong points of this paper.

  1. The paper is well motivated. It noticed the challenging in action-chunk based VLA models. The proposed model attempts to simutaneously maintain motion continuity across replanning steps and decide when to obtain new observations.

  2. There are some new ideas in the proposed model. In particular, it proposes to use he geometry of previous denoising trajectories as directional memory. In this way it preserves the direction of previous motion while adapting its magnitude to the current observation.

  3. The proposed idea seems straightforward and general. Ready to be deployed into a variaty of VLA models with action expert.

  4. The authors evaluate 40 tasks across four LIBERO suites, compare against several recent action-chunking methods, and conduct component studies. The ablations provide evidence that both adaptive execution and directional geometric memory contribute to performance.

Critical weaknesses
  1. The improvement over the baselines is rather marginal. In the main Table 1, GRACE achieves 97.60% success rate. In comparison, 97.40% for DVAC. Considering additonal computaton is required for estimating the geometry, the practical advantage over recent adaptive chunking methods is therefore less clear.

  2. The stability measure may not reliably indicate action correctness. The proposed method uses agreement among late denoising predictions to determine execution length. However, low prediction variance does not necessarily mean that the action is safe or appropriate to execute for a longer period. The paper does not sufficiently evaluate or discuss this issue. Neither is rigorous math proof provided.

  3. The experiments are only conducted on one simulation benchmark. No real robot experiments. All experiments are conducted on LIBERO benchmarks using the same pretrained PI-0.5 policy. No other policy is used for rule out the effect of different base policy. Therefore, the robustness of the method to more challenging scenarios in large-scale benchmarks, real-world perception errors and physical disturbances remains unclear.

  4. The paper focuses on success rate and the number of replanning calls, but does not clearly report the additional computation required by geometric guidance and stability estimation. Fewer replans do not necessarily mean lower wall-clock execution time. Fig. 3 is not well explained and should include some statistics regarding wall-clock time.

Questions for authors

See my detailed comments in the section "critical weaknesses".

Reviewer_j1uZOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes GRACE, a training-free method for flow-matching policies that addresses two decisions at each replanning step: how to make the newly generated action chunk consistent with the previous one, and how many actions from the new chunk to execute. For chunk continuation, GRACE uses the unexecuted leftover of the previous chunk to guide the new generation through an angle-based potential, with a directional weighting derived from the curvature of the previous generation’s denoising path, i.e., how its intermediate action chunks changed across denoising steps. For execution length, it executes the prefix up to the first action where the variance across late denoising estimates exceeds a rolling threshold. Built on a frozen pi0.5 policy, GRACE is evaluated on four LIBERO suites against seven baselines, achieving 97.60% overall success and reducing the number of replanning steps by 36.6% compared with pi0.5.

Critical strengths

Training-free with a broad, controlled baseline comparison. GRACE plugs into a frozen pi0.5 checkpoint without retraining, and is compared against seven controllers, including recent and concurrent work (RTC, BID, DVAC, PACE), under a shared checkpoint and protocol with many episodes per configuration. This makes the reported differences attributable to the controller itself, which is useful even though, as argued below, those differences are small.

Critical weaknesses
  • Marginal gain and incomplete validation. GRACE’s execution rule closely resembles that of DVAC, using late-estimate variance with a rolling threshold. However, GRACE outperforms DVAC by only 0.2 percentage points (97.60±0.1797.60 \pm 0.17 vs. 97.40±0.3397.40 \pm 0.33, Table 1), a difference that appears to be within run-to-run variation. GRACE’s adaptive execution without guidance (97.13, Table 2) even performs worse than DVAC. Moreover, a closely related method that combines guidance with adaptive chunking [1] is not compared. Also, the 36.6% replan reduction is never reported as wall-clock time or FLOPs, although guidance and stability computations add per-call cost. Finally, the evaluation is limited to a single near-saturated simulation benchmark and one base policy, with no real-robot experiments, leaving the method's practical effectiveness insufficiently validated.

  • Unclear problem setting and presentation. Section 3 does not clearly define the observations and states, action space, action chunks, or flow-matching sampling process, even though all the proposed mechanisms depend on these details. The term "memory" is never formally defined, and its role is conflated with that of the leftover reference from the previous chunk. The configurations in Table 2 are also not clearly specified. Notably, the "w/o geometric memory" ablation removes the entire guidance term rather than isolating the effect of curvature-based weighting. As a result, the ablation name overstates what the experiment actually tests.

  • Mechanism unmotivated and robustness untested. The paper does not explain why denoising-path curvature should indicate which directions of the previous plan are worth trusting. The empirical evidence is also limited: the ablation differences are within one standard deviation, and isotropic guidance performs worse than no guidance (96.99 vs. 97.13), with no explanation for this result. Moreover, the method always steers the new plan toward the leftover portion of the previous one, but the authors do not investigate what happens when that plan becomes outdated or incorrect (e.g., due to external perturbations). Sensitivity to the execution threshold is also not evaluated.

[1] So, Junhyuk, et al. "Improving generative behavior cloning via self-guidance and adaptive chunking." Advances in Neural Information Processing Systems 38 (2026): 84432-84456.

Questions for authors
  1. Please add a problem formulation: observation, action space, action chunk, and flow-matching sampling (time convention, velocity field, clean estimate).
  2. How does GRACE compare with SG-AC [1]?
  3. What are wall-clock time and FLOPs per episode vs. pi0.5 and DVAC? Do fewer replans reduce total inference time?
  4. What exactly does each Table 2 configuration change? Can you ablate only the curvature weighting while keeping guidance, with significance tests?
  5. How does GRACE behave when the leftover is outdated, e.g., under external perturbation? Does steering toward it hurt recovery?
  6. How sensitive are success and replans to the execution threshold?

[1] So, Junhyuk, et al. "Improving generative behavior cloning via self-guidance and adaptive chunking." Advances in Neural Information Processing Systems 38 (2026): 84432-84456.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.