Between Glances: Geometric Memory for Adaptive Action Chunking

For a robot acting between observations, each prediction must continue a movement and determine how far it should proceed before feedback returns. We introduce Geometric Replanning with Adaptive Chunk Execution (GRACE), a controller that connects these decisions through the geometry and stability of action generation. Curvature from the preceding denoising trajectory provides a geometric memory that weights directional guidance from the previous plan's remaining actions. Within the current guided generation, agreement among late action estimates determines the prefix to execute. Actual execution then positions the reference for the next observation, while the completed generation supplies its geometric memory. Direction, execution extent, and history thus evolve through one continuous control loop. Across four LIBERO suites, GRACE achieves the highest overall success at 97.60% while reducing replans by 36.6% relative to π0.5. Component comparisons support the contributions of adaptive execution, geometric guidance, and directional weighting. An anonymous demonstration is available at https://anonymous.4open.science/w/grace-demo-F4F2/.