Accelerating Robotic Policies with Cross-Cycle Batching

Recent generative robot policies produce action chunks through a sequence of dependent network forward passes. Each pass processes only a short action sequence, which can leave the GPU underutilized. Processing several updates together could use this spare capacity to reduce latency. However, successive updates of the same chunk must remain sequential. Overlapping prediction windows allow future chunks to begin sampling before their execution cycle. We use this overlap to develop Cross-Cycle Batching, a training-free inference schedule for pretrained flow policies. It staggers full-horizon chunks across control cycles, allowing their next updates to be batched under the latest observation. This reduces sequential calls per cycle in steady state while using the same per-chunk sampling budget as serial. Because each chunk retains the policy’s standard input form, the weights, architecture, and update rule remain unchanged. Experiments show up to a 3.1× denoising speedup while preserving task success, with benefits across policy architectures and in real-robot deployment.