acceptodds
Withdrawn from ICLR 2027

ASYNCHRONOUS ROBOT POLICIES

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Scaling vision-language foundation models has significantly advanced generalist robot manipulation, but the increased model scale introduces computational latency that degrades real-time physical reactivity. Existing Vision-Language-Action (VLA) architectures execute high-level semantic reasoning and action execution sequentially within the primary control loop, creating a fundamental tension between semantic-reasoning capability and robot agility. To address this, we propose Asynchronous Robot Policy (ARP), an architectural framework that decouples semantic reasoning from low-level action generation using asynchronous processes operating at independent frequencies. In ARP, a high-capacity Vision-Language Model (VLM) asynchronously updates a semantic memory at low frequency, while a lightweight Vision-Action Expert, conditioned on this semantic memory, generates robot actions at a much higher control frequency. Such a formulation, enables having a fast perception-action loop without compromising on the semantic capability. To overcome asynchronous memory staleness, ARP trains the vision-action expert on the semantic staleness and execution delay, with temporally mismatched observation across processes. On Franka manipulation tasks, ARP achieves an 18% higher success rate and 20% less time need for execution in quasi-static setups, while 3-5× baseline performance in dynamic simulation environments. Furthermore, ARP enables operating the VLM at 10× lower than its nominal frequency with no significant performance loss making it much more efficient than the baseline. These results highlight the architectural advantages of modeling robot policies as asynchronous processes, enabling real-world reactive deployment without sacrificing the scale of foundation models. Our models and source code will be made publicly available.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Generalist Robot Policies, Robot Manipulation, Asynchronous Processes

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 4 reviews, above 0% of the others.
Reviewer_pxFwOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ARP splits a robot policy into two asynchronous loops. A high-capacity VLM periodically produces a fixed-width semantic memory, while a lightweight Vision-Action Expert runs at a high control frequency using current visual/proprioceptive inputs and the latest memory. The expert is conditioned on memory staleness and execution delay through sinusoidal embeddings and AdaRMS, and is trained with temporally mismatched memory/action pairs plus action-prefix conditioning. On LIBERO, ARP reaches 84.8%, 86.6%, 89.3%, and 89.1% average success for 1B/2B/8B/14B variants. Under a 5 cm LIBERO perturbation, ARP-1B obtains 51.5% versus 9.5% for the synchronous 1B baseline; on the Dynamic Bump benchmark it obtains 44.7% versus 12.7% when a bump is guaranteed. Under VLM throttling, ARP remains around 79.0%-84.8% while the baseline falls from 81.5% to 45.0%. RoboLab and a small Franka study show modest efficiency and smoothness benefits.

Critical strengths
  1. The multi-rate decomposition is well motivated and directly addresses the latency-capability trade-off in large VLM-based policies.
  2. The fixed-width semantic-memory interface, explicit staleness/delay conditioning, and stale-memory training distribution form a coherent end-to-end design rather than an inference-only scheduling trick.
Critical weaknesses
  1. The distinction from prior asynchronous systems (AsyncVLA, AR-VLA, VLASH, RTC, and related cloud/edge policies) is not fully established. There is no direct, same-backbone comparison against the strongest asynchronous baselines, so the independent contribution of the architecture and the stale-memory training is unclear.
  2. Several comparisons are confounded by backbone, visual encoder, data, and pretraining differences. In particular, ARP and LAP-3B use different pretraining resources, and the paper does not provide matched compute/FLOP/parameter or wall-clock budgets for the full semantic-plus-action system.
  3. The dynamic benchmarks are largely author-designed and use narrow scripted perturbations (5 cm object shifts and seeded velocity impulses). These tests are useful diagnostics but may favor a fast local expert and do not establish robustness to diverse real-world dynamics.
  4. The physical evaluation is small: three tabletop tasks with ten trials each, using a progress score rather than a binary task-success analysis. Confidence intervals, per-task failures, and results across objects, operators, or environments are not reported.
  5. The semantic memory is fixed-width and transient, and the policy depends on a chosen staleness distribution. Performance outside the tested 0.5--5 Hz range, under communication drops, or on longer-horizon tasks is unknown.
  6. The reported real-world improvement is moderate (31.9% to 37.5% in the ARP-DR OID prefix-conditioning comparison), while the strongest dynamic gains come from custom simulation. The claims should distinguish architectural potential from demonstrated deployment reliability.
Questions for authors
  1. Please compare ARP against AsyncVLA, AR-VLA, VLASH, RTC, and a vision-gated VLA using the same backbone, data, parameter budget, hardware, and VLM/action rates.
Reviewer_Rdz7OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes a dual-system VLA, which has a slow VLM that updates a fixed-size memory at a low rate, and the memory gets fed into a fast action expert that acts on fresh images with the stale memory. In training, they randomize memory ages. The paper reports gains over a synchronous π0 on LIBERO, and even larger gains on two dynamic-perturbation tasks. And the method has little performance degradation when the VLM runs 10 times less often.

Critical strengths
  1. There are real-world experiments, and the videos are in the supplementary material.
Critical weaknesses
  1. The contribution with respect to prior work is unclear. The intro claims, "To our knowledge, ARP is the first truly asynchronous robotic control framework." What does "truly" even mean? Why are Helix and HiRT not truly async?
  2. The experiments of the paper do not show the VLM memory is even needed. LIBERO seems to be easy, because there are even people showing that a tiny policy can perform well on it (arXiv:2606.04233). And LIBERO-Perturb does not seem difficult, because even Diffusion Policy can do well on it (Fig 1). Since the tasks do not show the necessity of the VLM, the ablations and other experiments on these tasks are not informative.
Questions for authors
  1. How does the same action expert do without the VLM memory?
  2. Why are HiRT and Helix not "truly" asynchronous?
  3. How big is the action expert?
Reviewer_ycaoOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

ARP separates a slow vision-language model from a fast vision-action expert through asynchronous semantic memory. The action expert receives current observations and is trained with stale memory and execution-delay conditioning. Experiments evaluate manipulation performance, perturbation recovery, and robustness to reduced semantic-update frequencies in simulation and on a Franka robot.

Critical strengths

The staleness-training ablation shows that exposure to delayed context improves robustness when semantic updates become less frequent.

Critical weaknesses
  1. The paper does not establish a sufficiently distinct contribution beyond existing asynchronous control frameworks. Separating slow semantic processing from fast action generation is already explored, making the claimed conceptual advance unconvincing.

  2. The experiments do not adequately test the proposed contribution. Comparisons mainly against synchronous baselines cannot establish an advantage over existing asynchronous methods or isolate the benefits of the architecture from latency-aware training.

  3. The broader claims exceed the evidence. The limited real-robot evaluation and incomplete accounting of total system efficiency do not convincingly demonstrate scalable, semantically capable, and more efficient robot control.

Questions for authors
  1. What substantive capability does ARP introduce beyond AsyncVLA and AR-VLA?

  2. Does ARP outperform existing asynchronous approaches under matched training, hardware, and execution settings?

  3. What measurements directly support the claims of preserved semantic capability and improved total system efficiency?

Reviewer_VUeCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Asynchronous Robot Policies (ARP) to address the latency and reactiveness issue in larger and larger VLAs/ general robotic policies. ARP decouples high-level semantic reasoning from low-level action generation, allowing a large VLM to update semantic memory at a lower frequency while a lightweight vision-action expert generates actions at a higher frequency with training-time alignment similar to TT-RTC. Experiments show improvements in sim and real-world dynamic tasks, with relatively stable performance even when the VLM update frequency is reduced by 10×.

Critical strengths
  1. The method is clearly presented and easy to follow. The overall architecture and training strategy are well explained.

  2. The experiments show some improvements over the baselines, especially on dynamic tasks and under reduced VLM inference frequencies.

Critical weaknesses
  1. I am not fully convinced about the novelty and training cost of this approach. My understanding is that conventional asynchronous action chunking still runs the VLM and AE sequentially within each inference call, while ARP separates them into independent loops, with the AE continuously reading the latest, potentially stale KV cache. (PLease correct me if I am wrong) This requires training the AE to handle different memory delays, rather than simply applying action-prefix conditioning as in TT-RTC. How much additional training is actually required, and can this be applied to existing pretrained VLAs without joint retraining?

  2. The LIBERO results are not particularly convincing. LIBERO is already a heavily saturated benchmark I would say, some SR over 95% is somehow the lowest bar. Yet the best ARP model only achieves 89.3%, and even the reproduced π0 baselines perform substantially worse than the originally reported π0 results. I understand the authors freeze the vision encoder for controlled comparisons, but this still raises concerns about the overall implementation and whether the proposed architecture compromises policy performance.

  3. I am also unclear about how the simulation benchmarks, especially LIBERO-Perturb, actually evaluate latency and reactivity. Is the simulator running with real inference delays and asynchronous scheduling? (Or you can mannualy simulate that by doing simulated delay and async inference like RTC in kinetix) Otherwise, wouldn't a baseline that simply replans at a fixed, sufficiently high frequency with the latest observations provide the best reactivity? I would like to see a latency-matched baseline and more direct measurements of reaction time.

  4. The real-robot experiments are also limited to three tabletop tasks without dynamic perturbations (and why using manually designed "progress" instead of plain SR?), making it difficult to assess whether the claimed improvements in real-world reactivity actually hold.

  5. From the reported latency and also from my experience, the VLM and the AE can consume likely the same time, and in some cases the latter could be higher. How is the slow-loop and fast-loop assumption to be true at very first?

Questions for authors

See Weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.