- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes D²-VLA, a vision–language–action (VLA) policy that co-designs memory access and update frequency at the KV-cache interface of a pretrained VLM+action-expert stack. The method introduces block-causal temporal modeling with persistent KV reuse, consumer-specific historical KV reads (dual-memory), and a lightweight visual KV adapter that injects fresh observations between sparse VLM refreshes for fast action replanning (dual-frequency). The authors also introduce DOMINO-Long, a ten-task benchmark targeting long-horizon dynamic manipulation, and report substantial gains over π0.5 and PUMA on DOMINO and DOMINO-Long, as well as improvements on LIBERO-Long, RoboTwin 2.0, and eight real-robot tasks.
Critical strengths
-
The paper identifies and exploits a temporal asymmetry between VLM and action expert, leading to consumer-specific KV selection and decoupled refresh schedules—an elegant, low-level intervention at the cache interface. Block-wise causal KV caching adds temporal memory to a single-frame VLA without retraining the whole model for long sequences; the approach keeps stored blocks complete and only prunes read views, reducing risk of destructive compression.
-
Clear ablation showing incremental benefits from causal+dual-memory and then from dual-frequency; claims are supported by significant gains (e.g., 29.3% on DOMINO vs 9.6% π0.5; 60.0% on DOMINO-Long vs 35.4% π0.5).
-
Co-designing memory selection and update frequency at the KV boundary is likely to influence how future VLA systems scale temporal context under latency constraints.
Critical weaknesses
-
The fast-path adapter relies on an unchanged slow anchor (with instruction context) and omits text in the fast update; potential limitations if subgoals evolve or instructions encode multi-stage logic that shifts within a V cycle.
-
The online attention-based selector prunes 30% of historical visual tokens (retains 70%) by default; robustness to selector errors, distribution shifts, or noisy attention is not deeply analyzed.
-
KV positional handling (store raw K/V, reapply RoPE with compacted indices) is reasonable but could introduce subtle temporal distortions; there is no ablation isolating this choice.
-
Latency and memory overhead are referenced to appendices (Tab. 10, D.2) but are not summarized in the main text; quantitative speed/compute comparisons against frequent full VLM refresh baselines are essential for dual-frequency claims.
-
Fairness across baselines is not fully transparent: were strong slow–fast/asynchronous baselines (e.g., recent slow-fast/bridge-buffer systems) or AR-pretrained long-context video foundations included? Results against PUMA and π0.5 are strong but additional contemporaries would strengthen the case.
-
Sensitivity studies for key knobs (V/E ratio, N_slow/N_fast, retention ratio, gating strength) are mentioned as appendices; more main-text highlights would help practitioners.
-
Recent AR video-pretrained WAM/VLA systems that demonstrate strong gains from long visual history (e.g., Long-WAM) are not discussed; likewise, KV compression/selection alternatives (e.g., learned chunk gating) could be acknowledged more directly.
-
Contemporary asynchronous fast–slow VLA architectures (e.g., buffer-bridged or hierarchical triggers) present related decoupling strategies; a tighter conceptual comparison would be valuable.
Questions for authors
-
Could you provide main-text (not only appendix) measurements of end-to-end latency, throughput (Hz), and peak memory for D²-VLA vs π0.5 with frequent VLM refresh (e.g., refresh every E steps), and an ablation over V/E/N_slow/N_fast to show compute–accuracy trade-offs?
-
How sensitive is performance to the 70% token retention choice and to the attention-based scoring heuristic? Do you observe failure cases where attention misguides selection (e.g., distractors) and how stable are selections across layers/steps?
-
In the fast path, text is omitted and task information flows only via the slow anchor. Do tasks with evolving subgoals or instruction-conditioned phase changes suffer within a V-cycle? Have you tried injecting a light text encoding into the adapter?
-
Can you clarify how you avoid positional drift when reassigning compact indices under RoPE at read time? An ablation comparing raw-KV+RoPE-at-read vs storing RoPE-applied KV would be helpful.
-
How does D²-VLA compare to a strong asynchronous baseline with a latent bridge (e.g., DuoCore-style buffer) under matched compute? If not available, could you discuss expected differences and whether your adapter could complement a bridge buffer?
-
Will DOMINO-Long be publicly released with scripts and success evaluators? If so, what licenses and splits are planned to encourage standardized reporting?
9 comments
Rebuttal to weakness1 and question3
W1: The fast-path adapter ... within a V cycle. & Q3: In the fast path ... into the adapter?
We thank the reviewer for raising this concern. In D²-VLA, task information is preserved through the instruction-conditioned slow VLM anchor, while the fast-path adapter incorporates current visual observations to update the action conditioning. Therefore, the fast path can adapt to evolving visual states and transition between subgoals without requiring a full VLM refresh at every replanning step.
(i) Subgoal transitions within a VLM refresh cycle.
We observe that D²-VLA can successfully transition between instruction-conditioned subgoals within a single slow VLM cycle. For example, in a DOMINO-Long Color Memory scenario, the VLM is refreshed at step 225, and the robot has grasped the block by step 244. Following the fast adapter update at step 250, the robot begins transporting the block toward the correct left placement region at step 262, before the next VLM refresh at step 300. This demonstrates that the model can transition from grasping to target-directed transport within the same slow-anchor interval, rather than waiting for another full VLM update.
(ii) Preservation of task-relevant information.
We further evaluate whether earlier task-relevant visual cues continue to influence subsequent decisions through controlled target selection experiments on three DOMINO-Long tasks: Screen Cue, Motion Memory, and Color Memory. For each task, we construct 50 paired scenarios in which only the early visual cue changes, while the subsequent scene, instruction, and robot state at policy takeover remain identical.
Table: Target selection accuracy under controlled early-cue changes on DOMINO-Long.
| DOMINO-Long Task | Original Cue | Modified Cue | Both Correct (Paired) |
|---|---|---|---|
| Screen Cue | 47/50 (94%) | 44/50 (88%) | 41/50 (82%) |
| Motion Memory | 36/50 (72%) | 31/50 (62%) | 17/50 (34%) |
| Color Memory | 47/50 (94%) | 45/50 (90%) | 42/50 (84%) |
| Overall | 130/150 (86.67%) | 120/150 (80.00%) | 100/150 (66.67%) |
As shown in Table, D²-VLA achieves 86.67% and 80.00% target selection accuracy under the original and modified cues, respectively. More importantly, in 100 out of 150 paired scenarios (66.67%), the model correctly selects the corresponding different targets despite identical subsequent observations. In particular, Color Memory achieves 84% paired correctness, demonstrating that earlier visual cues continue to guide instruction-conditioned target selection.
Overall, these results provide complementary evidence that the slow-anchor-based conditioning preserves task-relevant information and supports subgoal progression. The within-cycle Color Memory example demonstrates that the model can transition between action phases without waiting for the next full VLM refresh, while the controlled cue-intervention experiments confirm that earlier task information continues to influence subsequent decisions. Together, these observations support the ability of our dual-frequency design to handle evolving subgoals under the existing instruction. Given this evidence that D²-VLA can handle evolving subgoals under a fixed instruction, and given the limited time and computational resources available, we did not train an additional model variant with a new text-encoding scheme.
Rebuttal to weakness2 and question2(1/2)
W2: The online attention-based ... is not deeply analyzed. & Q2: How sensitive is ... across layers/steps?
We thank the reviewer for raising concerns about the robustness and stability of our attention-based historical KV selection. We provide additional analyses of retention-ratio sensitivity, selection strategies, failure cases, and selection stability.
(i) Sensitivity to the retention ratio.
As reported in the Appendix, we evaluate eight historical visual-token retention ratios on DOMINO, ranging from 30% to 100%, while keeping the other model and inference configurations unchanged.
Table : Effect of historical visual-token retention ratio on DOMINO success rate.
| Historical KV Retention (%) | DOMINO SR (%) |
|---|---|
| 30 | 25.70 |
| 40 | 26.03 |
| 50 | 26.14 |
| 60 | 27.66 |
| 70 | 29.30 |
| 80 | 28.86 |
| 90 | 28.71 |
| 100 | 28.50 |
As shown in Table , 70% retention achieves the highest SR (29.30%) among all evaluated settings, motivating our default choice. Performance generally improves as the retention ratio increases from 30% to 70%, while retaining additional tokens beyond 70% does not yield further improvements. This suggests that filtering redundant historical context while preserving task-relevant information can benefit manipulation performance.
(ii) Sensitivity to the attention-based selection strategy.
To further evaluate the effectiveness of attention-based scoring, we compare three historical KV selection strategies under the same 70% token retention budget and model configuration. Random selection randomly retains historical visual tokens at all three historical KV reading points. Shared selection lets the action expert reuse the historical positions selected by the VLM, while retaining the original fast-path selection. Our default independent selection allows each consumer to select historical tokens according to its own attention scores.
Table R12: Comparison of historical KV selection strategies on DOMINO under the same 70% retention budget.
| Selection Strategy | Retention (%) | DOMINO SR (%) |
|---|---|---|
| Random Selection | 70 | 27.89 |
| Shared VLM Selection | 70 | 28.14 |
| Independent Attention Selection (Ours) | 70 | 29.31 |
As shown in Table R12, independent attention-based selection achieves the highest SR (29.31%), outperforming random selection (27.89%) by 1.42 percentage points and shared selection (28.14%) by 1.17 percentage points. These results support the effectiveness of attention-guided historical token selection and provide additional evidence for the benefit of consumer-specific selection under a matched token budget.
Rebuttal to weakness4, weakness6 and question1
Dear Reviewer G8S1,
Thank you for taking the time to provide your valuable and professional suggestions on our paper. We will address each of your questions one by one.
We thank you for highlighting the importance of inference efficiency and update-frequency sensitivity. We will include the measured policy response latency, throughput, and peak GPU memory in the main text of the revised paper, alongside the DOMINO ablation results for the update configurations.
(i) Inference efficiency and frequent VLM refresh.
We compare two inference configurations using the same D²-VLA model weights and action replanning frequency (H=E=25). Our default configuration refreshes the VLM every 75 steps and uses lightweight KV-adapter updates at intermediate replanning calls. The comparison configuration executes the full VLM every 25 steps with the adapter disabled.
Table : Inference efficiency and success rate on DOMINO.
As shown in Table above, our dual-frequency configuration achieves 13.11 percentage points higher SR, reduces average inference latency by 12.9%, and improves throughput by 14.8% compared with full-VLM refresh at every replanning call. It also slightly reduces peak GPU memory.
For reference, the official DOMINO π0.5 baseline achieves 57.49 ms/chunk, 17.40 chunks/s, and 15,303 MiB peak GPU memory under the same single-GPU measurement protocol. This baseline uses a different checkpoint and action horizon (H=50, E=25), and is therefore reported separately from the matched-weight comparison above.
All latency measurements are performed on a single NVIDIA H800 GPU, measured from the policy-side
get_action()call until the complete action chunk becomes available, excluding sensing, communication, and physical execution.(ii) Sensitivity to the VLM update period.
We further investigate how the slow/fast update ratio affects manipulation performance on DOMINO. We fix the action replanning interval at E=25 and vary the VLM refresh interval V. We also include the slow-only configuration as a reference to quantify the benefit of intermediate adapter updates.
Table: Effect of update frequency on DOMINO success rate.
Results and analysis. Compared with the slow-only configuration (16.10% SR), introducing intermediate adapter updates increases the success rate to 27.60% at V/E=50/25 (+11.50 percentage points). Extending the slow VLM refresh interval to V=75, while maintaining the same action replanning frequency, further improves SR to 29.31% (+1.71 percentage points).
These results highlight the importance of decoupling VLM refresh frequency from action replanning frequency. More frequent full-VLM execution does not necessarily improve manipulation performance. We hypothesize that maintaining a relatively stable history-conditioned slow anchor, while incorporating fresh observations through lightweight adapter updates and fast memory, helps preserve temporal consistency during action generation. In contrast, refreshing the full VLM at every replanning step changes the temporal conditioning structure learned during training and removes the intermediate fast-memory pathway, potentially weakening action continuity.
The V/E ablation further shows that the 75/25 configuration achieves the highest success rate among the evaluated update periods, while requiring fewer full-VLM refreshes than the 50/25 configuration. This supports our choice of a slower VLM refresh cycle combined with high-frequency adapter-based action updates.
Overall, these results support the effectiveness of our dual-frequency design: rather than repeatedly executing the full VLM, D²-VLA combines periodic history-conditioned VLM updates with lightweight high-frequency adaptation, achieving improved manipulation success and lower policy response latency under the matched-weight comparison.