acceptodds
Under review as a conference paper at ICLR 2027

D-VLA: Dual-Memory Dual-Frequency Vision-Language-Action Model For Long Dynamic Manipulation

Abstract

Long-horizon manipulation requires robots to remember cues that are no longer in view while responding to moving objects. Yet vision-language-action (VLA) policies often rely on the latest observation, and refreshing their visual context typically requires another costly vision-language model (VLM) pass. We present D-VLA, which combines dual memory and dual-frequency control at the KV-cache interface of a pretrained VLA. D-VLA uses block-wise causal KV caching to encode observations incrementally and, guided by distinct temporal attention patterns, constructs separate historical KV read views for the VLM and action expert. Between periodic VLM updates, a gated adapter incorporates fresh visual features into the latest history-conditioned KV block, while a short fast-memory queue supports action replanning. We introduce DOMINO-Long, a ten-task benchmark requiring robots to use earlier visual cues when manipulating moving objects. D-VLA achieves complete-task success rates of 29.3% on DOMINO, compared with 9.6% for and 17.2% for PUMA, and 60.0% on DOMINO-Long, compared with 35.4% and 20.6%, respectively. It improves success rates on eight real-robot tasks and reaches 97.5% on LIBERO-Long and 74.3% on RoboTwin 2.0.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Vision-Language-Action Model, dynamic manipulation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_G8S1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes D²-VLA, a vision–language–action (VLA) policy that co-designs memory access and update frequency at the KV-cache interface of a pretrained VLM+action-expert stack. The method introduces block-causal temporal modeling with persistent KV reuse, consumer-specific historical KV reads (dual-memory), and a lightweight visual KV adapter that injects fresh observations between sparse VLM refreshes for fast action replanning (dual-frequency). The authors also introduce DOMINO-Long, a ten-task benchmark targeting long-horizon dynamic manipulation, and report substantial gains over π0.5 and PUMA on DOMINO and DOMINO-Long, as well as improvements on LIBERO-Long, RoboTwin 2.0, and eight real-robot tasks.

Critical strengths
  1. The paper identifies and exploits a temporal asymmetry between VLM and action expert, leading to consumer-specific KV selection and decoupled refresh schedules—an elegant, low-level intervention at the cache interface. Block-wise causal KV caching adds temporal memory to a single-frame VLA without retraining the whole model for long sequences; the approach keeps stored blocks complete and only prunes read views, reducing risk of destructive compression.

  2. Clear ablation showing incremental benefits from causal+dual-memory and then from dual-frequency; claims are supported by significant gains (e.g., 29.3% on DOMINO vs 9.6% π0.5; 60.0% on DOMINO-Long vs 35.4% π0.5).

  3. Co-designing memory selection and update frequency at the KV boundary is likely to influence how future VLA systems scale temporal context under latency constraints.

Critical weaknesses
  1. The fast-path adapter relies on an unchanged slow anchor (with instruction context) and omits text in the fast update; potential limitations if subgoals evolve or instructions encode multi-stage logic that shifts within a V cycle.

  2. The online attention-based selector prunes 30% of historical visual tokens (retains 70%) by default; robustness to selector errors, distribution shifts, or noisy attention is not deeply analyzed.

  3. KV positional handling (store raw K/V, reapply RoPE with compacted indices) is reasonable but could introduce subtle temporal distortions; there is no ablation isolating this choice.

  4. Latency and memory overhead are referenced to appendices (Tab. 10, D.2) but are not summarized in the main text; quantitative speed/compute comparisons against frequent full VLM refresh baselines are essential for dual-frequency claims.

  5. Fairness across baselines is not fully transparent: were strong slow–fast/asynchronous baselines (e.g., recent slow-fast/bridge-buffer systems) or AR-pretrained long-context video foundations included? Results against PUMA and π0.5 are strong but additional contemporaries would strengthen the case.

  6. Sensitivity studies for key knobs (V/E ratio, N_slow/N_fast, retention ratio, gating strength) are mentioned as appendices; more main-text highlights would help practitioners.

  7. Recent AR video-pretrained WAM/VLA systems that demonstrate strong gains from long visual history (e.g., Long-WAM) are not discussed; likewise, KV compression/selection alternatives (e.g., learned chunk gating) could be acknowledged more directly.

  8. Contemporary asynchronous fast–slow VLA architectures (e.g., buffer-bridged or hierarchical triggers) present related decoupling strategies; a tighter conceptual comparison would be valuable.

Questions for authors
  1. Could you provide main-text (not only appendix) measurements of end-to-end latency, throughput (Hz), and peak memory for D²-VLA vs π0.5 with frequent VLM refresh (e.g., refresh every E steps), and an ablation over V/E/N_slow/N_fast to show compute–accuracy trade-offs?

  2. How sensitive is performance to the 70% token retention choice and to the attention-based scoring heuristic? Do you observe failure cases where attention misguides selection (e.g., distractors) and how stable are selections across layers/steps?

  3. In the fast path, text is omitted and task information flows only via the slow anchor. Do tasks with evolving subgoals or instruction-conditioned phase changes suffer within a V-cycle? Have you tried injecting a light text encoding into the adapter?

  4. Can you clarify how you avoid positional drift when reassigning compact indices under RoPE at read time? An ablation comparing raw-KV+RoPE-at-read vs storing RoPE-applied KV would be helpful.

  5. How does D²-VLA compare to a strong asynchronous baseline with a latent bridge (e.g., DuoCore-style buffer) under matched compute? If not available, could you discuss expected differences and whether your adapter could complement a bridge buffer?

  6. Will DOMINO-Long be publicly released with scripts and success evaluators? If so, what licenses and splits are planned to encourage standardized reporting?

9 comments
AuthorsOpenReview ↗

Rebuttal to weakness4, weakness6 and question1

Dear Reviewer G8S1,

Thank you for taking the time to provide your valuable and professional suggestions on our paper. We will address each of your questions one by one.

W4: Latency and memory ... for dual-frequency claims. & W6: Sensitivity studies ... would help practitioners. & Q1: Could you provide ... compute–accuracy trade-offs?

We thank you for highlighting the importance of inference efficiency and update-frequency sensitivity. We will include the measured policy response latency, throughput, and peak GPU memory in the main text of the revised paper, alongside the DOMINO ablation results for the update configurations.

(i) Inference efficiency and frequent VLM refresh.

We compare two inference configurations using the same D²-VLA model weights and action replanning frequency (H=E=25H=E=25). Our default configuration refreshes the VLM every 75 steps and uses lightweight KV-adapter updates at intermediate replanning calls. The comparison configuration executes the full VLM every 25 steps with the adapter disabled.

Table : Inference efficiency and success rate on DOMINO.

Inference ConfigurationSR (%) ↑Avg. Latency (ms/chunk) ↓Throughput (chunk/s) ↑Peak GPU Memory (MiB) ↓
D²-VLA + KV Adapter (V=75V=75, E=25E=25)29.3164.9115.4123,235
Full VLM Every Call (V=25V=25)16.2074.5213.4223,491

As shown in Table above, our dual-frequency configuration achieves 13.11 percentage points higher SR, reduces average inference latency by 12.9%, and improves throughput by 14.8% compared with full-VLM refresh at every replanning call. It also slightly reduces peak GPU memory.

For reference, the official DOMINO π0.5\pi_{0.5} baseline achieves 57.49 ms/chunk, 17.40 chunks/s, and 15,303 MiB peak GPU memory under the same single-GPU measurement protocol. This baseline uses a different checkpoint and action horizon (H=50H=50, E=25E=25), and is therefore reported separately from the matched-weight comparison above.

All latency measurements are performed on a single NVIDIA H800 GPU, measured from the policy-side get_action() call until the complete action chunk becomes available, excluding sensing, communication, and physical execution.

(ii) Sensitivity to the VLM update period.

We further investigate how the slow/fast update ratio affects manipulation performance on DOMINO. We fix the action replanning interval at E=25E=25 and vary the VLM refresh interval VV. We also include the slow-only configuration as a reference to quantify the benefit of intermediate adapter updates.

Table: Effect of update frequency on DOMINO success rate.

Inference ConfigurationV/EV/E (steps)Intermediate Adapter Updates per CycleDOMINO SR (%)
Slow-only (fast path disabled)—016.10
D²-VLA (2× update ratio)50/25127.60
D²-VLA (3× update ratio)75/25229.31

Results and analysis. Compared with the slow-only configuration (16.10% SR), introducing intermediate adapter updates increases the success rate to 27.60% at V/E=50/25V/E=50/25 (+11.50 percentage points). Extending the slow VLM refresh interval to V=75V=75, while maintaining the same action replanning frequency, further improves SR to 29.31% (+1.71 percentage points).

These results highlight the importance of decoupling VLM refresh frequency from action replanning frequency. More frequent full-VLM execution does not necessarily improve manipulation performance. We hypothesize that maintaining a relatively stable history-conditioned slow anchor, while incorporating fresh observations through lightweight adapter updates and fast memory, helps preserve temporal consistency during action generation. In contrast, refreshing the full VLM at every replanning step changes the temporal conditioning structure learned during training and removes the intermediate fast-memory pathway, potentially weakening action continuity.

The V/EV/E ablation further shows that the 75/25 configuration achieves the highest success rate among the evaluated update periods, while requiring fewer full-VLM refreshes than the 50/25 configuration. This supports our choice of a slower VLM refresh cycle combined with high-frequency adapter-based action updates.

Overall, these results support the effectiveness of our dual-frequency design: rather than repeatedly executing the full VLM, D²-VLA combines periodic history-conditioned VLM updates with lightweight high-frequency adaptation, achieving improved manipulation success and lower policy response latency under the matched-weight comparison.

AuthorsOpenReview ↗

Rebuttal to weakness1 and question3

W1: The fast-path adapter ... within a V cycle. & Q3: In the fast path ... into the adapter?

We thank the reviewer for raising this concern. In D²-VLA, task information is preserved through the instruction-conditioned slow VLM anchor, while the fast-path adapter incorporates current visual observations to update the action conditioning. Therefore, the fast path can adapt to evolving visual states and transition between subgoals without requiring a full VLM refresh at every replanning step.

(i) Subgoal transitions within a VLM refresh cycle.

We observe that D²-VLA can successfully transition between instruction-conditioned subgoals within a single slow VLM cycle. For example, in a DOMINO-Long Color Memory scenario, the VLM is refreshed at step 225, and the robot has grasped the block by step 244. Following the fast adapter update at step 250, the robot begins transporting the block toward the correct left placement region at step 262, before the next VLM refresh at step 300. This demonstrates that the model can transition from grasping to target-directed transport within the same slow-anchor interval, rather than waiting for another full VLM update.

(ii) Preservation of task-relevant information.

We further evaluate whether earlier task-relevant visual cues continue to influence subsequent decisions through controlled target selection experiments on three DOMINO-Long tasks: Screen Cue, Motion Memory, and Color Memory. For each task, we construct 50 paired scenarios in which only the early visual cue changes, while the subsequent scene, instruction, and robot state at policy takeover remain identical.

Table: Target selection accuracy under controlled early-cue changes on DOMINO-Long.

DOMINO-Long TaskOriginal CueModified CueBoth Correct (Paired)
Screen Cue47/50 (94%)44/50 (88%)41/50 (82%)
Motion Memory36/50 (72%)31/50 (62%)17/50 (34%)
Color Memory47/50 (94%)45/50 (90%)42/50 (84%)
Overall130/150 (86.67%)120/150 (80.00%)100/150 (66.67%)

As shown in Table, D²-VLA achieves 86.67% and 80.00% target selection accuracy under the original and modified cues, respectively. More importantly, in 100 out of 150 paired scenarios (66.67%), the model correctly selects the corresponding different targets despite identical subsequent observations. In particular, Color Memory achieves 84% paired correctness, demonstrating that earlier visual cues continue to guide instruction-conditioned target selection.

Overall, these results provide complementary evidence that the slow-anchor-based conditioning preserves task-relevant information and supports subgoal progression. The within-cycle Color Memory example demonstrates that the model can transition between action phases without waiting for the next full VLM refresh, while the controlled cue-intervention experiments confirm that earlier task information continues to influence subsequent decisions. Together, these observations support the ability of our dual-frequency design to handle evolving subgoals under the existing instruction. Given this evidence that D²-VLA can handle evolving subgoals under a fixed instruction, and given the limited time and computational resources available, we did not train an additional model variant with a new text-encoding scheme.

AuthorsOpenReview ↗

Rebuttal to weakness2 and question2(1/2)

W2: The online attention-based ... is not deeply analyzed. & Q2: How sensitive is ... across layers/steps?

We thank the reviewer for raising concerns about the robustness and stability of our attention-based historical KV selection. We provide additional analyses of retention-ratio sensitivity, selection strategies, failure cases, and selection stability.

(i) Sensitivity to the retention ratio.

As reported in the Appendix, we evaluate eight historical visual-token retention ratios on DOMINO, ranging from 30% to 100%, while keeping the other model and inference configurations unchanged.

Table : Effect of historical visual-token retention ratio on DOMINO success rate.

Historical KV Retention (%)DOMINO SR (%)
3025.70
4026.03
5026.14
6027.66
7029.30
8028.86
9028.71
10028.50

As shown in Table , 70% retention achieves the highest SR (29.30%) among all evaluated settings, motivating our default choice. Performance generally improves as the retention ratio increases from 30% to 70%, while retaining additional tokens beyond 70% does not yield further improvements. This suggests that filtering redundant historical context while preserving task-relevant information can benefit manipulation performance.

(ii) Sensitivity to the attention-based selection strategy.

To further evaluate the effectiveness of attention-based scoring, we compare three historical KV selection strategies under the same 70% token retention budget and model configuration. Random selection randomly retains historical visual tokens at all three historical KV reading points. Shared selection lets the action expert reuse the historical positions selected by the VLM, while retaining the original fast-path selection. Our default independent selection allows each consumer to select historical tokens according to its own attention scores.

Table R12: Comparison of historical KV selection strategies on DOMINO under the same 70% retention budget.

Selection StrategyRetention (%)DOMINO SR (%)
Random Selection7027.89
Shared VLM Selection7028.14
Independent Attention Selection (Ours)7029.31

As shown in Table R12, independent attention-based selection achieves the highest SR (29.31%), outperforming random selection (27.89%) by 1.42 percentage points and shared selection (28.14%) by 1.17 percentage points. These results support the effectiveness of attention-guided historical token selection and provide additional evidence for the benefit of consumer-specific selection under a matched token budget.

Reviewer_jKpmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a memory-enhanced VLA called D²-VLA for long-horizon dynamic manipulation. It combines a block-causal KV caching, separate historical token selection for the VLM and action expert, and a visual adapter for refreshing action conditions between VLM updates. The authors also introduce DOMINO-Long and evaluate the method on dynamic and static simulation benchmarks and eight real-world robotic tasks.

Critical strengths
  1. This method solves a useful practical problem: retaining early task information while responding to changes in the current scene. KV features that update historical conditions with new visual inputs are a sensible interface for combining these requirements into existing VLAs.
  2. The empirical results are encouraging in a variety of contexts. The improvements to DOMINO and DOMINO-Long are substantial, and real robot experiments provide useful evidence beyond simulations. Comparison with uncompressed KV reads also makes the computational trade-offs more concrete.
Critical weaknesses
  1. The benefits of consumer-specific memory selection are not isolated. Comparison with full KV reads supports selecting historical tokens, but does not establish that the VLM and action expert need different selections. On DOMINO, selection improves the success rate from 28.5% to 29.3% without reporting any uncertainty. A shared selector using the same token budget would be a more direct test of the claimed attention asymmetry. This is important because asymmetry is a core motivation for this approach.
  2. The evidence for long-term memory is not as direct as the overall task success results suggest. During rapid updates, the specialist only accesses remote information via the latest slow anchor; it cannot directly revisit earlier slow blocks. This creates a potentially significant bottleneck when old cues become relevant again. DOMINO-Long is well-suited for testing this, but lacks baselines with memory enabled and analysis of changing cue-to-action latency. Therefore, the current results are unclear how much benefit comes from retaining long-distance cues versus improved dynamic execution.
Questions for authors
  1. At the same retention ratio, how does independent selection compare with using one shared selection for the VLM and action expert? Is the difference consistent across evaluation runs?
  2. On DOMINO-Long, how does performance change as the delay between cue presentation and use increases, particularly after the cue-containing block leaves the slow queue? A comparison with the block-causal KV-cache prototype would help interpret the result.
4 comments
AuthorsOpenReview ↗

Rebuttal to weakness1 and question1

Dear Reviewer jKpm,

Thank you highlighting and appreciating our contributions in exploring long dynamic manipulation for Vision-Language-Action models. Below, we will answer your questions one by one.

W1:The benefits of consumer-specific ... for this approach. & Q1: At the same retention ... across evaluation runs?

We thank you for highlighting the need to isolate the contribution of consumer-specific memory selection. To directly address this concern, we conducted an additional controlled ablation comparing independent and shared history selection under identical model weights, KV-cache lengths, inference configurations, and a 70% historical token retention budget. Under shared selection, the action expert reuses the historical visual token positions selected by the VLM, whereas independent selection allows each component to select historical information based on its own attention scores.

Table : Ablation of historical KV selection strategies on DOMINO.

History SelectionToken RetentionDOMINO SR (%)
Full KV Reading100%28.50
Shared Selection70%28.14
Independent Selection (D²-VLA)70%29.31

Results. Under the same 70% token budget, independent selection achieves 29.31% SR, outperforming shared selection (28.14%) by 1.17 percentage points. This comparison directly evaluates the benefit of allowing the VLM and action expert to independently retrieve task-relevant historical information, supporting our motivation for consumer-specific memory selection.

Evaluation reliability. To reduce evaluation randomness, we evaluate each of the 35 DOMINO tasks over 100 episodes per configuration, resulting in 3,500 planned evaluation episodes per setting. This extensive evaluation reduces sensitivity to individual episode outcomes.

Overall, these results provide direct evidence of a modest performance advantage from independent history selection under a matched token budget, supporting the attention asymmetry underlying our dual-memory design.

AuthorsOpenReview ↗

Rebuttal to weakness2 and question2(1/2)

W2:The evidence for ... improved dynamic execution.. & Q2: On DOMINO-Long ... help interpret the result.

We thank the reviewer for raising this important concern. To investigate whether D²-VLA preserves long-range visual information and how performance changes with increasing cue-to-action delays, we conduct controlled evaluations on two tasks from DOMINO-Long: Screen Cue and Color Memory, comparing our method against the block-causal KV-cache prototype. We further use controlled early-cue interventions on three DOMINO-Long tasks to distinguish memory-dependent decision-making from improved dynamic execution.

Experimental setup. For each of the two tasks, we evaluate the same 50 initial scenes under five additional waiting intervals (0, 5, 10, 15, and 20 seconds), keeping each scene’s initial configuration and instruction fixed across conditions. During the first 5 seconds of each rollout, the model runs inference and executes actions as usual. We then hold the environment state fixed for the assigned interval. During this wait, inference continues and the KV cache is updated, but no actions are executed. Action execution resumes when the interval ends. In the 0-second condition, the rollout continues without interruption. By the end of a 10-second wait, the KV entries corresponding to the first frame of the visual cue have been evicted from the memory queue. We report first target selection accuracy, defined as the percentage of trials in which the model’s first target choice matches the visual cue, regardless of whether it subsequently completes the task.

Table : Target selection accuracy (%) under increasing cue-to-action delays on DOMINO-Long.

MethodDOMINO-Long Task0s5s10s15s20s
D²-VLAScreen Cue9496723436
Causal KV-cacheScreen Cue8280622220
D²-VLAColor Memory9476444244
Causal KV-cacheColor Memory7862303028
D²-VLA (Avg.)Both tasks9486583840
Causal KV-cache (Avg.)Both tasks8071462624

Long-range memory retention. The delays in Table denote additional waiting time beyond the inherent cue-to-action interval of 5 seconds. D²-VLA achieves 94% average target-selection accuracy with no additional wait and maintains 86% accuracy at a total interval of 10 seconds. At the longest interval tested, approximately 25 seconds, D²-VLA still selects the correct target on 36% of Screen Cue trials and 44% of Color Memory trials. Its average accuracy thus falls to 40%, a 46-percentage-point drop from 10 seconds.

D²-VLA outperforms the causal KV-cache prototype on both tasks at every delay. Its accuracy gains, averaged over the two tasks, are 14, 15, 12, 12, and 16 percentage points for additional waits of 0, 5, 10, 15, and 20 seconds, respectively. Across all task–delay combinations, D²-VLA achieves 63.2% average accuracy versus 49.4% for the prototype (+13.8 percentage points). These results show that D²-VLA can use an earlier visual cue reliably at 10 seconds. Although target selection becomes less reliable at 20 seconds, D²-VLA retains a 16-percentage-point advantage over causal caching even at this longest interval (40% vs. 24%).

Memory beyond the slow queue. Notably, at 10 additional seconds on Screen Cue, the original cue-containing block has already left the slow-memory queue by the time the model makes its first target selection, yet D²-VLA still achieves 72% accuracy, compared with 62% for the causal prototype. This provides evidence that earlier cue information can continue to influence decisions even when the original block is no longer directly accessible, consistent with information being carried forward through subsequent history-conditioned slow anchors.

AuthorsOpenReview ↗

Rebuttal to weakness2 and question2(2/2)

Distinguishing memory from dynamic execution. To further investigate whether the improvements arise from retaining earlier information rather than better low-level manipulation, we conduct controlled early-cue intervention experiments on three DOMINO-Long tasks: Screen Cue, Motion Memory, and Color Memory. For each task, we construct 50 paired scenarios in which only the early visual cue is changed, while the subsequent scene, original instruction, and robot state at policy takeover remain identical. The modified cue specifies a different correct target, requiring the model to adjust its decision based on previously observed information.

Table : Target selection accuracy under controlled early-cue changes on DOMINO-Long.

DOMINO-Long TaskOriginal CueModified CueBoth Correct (Paired)
Screen Cue47/50 (94%)44/50 (88%)41/50 (82%)
Motion Memory36/50 (72%)31/50 (62%)17/50 (34%)
Color Memory47/50 (94%)45/50 (90%)42/50 (84%)
Overall130/150 (86.67%)120/150 (80.00%)100/150 (66.67%)

Results and analysis. As shown in Table, D²-VLA achieves 86.67% target selection accuracy under the original cues and maintains 80.00% accuracy when the early cues are modified. More importantly, in 100 out of 150 paired scenarios (66.67%), the model correctly selects the corresponding different targets under both cue conditions, despite receiving identical subsequent scenes and robot states at policy takeover.

For the same two DOMINO-Long tasks used in our delay evaluation (Screen Cue and Color Memory), the model achieves 83% paired correctness (83/100 scenarios). Since changing only the earlier cue changes the correct target while the subsequent scene remains identical, these results provide direct evidence that the model's target selection depends on retained visual information rather than solely on the current observation.

Furthermore, we evaluate target selection rather than complete-task success, avoiding confounding the memory-dependent decision with downstream grasping and placement success. This directly addresses the reviewer's concern that improvements might arise solely from better dynamic execution.

Overall, these complementary experiments show that D²-VLA maintains stronger long-range cue utilization than the causal KV-cache prototype across the evaluated delays, even though memory-dependent accuracy decreases as the delay increases. The controlled early-cue interventions further demonstrate that previously observed visual information directly influences subsequent target selection. Together, these findings support the contribution of long-term visual memory beyond improvements in dynamic execution.

AuthorsOpenReview ↗

Follow-up on Our Response to Your Review

Dear Reviewer jKpm,

Thank you for your time and thoughtful feedback! We have provided detailed responses to your comments and hope they address your main concerns. As the discussion period is approaching its end, we would greatly appreciate any further feedback you may have, particularly on whether any concerns remain unresolved. We would be happy to provide additional clarification. Thank you again for your time and consideration!

Best Regards,

Submission 8382 Authors

Reviewer_Z4khOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes D²-VLA, which combines historical memory and dual-frequency control at the KV interface of π₀.₅. Through block-causal training, the method reuses KV representations from past observations and independently selects historical visual tokens for the VLM and action expert. Between VLM updates, it uses current images and a lightweight adapter to refine the latest history-conditioned KV block, while maintaining a short queue to support action replanning.

Critical strengths
  1. The method integrates historical caching, consumer-specific reads, and visual residual updates into an existing VLA, shares the action expert, and trains solely with the action objective, while providing detailed cache layouts, gated updates, and inference procedures that establish a clear implementation path.

  2. D²-VLA outperforms π₀.₅ by 24.6 percentage points on DOMINO-Long, consistently surpasses both π₀.₅ and MemoryVLA on the real-world dynamic tasks, and achieves strong results on static benchmarks with the high-frequency pathway disabled, demonstrating benefits across different manipulation settings.

Critical weaknesses
  1. Table 3 introduces causal training and independent history selection together, while the dual-frequency ablation does not isolate the effects of visual updates, fast memory, and training configuration, so the results establish the effectiveness of the overall system but do not show how much the dual-memory, dual-frequency design itself contributes.

  2. DOMINO-Long lacks controlled tests that vary earlier cues while keeping subsequent scenes identical, or clear the history before a decision, leaving it unclear whether the success-rate gains come from retaining earlier information or simply from better grasping and placement.

Questions for authors
  1. With the model, cache length, and token budget held fixed, how much does selecting history separately for the VLM and action expert improve performance over shared selection?

  2. At the same replanning frequency, how do adapter updates compare with running the full VLM at every call in terms of success rate and actual response latency?

  3. If subsequent scenes remain identical and only the earlier cues change, can the model correctly adjust its target selection accordingly?

  4. How are the DOMINO-Long test configurations separated from the training demonstrations to prevent the model from memorizing fixed scene–action associations?

6 comments
AuthorsOpenReview ↗

Rebuttal to weakness1 and question1

Dear Reviewer Z4kh,

Thank you for taking the time to provide your valuable and professional suggestions on our paper. We will address each of your questions one by one.

W1:Table 3 introduces ... design itself contributes.

We thank you for highlighting the need to disentangle the contributions of causal training, dual-memory selection, and dual-frequency updates. To address this concern, we conducted a more fine-grained ablation study. All variants follow the same training configuration, including 75K training steps and identical training hyperparameters, with changes restricted to the components under investigation.

Table : Component-wise ablation on DOMINO. All variants are trained for 75K steps with matched training hyperparameters.

ConfigurationSR (%)MS
(1) π₀.₅ Baseline9.6026.17
(2) + Causal training14.9031.46
(3) + Independent history selection16.1033.36
(4) + Dual-frequency adapter, w/o fast history20.1036.76
(5) Full dual-frequency, shared selection28.1440.56
(6) Full D²-VLA, independent selection29.3140.61

The results allow us to disentangle the contributions of individual components:

(i) Causal training. Comparing (1) and (2), causal training improves SR from 9.60% to 14.90% (+5.30 percentage points), separating its contribution from subsequent memory selection mechanisms.

(ii) Independent history selection. Adding consumer-specific history selection to the causal model improves SR from 14.90% to 16.10% ((2) vs. (3)). More importantly, under the full dual-frequency configuration, independent selection outperforms shared selection (29.31% vs. 28.14%, (6) vs. (5)) with the same 70% historical token retention budget. This directly isolates the benefit of allowing the VLM and diffusion expert to select historical information independently.

(iii) Adapter-based visual updates. Comparing (3) and (4), introducing dual-frequency adapter updates while disabling accumulated fast history improves SR from 16.10% to 20.10% (+4.00 percentage points). This isolates the contribution of intermediate adapter-based visual updates without the additional benefit of retaining previous fast-path KV states.

(iv) Fast memory. Comparing (4) and (6), enabling fast history while preserving the adapter updates and independent selection further improves SR from 20.10% to 29.31% (+9.21 percentage points). This demonstrates that accumulating fast-path temporal information provides substantial additional benefits beyond individual visual updates.

Overall, relative to the causal model with independent history selection (16.10% SR), the complete dual-frequency design achieves 29.31% SR, a gain of 13.21 percentage points. The controlled comparisons attribute this improvement to both adapter-based intermediate updates (+4.00 percentage points) and fast memory (+9.21 percentage points). Together with the matched shared-selection comparison, these results provide component-level evidence that the dual-memory, dual-frequency design contributes beyond causal training alone.

Q1:With the model ... over shared selection?

To isolate the contribution of independent history selection, we compare independent and shared selection using the same D²-VLA model weights, KV-cache lengths, 70% historical token retention budget, and inference configuration. Under shared selection, the action expert reuses the historical visual token positions selected by the VLM, whereas independent selection allows each component to select historical tokens based on its own attention scores.

To reduce evaluation randomness, we conduct 100 evaluation trials per task across all 35 DOMINO tasks (3,500 scheduled trials per configuration).

Table : Independent vs. shared history selection on DOMINO.

History SelectionToken RetentionDOMINO SR (%)
Shared (VLM-selected positions)70%28.14
Independent (D²-VLA)70%29.31

Results. Under identical model, cache, and token-budget settings, independent selection improves the success rate from 28.14% to 29.31% (+1.17 percentage points). This controlled comparison supports the benefit of consumer-specific memory selection, allowing the VLM and action expert to retrieve different historical information according to their respective requirements.

AuthorsOpenReview ↗

Rebuttal to weakness2 and question3

W2:DOMINO-Long lacks ... better grasping and placement.&Q3:If subsequent scenes ... target selection accordingly?

We thank the reviewer for this important suggestion. To directly verify whether the model retains and utilizes earlier visual information, rather than merely benefiting from improved manipulation capabilities, we conducted a controlled paired evaluation on three tasks from our DOMINO-Long benchmark: Screen Cue, Motion Memory, and Color Memory.

Experimental setup. For each task, we construct 50 paired scenarios (150 pairs in total). Within each pair, we modify only the early visual cue while keeping the subsequent scene, scene seed, original instruction, and the visual observation and robot state at policy takeover identical. The modified cue specifies a different correct target, requiring the model to make a different decision despite observing the same subsequent scene.

Importantly, we measure target selection accuracy rather than overall task success rate, allowing us to evaluate whether the model correctly utilizes earlier cues without conflating target selection with subsequent manipulation success. Specifically, for Screen Cue and Motion Memory, we evaluate the first clearly approached object; for Color Memory, we evaluate the first selected placement region after grasping the block.

Table : Target selection accuracy under controlled early-cue changes on DOMINO-Long.

DOMINO-Long TaskOriginal CueModified Cue
Screen Cue47/50 (94%)44/50 (88%)
Motion Memory36/50 (72%)31/50 (62%)
Color Memory47/50 (94%)45/50 (90%)
Overall130/150 (86.67%)120/150 (80.00%)

Results and analysis. As shown in Table, the model achieves 86.67% target selection accuracy under the original cues and maintains 80.00% accuracy when the early cues are modified. More importantly, in 100 out of 150 paired scenarios (66.67%), the model correctly selects the corresponding different targets under both cue conditions, despite receiving identical subsequent visual observations and robot states.

These results directly address your concern: when the subsequent scene remains identical, changing only the earlier cue can lead the model to correctly adjust its target selection. Since the correct target depends on information presented earlier rather than the current visual observation, this provides direct evidence that D²-VLA retains and utilizes earlier visual cues for subsequent decision-making.

Furthermore, evaluating target selection instead of complete-task success helps distinguish the contribution of visual memory from improvements in downstream grasping and placement execution. Therefore, the observed performance cannot be explained solely by better low-level manipulation, supporting the effectiveness of long-term visual memory in DOMINO-Long.

AuthorsOpenReview ↗

Rebuttal to question2 and question4

Q2: At the same ... actual response latency?

We thank the reviewer for this question. In A1, the VLM is refreshed every 75 steps, while the adapter incorporates fresh visual observations at intermediate replanning calls. In A2, the full VLM is executed every 25 steps with the adapter disabled. Both configurations generate a new action chunk every 25 steps.

Table : Success rate, inference latency, throughput, and peak GPU memory on DOMINO.

Inference ConfigurationSR (%) ↑Avg. Latency (ms/chunk) ↓Throughput (chunk/s) ↑Peak GPU Memory (MiB) ↓
A1: D²-VLA + KV Adapter (V=75V=75,E=25E=25)29.3164.9115.4123,235
A2: Full VLM Every Call (V=25V=25)16.2074.5213.4223,491

Results. As shown in Table, our dual-frequency strategy achieves a higher success rate (29.31% vs. 16.20%, +13.11 percentage points) while reducing average inference latency from 74.52 to 64.91 ms/chunk (12.9% reduction) and improving throughput from 13.42 to 15.41 chunks/s. Peak GPU memory is also slightly reduced. Latency is measured from the policy-side get_action() call until the complete action chunk is available on a single NVIDIA H800 GPU, excluding sensing, communication, and action execution.

Analysis. Refreshing the full VLM more frequently does not necessarily improve performance. We hypothesize that frequent full-VLM refreshes may disrupt the temporal consistency of the learned action conditioning. Our training employs a slow VLM anchor with intermediate adapter-refined KV updates and short-term fast memory. Switching to full-VLM inference at every replanning step changes this learned temporal conditioning structure and removes the intermediate fast-memory pathway, potentially weakening temporal continuity during action generation. This is a plausible explanation rather than a separately verified causal mechanism.

In all, these results support our dual-frequency design: maintaining a relatively stable slow VLM anchor while using lightweight adapter updates enables efficient high-frequency replanning, achieving both higher success rates and lower inference latency than full-VLM refresh at every call.

Q4: How are the DOMINO-Long ... fixed scene–action associations?

To prevent memorization of fixed scene–action associations, DOMINO-Long uses distinct scene-generation seeds for training demonstrations and test configurations, ensuring that test scenes do not reuse training scene seeds. Each DOMINO-Long task contains 50 different held-out initial scenes, allowing us to evaluate generalization to unseen scene configurations. For example, Color Memory keeps the block lanes and placement regions fixed and uses the scene seed to sample two cue colors independently, with the first identifying the block to grasp and the second identifying the region in which to place it.

Furthermore, in our controlled paired evaluation on three DOMINO-Long tasks, we modify only the early visual cue while keeping the subsequent scene and robot state identical. This requires the model to select different correct targets under identical subsequent observations, making it insufficient to rely solely on memorized scene–action associations.

Together, the disjoint training/test seeds and controlled cue variations reduce the risk of training-scene memorization and provide evidence that the model generalizes beyond fixed scene–action associations.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised Manuscript

Dear Reviewers and Area Chairs,

Thank you for your constructive comments and suggestions. We have revised the main manuscript by expanding Ablation Studies with additional experiments and analyses addressing component contributions, memory-dependent decision-making, and inference efficiency. The revised content is highlighted in blue.

Specifically, we have made the following changes in the main paper:

  1. Expanded the component-wise ablation on DOMINO. We separately evaluate causal training, independent history selection, intermediate adapter updates, and fast memory under matched training settings, clarifying the contribution of each component.

  2. Added a controlled comparison of historical KV selection strategies. We compare full-KV reading, shared selection, and independent selection using identical model weights and cache lengths. The matched 70% retention comparison isolates the benefit of consumer-specific history selection.

  3. Added an inference efficiency comparison on DOMINO. We report success rate, inference latency, throughput, and peak GPU memory for adapter-based updates versus full-VLM execution at every replanning call, using identical weights and replanning frequencies. We also clarify the latency measurement protocol and the interpretation of this inference-time comparison.

  4. Added controlled early-cue intervention experiments on DOMINO-Long. We change only the earlier visual cue while keeping subsequent observations and robot states identical. By measuring target selection accuracy separately from manipulation success, these experiments assess whether earlier cues influence subsequent decisions.

  5. Added evaluations under increasing cue-to-action delays. We compare D²-VLA with the causal KV-cache baseline across five delay settings on Screen Cue and Color Memory, examining how memory-dependent target selection changes with longer delays and discussing performance after the original cue block leaves the slow-memory queue.

Thank you again for your time and feedback, which have helped us strengthen the experimental evidence and clarify the scope of our conclusions.

Best regards,
Submission 8382 Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.