- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
The paper proposes JADE, which incorporates a pretrained VLM into iterative action denoising. A frozen OAT encoder maps noisy actions to compact latents, which the VLM processes alongside observation context to guide the action expert. An auxiliary objective trains the VLM to recover clean action latents. Experiments on RoboTwin 2.0, RoboDojo, and four real-world bimanual tasks show improvements over . Using fewer denoising steps partially offsets the higher per-step cost.
Critical strengths
- The approach makes VLM conditioning depend on the evolving action candidate, with compact registers and prefix caching limiting the computational overhead.
- Evaluation spans two simulation benchmarks and real-world manipulation, with consistent improvements over and other baselines.
- Ablations cover conditioning, alignment, action representations, and feature updates. The 2-step variant improves success rates at approximately the latency of 10-step .
Critical weaknesses
- The ablations do not fully isolate the benefit of VLM processing from that of OAT representations and the additional fusion pathway.
- The main tables emphasize 10-step JADE, which is approximately slower than . The near-matched-latency variant offers a smaller RoboDojo gain: 2.76 rather than 5.76 percentage points. This distinction should be clearer in the main results.
Questions for authors
- How would a separately trained baseline perform if OAT latents were passed directly to the action expert through an adapter and the same fusion mechanism, bypassing the VLM?
- Could the main tables include latency and separate rows for 2-step, Balanced, and 10-step JADE, ideally alongside latency for the selected baselines under a common protocol?
- For the Queries variants in Table 3(a), do VLM features depend on the noisy action or timestep, and are they recomputed at each step? Are these variants trained separately using the same fusion architecture as JADE?
4 comments
Response to Reviewer TkTF
Dear Reviewer TkTF,
I hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.
Thank you for your time and effort in reviewing our paper.
Best regards,
The Authors
Thank you for the additional experiments and clarifications. The ablation addressing W1 resolves my main concern, so I am raising my recommendation to Accept.
Dear Reviewer TkTF,
Thank you for taking the time to review our additional experiments and share your update. We’re glad that the new ablation addressed the main concern you raised in W1. We appreciate your thoughtful feedback throughout the review process.
Best regards,
The Authors
Response to Reviewer TkTF
Thank you for your careful review and constructive feedback. We address your concerns below.
W1 & Q1: Contribution of VLM Processing
To disentangle the contributions of VLM processing and the fusion mechanism, we conduct three separately trained ablations on RoboTwin C2R, under the same training and evaluation protocol:
VLM processing: Compared with Direct-OAT, JADE improves Clean and Random success rates by 7.6 and 7.3 percentage points, respectively, despite using the same OAT representations and fusion mechanism. This demonstrates that VLM processing of noisy-action representations provides substantial benefits beyond OAT representations and the additional fusion pathway alone.
Fusion design: AdaLN and Single-layer Fusion both retain VLM processing but underperform full JADE by 4.4 and 4.6 percentage points on average, respectively, further supporting the effectiveness of JADE's layer-wise cross-attention design.
W2 & Q2: Latency and Speed–Accuracy Trade-off
We agree that gains at comparable latency should be distinguished from those obtained with a larger inference budget. Two-step, Balanced, and ten-step JADE take approximately 78, 132, and 203 ms per policy query, respectively, compared with 74 ms for ten-step native π₀.₅. On RoboDojo, two-step JADE improves success by 2.76 pp at nearly matched latency, while ten-step JADE achieves a 5.76 pp gain with a larger inference budget. These timings measure model-side generation of a 50-action chunk, excluding communication, simulation, and robot execution. We will report these configurations separately in the main tables, alongside baselines timed under a unified protocol, to make the performance–latency trade-off explicit.
Q3: Implementation Details of Queries Variants
The Queries variants replace the eight OAT latents with eight learnable query tokens. Unlike OAT latents, which encode the current noisy action, Queries do not directly depend on the noisy action but retain the same timestep conditioning as JADE.
VLM features are recomputed at every denoising step with the updated timestep conditioning, rather than being fixed or reused across steps.
All Queries variants are independently trained using the same architecture and training protocol as JADE, with matched alignment settings (λ_align = 0 / 0.1). Therefore, the key controlled difference is whether the VLM processes structured representations of the current noisy action or learnable queries. We will clarify these implementation details in the revised manuscript.