acceptodds
Under review as a conference paper at ICLR 2027

Vision-Language Models Are Joint Denoisers in Robotic Action Generation

Abstract

Diffusion-based vision-language-action (VLA) models cascade pretrained vision-language models (VLMs) with diffusion action experts, but typically restrict the VLM to providing static conditioning. This separation limits how much pretrained perception and reasoning can guide action generation, especially in out-of-distribution settings. We introduce JADE, a Joint Action DEnoiser that enables the VLM and action expert to jointly refine actions. Our key insight is that structured action latents allow the VLM to read the current noisy action together with visual observations and language instructions. At every denoising step, the VLM contextualizes the noisy action and guides the action expert through gated cross-attention, thereby its guidance evolves with the action being generated. We further supervise the VLM to recover clean action latents from noisy ones, which gives it a direct action-learning signal and couples perception with control. Additionally, joint denoising enables JADE to generate actions with fewer sampling steps, helping offset the additional cost of VLM participation. An adaptive step-size schedule further balances task performance and inference latency. On RoboDojo, JADE achieves an average success rate of 12.67%, the highest among the evaluated -based methods, and is competitive with recent VLA and WAM foundation models pretrained at scale. It also improves performance on RoboTwin 2.0 and raises the average success rate across four real-world bimanual tasks by 20.8 percentage points compared with .

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Flow Matching, Multimodal Reasoning, Action Generation, Robot Learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.80 over 5 reviews, above 85% of the others.
Reviewer_TkTFOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

The paper proposes JADE, which incorporates a pretrained VLM into iterative action denoising. A frozen OAT encoder maps noisy actions to compact latents, which the VLM processes alongside observation context to guide the action expert. An auxiliary objective trains the VLM to recover clean action latents. Experiments on RoboTwin 2.0, RoboDojo, and four real-world bimanual tasks show improvements over π0.5\pi_{0.5}. Using fewer denoising steps partially offsets the higher per-step cost.

Critical strengths
  1. The approach makes VLM conditioning depend on the evolving action candidate, with compact registers and prefix caching limiting the computational overhead.
  2. Evaluation spans two simulation benchmarks and real-world manipulation, with consistent improvements over π0.5\pi_{0.5} and other baselines.
  3. Ablations cover conditioning, alignment, action representations, and feature updates. The 2-step variant improves success rates at approximately the latency of 10-step π0.5\pi_{0.5}.
Critical weaknesses
  1. The ablations do not fully isolate the benefit of VLM processing from that of OAT representations and the additional fusion pathway.
  2. The main tables emphasize 10-step JADE, which is approximately 2.7×2.7\times slower than π0.5\pi_{0.5}. The near-matched-latency variant offers a smaller RoboDojo gain: 2.76 rather than 5.76 percentage points. This distinction should be clearer in the main results.
Questions for authors
  1. How would a separately trained baseline perform if OAT latents were passed directly to the action expert through an adapter and the same fusion mechanism, bypassing the VLM?
  2. Could the main tables include latency and separate rows for 2-step, Balanced, and 10-step JADE, ideally alongside latency for the selected baselines under a common protocol?
  3. For the Queries variants in Table 3(a), do VLM features depend on the noisy action or timestep, and are they recomputed at each step? Are these variants trained separately using the same fusion architecture as JADE?
4 comments
AuthorsOpenReview ↗

Response to Reviewer TkTF

Thank you for your careful review and constructive feedback. We address your concerns below.

W1 & Q1: Contribution of VLM Processing

To disentangle the contributions of VLM processing and the fusion mechanism, we conduct three separately trained ablations on RoboTwin C2R, under the same training and evaluation protocol:

  • Direct-OAT: Directly injects OAT latents into the Action Expert using the same layer-wise cross-attention as JADE, bypassing VLM processing of action representations.
  • AdaLN Fusion: Retains VLM processing of noisy-action representations but replaces cross-attention fusion with AdaLN conditioning.
  • Single-layer Fusion: Retains VLM processing but applies cross-attention fusion at only one layer instead of across multiple layers.
MethodClean ↑Random ↑Avg ↑
π₀.₅68.546.057.3
Direct-OAT71.747.159.4
AdaLN Fusion73.551.462.5
Single-layer Fusion74.749.862.3
JADE (Full)79.354.466.9

VLM processing: Compared with Direct-OAT, JADE improves Clean and Random success rates by 7.6 and 7.3 percentage points, respectively, despite using the same OAT representations and fusion mechanism. This demonstrates that VLM processing of noisy-action representations provides substantial benefits beyond OAT representations and the additional fusion pathway alone.

Fusion design: AdaLN and Single-layer Fusion both retain VLM processing but underperform full JADE by 4.4 and 4.6 percentage points on average, respectively, further supporting the effectiveness of JADE's layer-wise cross-attention design.

W2 & Q2: Latency and Speed–Accuracy Trade-off

We agree that gains at comparable latency should be distinguished from those obtained with a larger inference budget. Two-step, Balanced, and ten-step JADE take approximately 78, 132, and 203 ms per policy query, respectively, compared with 74 ms for ten-step native π₀.₅. On RoboDojo, two-step JADE improves success by 2.76 pp at nearly matched latency, while ten-step JADE achieves a 5.76 pp gain with a larger inference budget. These timings measure model-side generation of a 50-action chunk, excluding communication, simulation, and robot execution. We will report these configurations separately in the main tables, alongside baselines timed under a unified protocol, to make the performance–latency trade-off explicit.

Q3: Implementation Details of Queries Variants

The Queries variants replace the eight OAT latents with eight learnable query tokens. Unlike OAT latents, which encode the current noisy action, Queries do not directly depend on the noisy action but retain the same timestep conditioning as JADE.

VLM features are recomputed at every denoising step with the updated timestep conditioning, rather than being fixed or reused across steps.

All Queries variants are independently trained using the same architecture and training protocol as JADE, with matched alignment settings (λ_align = 0 / 0.1). Therefore, the key controlled difference is whether the VLM processes structured representations of the current noisy action or learnable queries. We will clarify these implementation details in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer TkTF

Dear Reviewer TkTF,

I hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Best regards,
The Authors

Thank you for the additional experiments and clarifications. The ablation addressing W1 resolves my main concern, so I am raising my recommendation to Accept.

Dear Reviewer TkTF,

Thank you for taking the time to review our additional experiments and share your update. We’re glad that the new ablation addressed the main concern you raised in W1. We appreciate your thoughtful feedback throughout the review process.

Best regards,
The Authors

Reviewer_4R1NOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper observes that in π0-style VLAs, the VLM serves only as a static condition and never sees the action being generated. To address this, JADE encodes the current noisy action into compact OAT latents, feeds them to the VLM at every denoising step, and injects the resulting features into the action expert through gated cross-attention. An auxiliary loss further trains the VLM to predict the clean action latents. Built on π0.5, the method shows clear gains on RoboTwin 2.0, RoboDojo, and four real-robot tasks, and its 2-step variant already outperforms 10-step π0.5 at comparable latency.

Critical strengths
  1. The question is simple and well motivated. Allowing the VLM to see the action it helps generate is a natural extension of the "VLM + action expert" paradigm, and the paper builds a coherent story around this idea.

  2. The design is practical. Prefix KV caching, only eight register tokens per step, and a zero-initialized gate make the method easy to plug into an existing π0.5 pipeline, while the 2-step variant keeps inference latency close to that of the original model.

  3. The improvements over π0.5 are consistent across two simulation benchmarks and a real robot.

Critical weaknesses

My main concerns are the limited novelty and whether the experiments truly support the central claim.

  1. The positioning relative to prior work is weak. Several VLAs already feed noisy actions into the language model or the main transformer and perform denoising there (e.g., HybridVLA and Dita). These works are neither discussed nor compared, so the novelty claimed in the title and introduction is overstated. What remains is largely a specific engineering design on top of π0.5.

  2. The source of the improvement is not clearly isolated. The learnable-query variant alone already accounts for a considerable part of the RoboTwin gain (57.3 → 61.3), which suggests that the extra capacity plays a notable role. In addition, RF-Off still relies on a VLM co-trained with the alignment loss, and π0.5 is not evaluated on the same subset in Table 3(c). Consequently, the central claim that the gain comes from the VLM dynamically reading the noisy action is not convincingly supported.

  3. The few-step and adaptive-sampling claims require fairer baselines. π0.5 was never trained for few-step sampling, and no distilled or few-step-oriented baseline is provided. Moreover, the adaptive "Balanced" schedule (about 5 NFE) performs even worse than fixed 2-step sampling on RoboTwin, and a fixed 5-step baseline is missing.

  4. The OOD and reasoning claims are not adequately tested. Visual randomization is the only OOD setting, performance on the language-heavy Open category remains close to zero, and no results over multiple seeds or confidence intervals are reported. With only 30 trials per task, the real-robot evaluation is also too small to be conclusive.

Questions for authors

See Weakness.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 4R1N (1/3)

Thank you for your careful review and constructive feedback. We address your concerns below.

W1 — Novelty and positioning relative to prior work

We thank the reviewer for highlighting HybridVLA and Dita. We acknowledge that these relevant works were omitted from our citations and discussion.

JADE enables the VLM to dynamically perceive the evolving noisy action state and guide the action expert throughout denoising. At each denoising step, the VLM processes a structured representation of the same noisy action currently processed by the action expert, and provides action-conditioned guidance through gated cross-attention. This allows the VLM to continually adapt its guidance to the current action state, while the action expert handles precise continuous action generation. In comparison, HybridVLA jointly performs diffusion noise prediction and autoregressive action prediction within a shared LLM, whereas Dita directly denoises actions using a multimodal Transformer.

Our contribution centers on this interaction between the VLM and the action expert throughout denoising. We will add the missing citations and discussion of HybridVLA and Dita to Related Work, and revise the Introduction to clarify this distinction and the scope of our contribution.

W2 — Source of performance gains

The learnable-query baseline. We acknowledge that learnable queries account for part of the improvement, but they do not fully explain JADE's gains. As shown in Table 3(a), action latents consistently outperform learnable queries on RoboTwin, improving average success by 4.7 pp without alignment (61.3% → 66.0%) and 7.9 pp with alignment (59.0% → 66.9%). The advantage is particularly pronounced on the more challenging RoboDojo benchmark: even without alignment, action latents improve success from 7.68% to 11.15% (+45.2% relative) and Score from 11.33 to 16.59 (+46.4% relative). These consistent gains across both benchmarks, especially on challenging tasks, suggest that structured, action-dependent representations provide substantial benefits beyond learnable queries and the additional fusion pathway.

The contribution of dynamic action conditioning. The original RF-Off, Static, Fixed, and Lagged variants are inference-time interventions on the same JADE checkpoint. RF-Off therefore retains the effects of joint training and only tests the fusion pathway at inference. To provide more direct evidence, we independently train Fixed (initial noise held fixed, timestep updated) and Lagged (features from the preceding denoising step), and evaluate each with its training-time conditioning mechanism. On the same RoboDojo 10-task subset as Table 3(c), JADE outperforms these controls by 4.5 Score / 3.8 pp SR and 3.1 Score / 1.1 pp SR, respectively. Thus, the benefit of updating VLM guidance persists with consistent training and inference mechanisms. We further include an independently trained OAT-to-expert control that directly injects OAT action latents through the same gated cross-attention fusion pathway, bypassing VLM processing during both training and inference. This control achieves 15.9 Score / 8.3% SR, compared with 23.2 / 15.0% for JADE, providing further evidence that VLM processing contributes useful guidance beyond OAT representations and the fusion pathway alone. We have also added native π0.5 under the same task subset and evaluation protocol. We will include these additional results in the revised manuscript and clarify the distinction between inference-time interventions and independently trained controls.

MethodTrainingInferenceScoreSR (%)
Native π0.5NativeNative15.48.1
OAT → Expert (VLM bypass)OAT → ExpertVLM bypass15.98.3
RF-OffDynamicRF-Off18.111.1
StaticDynamicStatic19.311.9
FixedDynamicFixed19.712.1
LaggedDynamicLagged20.814.3
Fixed (separately trained)FixedFixed18.711.2
Lagged (separately trained)LaggedLagged20.113.9
JADE (full method)DynamicDynamic23.215.0

Static reuses the initial VLM features; Dynamic updates them from the current noisy action at each step. SR denotes full-task success rate.

AuthorsOpenReview ↗

Response to Reviewer 4R1N (2/3)

W3 — Few-step baselines and adaptive sampling

We have extended our sampling experiments with π0.5+MIP [1], a few-step-oriented baseline using supervised iterative regression, and additional fixed 5-step evaluations of native π0.5 and JADE. We also evaluate JADE under fixed 2-step, fixed 5-step, and Balanced sampling on RoboTwin and RoboDojo, enabling comparisons across sampling budgets and at approximately matched NFE.

Few-step effectiveness. At the same 2 NFE, JADE achieves 66.0% average success on RoboTwin, substantially outperforming π0.5+MIP (58.1%, +7.9 pp). This demonstrates that JADE's few-step advantage persists even against a baseline explicitly optimized for few-step generation, rather than merely benefiting from an unoptimized native π0.5 baseline.

Adaptive sampling effectiveness. At approximately matched NFE (~5), Balanced outperforms fixed 5-step sampling on both benchmarks: +1.3 pp on RoboTwin (63.9% → 65.2%), and +1.66 pp SR / +2.72 Score on RoboDojo (10.82% → 12.48% SR; 15.34 → 18.06 Score). This indicates that adaptive step allocation provides benefits beyond simply increasing the number of denoising evaluations.

The benefit of additional refinement also depends on task demands. On RoboTwin, fixed 2-step sampling already approaches 10-step performance (66.0% vs. 66.9%), suggesting that its relatively simpler tasks may require little additional denoising. Consequently, fixed 2-step slightly outperforms Balanced (66.0% vs. 65.2%) while using fewer evaluations. In contrast, RoboDojo exhibits a larger gap between 2-step and 10-step sampling (9.67% vs. 12.67% SR), consistent with a greater need for iterative refinement. Here, Balanced achieves 12.48% SR with only 4.985 NFE, retaining 98.5% of the 10-step performance.

Together, these results demonstrate JADE's strong few-step generation capability and the value of adaptive sampling at a comparable computational budget, particularly on more challenging tasks where additional action refinement is beneficial.

RoboTwin success rates (%)

MethodSampling configurationNFECleanRandomAvg.
Native π0.5Native 10-step1068.546.057.3
π0.5 + MIP [1]Two-step269.946.358.1
π0.5 + MIP [1]Five-step569.246.858.0
Native π0.5Fixed 5-step567.745.956.8
JADEFixed 2-step280.151.866.0
JADEFixed 5-step576.651.263.9
JADEBalanced4.9977.952.465.2

RoboDojo results

MethodSampling configurationNFEScoreSR (%)
JADEFixed 2-step214.529.67
JADEFixed 5-step515.3410.82
JADEBalanced4.98518.0612.48
JADEFixed 10-step1018.2912.67

Reference

[1] Chaoyi Pan et al. Much Ado About Noising: Dispelling the Myths of Generative Robotic Control. ICLR 2026.

AuthorsOpenReview ↗

Response to Reviewer 4R1N (3/3)

W4 — OOD generalization and evaluation reliability

(1) OOD evaluation. Our evaluation covers multiple distribution shifts beyond visual appearance randomization. RoboTwin 2.0 evaluates generalization from Clean training scenes to Random test environments, including variations in appearance, scene clutter, and tabletop height, testing visual and spatial robustness. RoboDojo provides two complementary settings: Generalization (12 tasks) evaluates variations in object instances and spatial configurations, while Open (8 tasks) evaluates unseen task specifications involving language/image-conditioned manipulation and skill transfer. We further extend our real-robot evaluation with additional OOD trials, as detailed below. Thus, our evaluation covers visual, spatial, object-level, and task-level distribution shifts.

(2) Open-category performance and reasoning. RoboDojo's Open category primarily evaluates open-vocabulary instruction grounding and the transfer of learned manipulation skills to unseen task specifications, rather than language reasoning in isolation. These tasks lack corresponding task-specific demonstrations during training and remain challenging for existing VLAs. Native π0.5 achieves only 1.67% SR / 1.98 Score, reflecting the difficulty of this setting. Nevertheless, JADE improves performance to 2.42% SR / 3.05 Score, corresponding to 44.9% and 54.0% relative improvements, respectively. These results support improved semantic grounding and task-level generalization over the base policy, while the low absolute performance indicates that general-purpose reasoning remains an open challenge.

(3) Multiple seeds and real-world reliability. Our RoboDojo evaluation follows the official protocol across all 42 tasks and three evaluation seeds (0, 1, 2), totaling 6,300 trials. The per-seed success rates are 12.38%, 12.82%, and 12.82%, with corresponding scores of 18.07, 18.26, and 18.55, demonstrating consistent results across the evaluated seeds.

For real-world evaluation, JADE outperforms π0.5 on all four manipulation tasks, improving average success from 49.2% to 70.0% (+20.8 pp). We additionally evaluated Tool Placement and Object Arrangement under separate real-world OOD conditions, with 30 trials per task and policy. The additional real-world OOD conditions introduce distractor objects, change object and container placements, and replace the original containers, while keeping the task objectives unchanged. On Tool Placement, JADE improves success from 10.0% to 16.7% (+6.7 pp); on Object Arrangement, success improves from 26.7% to 36.7% (+10.0 pp). Across these 60 OOD trials per policy, the combined success rate increases from 18.3% to 26.7% (+8.3 pp). Although the limited trial counts prevent strong statistical conclusions, these additional results provide preliminary evidence that JADE's improvements extend beyond the original real-world test conditions.

We will include these additional evaluations and the corresponding results in the revised manuscript and supplementary material.

Reviewer_PQTnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes JADE, a joint action denoiser that enables a pretrained VLM to participate dynamically in action generation. JADE feeds representations of noisy actions from a frozen OAT encoder into the VLM, guides an action expert through gated cross-attention, and introduces auxiliary supervision to align VLM predictions with clean action latents. An adaptive step-size schedule further reduces inference cost. Experiments demonstrate improvements over pi0.5 in simulation and real-world tasks, with ablations examining action conditioning, auxiliary supervision, and inference-time feature updates.

Critical strengths
  1. The idea of using a pretrained VLM to provide action-dependent guidance throughout denoising is novel and insightful. It provides a valuable perspective on making greater use of pretrained VLM capabilities.
  2. The paper conducts extensive experiments, demonstrating consistent overall improvements over pi0.5 and supporting the practical value of the approach.
Critical weaknesses
  1. The evaluation protocols and their relationship across experiments are insufficiently clear. Although the Figure 4 caption distinguishes subset evaluations from full-benchmark results, Table 3(c), the left panel of Figure 4, and the right table of Figure 4 use different task sets and per-task trial counts without adequately explaining the rationale for these choices or how the evaluation runs relate. This makes the representativeness of the subset results difficult to assess and makes Figure 4 confusing. Moreover, JADE (Balanced) markers appear near NFE=4 in the left panel of Figure 4, while the right table reports approximately five evaluations. This makes Figure 4 even more confusing.
  2. The evidence for the second contribution that joint denoising improves few-step sampling efficiency remains limited. Low-NFE JADE outperforms ten-step pi0.5, but this does not clearly distinguish improved overall policy quality from improved robustness to reducing the sampling budget. On RoboDojo, reducing JADE from ten to two steps substantially lowers its success rate. Moreover, JADE (Balanced) lacks a fixed five-step comparison, leaving the contribution of adaptive scheduling unclear.
Questions for authors
  1. What motivated the different task subsets and per-task trial counts used in Table 3(c) and Figure 4, and how representative are the subsets of the full benchmarks? Please also report the actual mean NFE of JADE (Balanced) on each plotted subset to clarify the marker positions.
  2. Beyond achieving higher absolute success rates at low NFE, does JADE retain a larger fraction of its ten-step performance as the sampling budget decreases than native π0.5?
  3. Can you compare Balanced with fixed five-step sampling under matched evaluation conditions to clarify the benefit of adaptive scheduling?
  4. Given the emphasis on OOD generalization in the abstract and introduction, can you clarify JADE’s main advantages and failure modes under distribution shifts?
5 comments
AuthorsOpenReview ↗

Response to Reviewer PQTn

Thank you for your careful review and constructive feedback. We address your concerns below.

W1 & Q1 — Evaluation protocols and subsets

The different task subsets serve distinct experimental objectives. For Table 3(c), we select two tasks from each of RoboDojo's five capability categories to ensure balanced coverage when examining step-wise VLM updates, using 100 trials per task. For Figure 4, we select 10 tasks per benchmark based on historical baseline performance (3 easy, 4 medium, and 3 hard-but-solvable tasks), using 50 trials per task and condition to evaluate multiple NFE settings. All methods within each experiment share the same tasks and evaluation scenarios.

These subsets are intended for controlled analysis rather than direct estimates of full-benchmark performance. Our main sampling-efficiency conclusions are independently supported by full-benchmark evaluations: 2-step JADE achieves 9.67% SR on RoboDojo, outperforming native π₀.₅ at both 2 steps (3.53%) and 10 steps (6.91%). Similarly, on RoboTwin, 2-step JADE achieves 66.0% average SR versus 57.3% for 10-step native π₀.₅.

Regarding the Balanced markers in Figure 4, their positions reflect the actual mean NFE on the evaluated subsets: 4.333 (RoboTwin Clean), 4.452 (RoboTwin Random), and 4.327 (RoboDojo). In contrast, the right-hand table reports full-benchmark mean NFE of 4.995 on RoboTwin and 4.985 on RoboDojo. Since Balanced adaptively determines its step count, the average NFE varies with the evaluated task distribution. We will explicitly clarify this distinction and the evaluation protocols in the revision.

W2 & Q2 — Few-step performance retention

We additionally evaluated native π₀.₅ at 2 and 5 NFE and JADE at fixed 5 NFE on the full RoboDojo benchmark (42 tasks, 3 seeds, 6,300 trials), following the same evaluation protocol.

ModelNFEScore ↑SR (%) ↑SR Retention ↑
π₀.₅27.643.5351.1%
π₀.₅58.025.3777.7%
π₀.₅1011.416.91100%
JADE214.529.6776.3%
JADE515.3410.8285.4%
JADE (Balanced)≈518.0612.4898.5%
JADE1018.2912.67100%

SR Retention = SR(NFE) / SR(10) for each model.

The results show that JADE achieves not only higher absolute performance at low NFE but also better relative performance retention. When reducing NFE from 10 to 2, JADE retains 76.3% of its original success rate, compared with only 51.1% for native π₀.₅ (+25.2 percentage points). At 5 NFE, JADE also retains 85.4%, compared with 77.7% for π₀.₅. Moreover, 2-step JADE already outperforms 10-step π₀.₅ (9.67% vs. 6.91% SR). These results demonstrate that JADE's few-step advantage is not merely due to stronger overall policy performance, but also reflects improved robustness to reduced sampling budgets.

Q3 — Benefit of adaptive sampling

As shown in the table in W2 & Q2, under nearly identical sampling budgets (4.985 vs. 5 NFE), JADE (Balanced) improves the Score from 15.34 to 18.06 (+2.72) and SR from 10.82% to 12.48% (+1.66 pp) compared with fixed 5-step sampling. Moreover, Balanced retains 98.5% of the 10-step success rate, compared with 85.4% for fixed 5-step sampling. These results demonstrate that adaptive scheduling provides additional performance gains under matched sampling budgets, rather than merely benefiting from more denoising steps.

Q4 — OOD advantages and failure modes

JADE’s main advantage is higher task success under visual and scene changes. When trained on Clean demonstrations and evaluated under Random conditions, it improves RoboTwin success from 46.0% to 54.4%. Additional real-robot OOD tests show improvements from 10.0% to 16.7% on Tool Placement and 26.7% to 36.7% on Object Arrangement. The additional real-world OOD conditions introduce distractor objects, change object and container placements, and replace the original containers, while keeping the task objectives unchanged. On RoboDojo, the largest capability gains occur in Precision (+10.83 pp) and Long-Horizon (+9.33 pp), although these gains are not separate evidence of OOD robustness.

The remaining limitations are scene-sensitive coordination, language-conditioned task composition, and reliable multi-stage completion. Specifically, JADE’s Handover Mic success drops from 99% to 15% under Clean-to-Random evaluation. RoboDojo Open success remains only 2.42%, with 0% on Stack Blocks by Language. Organize Table achieves a Score of 45.67 but only 2.67% full-task success, indicating partial progress without reliable completion. Thus, our results support improved execution under the evaluated shifts, rather than general OOD robustness. We will make this scope and these limitations explicit in the revised paper.

AuthorsOpenReview ↗

Response to Reviewer PQTn

Dear Reviewer PQTn,

I hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Best regards,
The Authors

Thank you for the detailed rebuttal and additional experiments. The clarified evaluation protocols and new full-benchmark comparisons address most of my concerns, particularly regarding few-step performance retention and adaptive scheduling. I also appreciate your discussion of OOD generalization and failure cases. While some limitations remain, the additional evidence strengthens the paper's claims. I encourage the authors to incorporate these clarifications and results into the revised manuscript. I will maintain my original rating (Weak Accept).
As a minor question for future work, could the Balanced adaptive sampling strategy also be applied to native π0.5\pi_{0.5} or other flow-based action policies, or are its benefits specific to JADE's joint denoising mechanism?

Reviewer_zvydOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  1. Building on RepFusion's ( Pan et al. 2026) approach, the authors propose JADE, which uses OAT-encoded noisy actions to guide a π0.5 action expert through the VLM.
  2. The authors report improvements over π0.5 on RoboTwin, RoboDojo, and real-world manipulation tasks, including with fewer denoising steps.

Reference: Pan et al. (2026). RepFusion: Leveraging Multimodal Priors for Denoising in Representation Space. arXiv:2606.14700.

Critical strengths
  1. Consistent improvements across simulation and real-world benchmarks. JADE improves over π0.5 on RoboTwin 2.0 ( ~9 pp), RoboDojo ( ~6 pp), and four real-world manipulation tasks ( ~20 pp), demonstrating practical effectiveness across different settings.

  2. Strong few-step action generation. JADE achieves better success rates than 10-step π0.5 using only two denoising steps, with nearly identical inference latency (78 ms vs. 74 ms). This is a meaningful practical improvement in action generation efficiency.

Critical weaknesses
  1. Unmatched training and inference compute. JADE requires separate OAT pretraining, 23% slower training updates, and 2.74× higher latency for its main 10-step results. Could π0.5 achieve similar gains with equivalent training compute or a larger action expert under matched inference compute?

  2. Limited architectural novelty. RepFusion already establishes the central idea of repeatedly conditioning a pretrained multimodal model on evolving noisy latents during denoising. JADE largely transfers this mechanism to robotic actions using OAT representations and gated cross-attention, with limited evidence of a fundamentally new contribution.

Questions for authors
  1. Could the authors evaluate stronger π0.5 baselines by increasing fine-tuning steps, scaling the action expert, or using more denoising steps under matched training and inference compute budgets?

  2. Could the authors compare JADE against simpler forms of iterative VLM-guided denoising, such as single-layer fusion, AdaLN conditioning, or partial VLM execution? Are the additional architectural components necessary for the reported improvements?

3 comments
AuthorsOpenReview ↗

Response to Reviewer zvyd

Thank you for your careful review and constructive feedback. We address your concerns below.

W1 & Q1 — Training and inference compute

Replacing π₀.₅’s action expert with a newly initialized, larger module would disrupt its pretrained policy and introduce initialization differences. We therefore retain the original pretrained model and strengthen it through additional training and sampling compute for a more controlled comparison.

We additionally evaluated two stronger native π₀.₅ baselines on the full RoboTwin 2.0 benchmark (50 tasks, Clean/Random):

  1. Longer training: extending training from 20k to 25k steps (1.25× training time), comparable to JADE's 1.23× training-update cost;

  2. More denoising steps: increasing NFE from 10 to 20, achieving inference latency comparable to 10-step JADE.

MethodNFEClean SRRandom SRAvg. SR
Native π₀.₅1068.546.057.3
π₀.₅ (25k training steps)1067.344.255.8
Native π₀.₅2069.247.158.2
JADE (ours)1079.354.466.9

These results show that additional computation alone does not reproduce JADE's gains. Extending π₀.₅ training to 25k steps does not improve performance (57.3% → 55.8%), while doubling its denoising steps yields only a marginal gain (57.3% → 58.2%). Notably, JADE still outperforms the 20-step baseline by 8.7 percentage points at comparable inference latency.

Furthermore, 2-step JADE achieves 66.0% average success, outperforming 10-step native π₀.₅ (57.3%) at comparable inference latency. We will include these additional baseline experiments and their training and evaluation settings in the revised manuscript, together with a detailed training-cost breakdown that accounts for OAT pretraining, to make the computational comparison more transparent.

W2 & Q2 — Architectural novelty and simpler alternatives

JADE extends iterative multimodal guidance to action-aware joint denoising, where the VLM processes structured noisy-action representations and is jointly optimized with the action expert through flow matching and clean-action latent alignment. Unlike RepFusion's frozen MLLM and AdaLN conditioning, JADE learns action-aware representations and guides the action expert through layer-wise gated cross-attention.

To examine whether simpler architectures can achieve comparable performance, we additionally trained two variants on RoboTwin 2.0: (1) AdaLN conditioning, replacing gated cross-attention with feature modulation; and (2) single-layer fusion, retaining only one fusion layer. Both preserve the same OAT representations, training objectives, and evaluation protocol.

MethodClean SRRandom SRAvg. SR
Native π₀.₅68.546.057.3
AdaLN Conditioning73.551.462.5
Single-layer Fusion74.749.862.3
JADE (Full)79.354.466.9

Both simplified variants improve over native π₀.₅, confirming the benefit of action-conditioned VLM guidance. However, full JADE consistently outperforms both variants across Clean and Random settings, with 4.4 and 4.6 percentage-point gains in average success rate over AdaLN and single-layer fusion, respectively. This indicates that simpler conditioning captures part of the benefit, while layer-wise gated cross-attention enables more effective integration of action-aware VLM features throughout denoising, supporting the effectiveness of JADE's architectural design. These results further suggest that the information extracted by the VLM from noisy actions provides useful guidance for the action expert, helping it generate actions more effectively.

AuthorsOpenReview ↗

Response to Reviewer zvyd

Dear Reviewer zvyd,

I hope this message finds you well. As the discussion period is nearing its end, we wanted to ensure we have addressed all your concerns satisfactorily. If there are any additional points or feedback you would like us to consider, please let us know. Your insights are invaluable to us, and we’re eager to address any remaining issues to improve our work.

Thank you for your time and effort in reviewing our paper.

Best regards,
The Authors

Dear Reviewer zvyd,

I hope this message finds you well. We wanted to gently follow up and express our hope that our previous responses have helped clarify your concerns. If there are any additional questions or suggestions you would like us to consider, please feel free to let us know. We greatly appreciate your feedback and would be happy to provide any further clarification.

Thank you again for your time and effort in reviewing our paper.

Best regards,
The Authors

Reviewer_6f95OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces JADE, which lets the VLM process current action latents at each denoising step and guide the action expert through gated cross-attention, with clean-latent supervision. Evaluation covers two simulation benchmarks and four real-world tasks.

Critical strengths
  1. Well-motivated dynamic guidance. The VLM updates its guidance using the current noisy action throughout denoising.

  2. Consistent empirical gains. Average success rises from 57.3% to 66.9% on RoboTwin and from 49.2% to 70.0% across four real-world tasks (Table 1, §4.6).

  3. Strong ablation evidence. Independently trained query-token, VLM-bypass, and fixed-conditioning controls support the contributions of action conditioning and VLM processing (Tables 3, 7–8).

Critical weaknesses
  1. Higher performance comes with substantial inference cost. Ten-step latency increases from approximately 74 to 203 ms; two steps reduce it to 78 ms, but RoboDojo success falls from 12.67% to 9.67% (Table 2).
  2. Open-task capability remains limited. RoboDojo Open success is only 2.42%, with zero success on six of eight tasks, indicating limited transfer to unseen tasks (Table 5).
Questions for authors
  1. What are the real-world success rates and latencies of the two-step and Balanced configurations? Do they preserve most of the ten-step gains?

  2. Do Open-category failures mainly arise from instruction understanding, action execution, or other factors? Is this supported by failure analysis?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6f95 (1/2)

Thank you for your constructive feedback. We received your review at 03:16 AoE on November 17, with less than two days left in the discussion period. We address your main points below and will provide a fuller discussion in a future revision.

W1 — Inference cost and the performance–latency trade-off

Ten-step JADE takes about 203 ms per 50-action chunk on the model side, compared with 74 ms for ten-step native π₀.₅. Each JADE step updates the VLM’s guidance using the current noisy action. Reducing JADE to two steps lowers RoboDojo success from 12.67% to 9.67%. At 78 ms, however, those two steps still outperform ten-step native π₀.₅ by 2.76 percentage points (9.67% vs. 6.91%). When both models are reduced to two steps, JADE retains 76.3% of its own ten-step success rate, compared with 51.1% for native π₀.₅.

Near-ten-step success on RoboDojo does not require the full 203 ms. Using the same checkpoint and about five model evaluations per chunk on average, Balanced recovers 2.81 of the 3.00 percentage points lost when going from ten steps to two (Table 2). It reaches 12.48% success versus 12.67% for ten-step JADE, at about 132 ms rather than 203 ms. Thus, two steps are sufficient to outperform native π₀.₅ near its latency, while Balanced preserves nearly all of JADE’s ten-step success rate at about 35% lower model-side latency.

Q1 — Real-world performance and latency of two-step and Balanced configurations

On the two real-robot tasks tested, Balanced preserved most of ten-step JADE’s gain over ten-step native π₀.₅ (83.3%), while two-step JADE preserved just over half (58.3%). With 30 trials per task and configuration, Balanced succeeded in 45 of 60 trials (75.0%) and two-step JADE in 42 of 60 (70.0%).

ConfigurationTool PlacementObject ArrangementOverall successModel-side latency per chunk
Native π₀.₅, ten steps17/3018/3058.3%~74 ms
Native π₀.₅, two steps15/3014/3048.3%~48 ms
JADE, two steps24/3018/3070.0%~78 ms
JADE, Balanced25/3020/3075.0%~132 ms
JADE, ten steps26/3021/3078.3%~203 ms

At the same two-step budget, JADE outperformed native π₀.₅ on both tasks: 42 vs. 29 successes in 60 trials overall, a 21.7 percentage-point advantage. Reducing each policy from ten steps to two preserved 89.4% of JADE’s own ten-step success rate (42/47), compared with 82.9% for native π₀.₅ (29/35). On the real-robot tasks we tested, reducing both policies from ten steps to two preserved a larger fraction of the original success rate for JADE than for native π₀.₅.

AuthorsOpenReview ↗

Response to Reviewer 6f95 (2/2)

W2 & Q2 — Open-task generalization and failure analysis

W2. Limited transfer to unseen Open tasks. We agree that transfer remains limited: JADE achieved no full-task successes on six of the eight Open tasks (Table 5). Across all eight, JADE improved the mean success rate from 1.67% to 2.42% and the mean score from 1.98 to 3.05 over native π₀.₅. RoboDojo provides no task-specific training demonstrations for these eight tasks. The underlying skills appear in other training tasks, but the policy must apply them to new instructions and goals.

Q2. Sources of failure and task-level analysis.

We analyzed 80 failure videos, sampling 10 failed trials from each of the eight Open tasks. Among them, 43 cases (53.75%) exhibit failures in language understanding, visual grounding, or reasoning, while 21 cases (26.25%) involve failures to accurately translate intended manipulations into physical actions. The remaining 16 cases (20.00%) fall outside these two categories. These results indicate that high-level understanding and grounding limitations are the most frequent failure modes in our sample, potentially related to the degradation or insufficient preservation of language reasoning, visual grounding, and compositional generalization capabilities during large-scale action pretraining. Regarding execution failures, since RoboDojo Open provides no task-specific training data, policies must rely on previously learned manipulation skills and their generalization to unseen scenarios. Insufficient coverage of similar manipulation distributions can therefore lead to inaccurate execution even when the task intention is correctly understood. We further examine two representative tasks to illustrate JADE's improvements.

General Pickup — Visual grounding and action execution. This task evaluates whether the policy can identify a language-specified target among distractors and successfully grasp and lift it. JADE achieves 18.67% SR, compared with 12.00% for π₀.₅. Since both methods share the same pretrained VLM backbone, this improvement is consistent with JADE more effectively leveraging existing semantic understanding and visual grounding knowledge. Through dynamic VLM guidance at each denoising step, JADE enables task-relevant visual-language information to participate more effectively in action generation, resulting in more successful language-conditioned manipulation.

Classify Objects by Language — Skill composition and multi-step execution. This task requires understanding the instruction-specified mapping between object categories and target baskets and consistently following this mapping across successive manipulation steps. JADE improves the mean score from 0.60 to 2.87, with a full-task success rate of 0.67%. Notably, incomplete trials alone contribute approximately 2.20 points to JADE's mean score, exceeding π₀.₅'s entire mean score of 0.60. Since partial credit requires correctly sorting an entire object category into its designated basket, this improvement demonstrates JADE's enhanced ability to complete instruction-conditioned intermediate subgoals, although reliably completing the entire manipulation sequence remains challenging.

Overall, our failure analysis identifies high-level understanding and visual grounding limitations as the most frequent observed bottlenecks, followed by difficulties in translating task intentions into accurate physical actions. JADE more effectively utilizes existing VLM knowledge to guide action generation, improving language-conditioned manipulation and the completion of intermediate compositional subgoals, while broader reasoning and long-horizon generalization remain challenging.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Revision Summary

We sincerely thank all reviewers for their time and constructive feedback, which helped us clarify our contribution and strengthen the experimental comparisons. We also appreciate the positive recognition of JADE’s practical design and few-step performance. We have incorporated additional experiments and clarifications into the revised manuscript, with changes highlighted in blue. The main updates are:

  • VLM processing and action conditioning — 4R1N, TkTF, zvyd. We added independently trained OAT-to-expert, Fixed/Lagged, AdaLN, and single-layer fusion controls, and clarified the distinction between independently trained controls and inference-time interventions. See Sections 4.3–4.5 and Appendix H.1.
  • Sampling efficiency and computational cost — PQTn, TkTF, 4R1N, zvyd. We added few-step baseline and fixed-five-step comparisons, performance-retention results, and native-model evaluations with additional training or denoising steps. The main tables now distinguish JADE’s sampling configurations and report available model-side latencies. See Tables 1–2, Section 4.4, and Appendices H.2–H.3.
  • Evaluation protocols and limitations — PQTn, 4R1N. We clarified task subsets, trial counts, and mean NFE; reported results across three evaluation seeds and additional real-world OOD trials; and expanded the discussion of remaining failure modes. See Appendices E–F and H.4.
  • Related work and positioning — 4R1N. We added discussion of HybridVLA and Dita and clarified JADE’s interaction between the VLM and a separate action expert. See Section 2.

As the discussion period is nearing its end, we would greatly appreciate it if you could take another look at our responses and the revised manuscript when time permits. In particular, it would be helpful to know whether the additional experiments and clarifications address your main concerns, and whether any issues remain unresolved. We would be happy to provide further explanations or point to the relevant results during the remaining discussion period. Thank you again for your time and thoughtful consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.