acceptodds
Under review as a conference paper at ICLR 2027

Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models

Abstract

Action decoders in vision-language-action (VLA) models are commonly trained by behavior cloning to reproduce demonstrated action sequences. To enrich this supervision, prior work adds training signals based on future observations and motion. These signals describe executions and outcomes, but do not explicitly identify the local objective an action segment serves under the instruction. We propose Intention Distillation (INDI), which makes this behavior-level intent a supervised intermediate state that action generation is trained to use. A frozen teacher vision-language model (VLM) provides multimodal targets from executed demonstration segments under the task instruction. The student learns to predict these targets at an intermediate decoder layer using only its current observations, instruction, and robot state. Context routing and an intent-mismatch objective encourage downstream predictions to depend on the recovered intent, while visual-outcome and textual-purpose targets provide complementary supervision. The teacher and target-generation modules are removed at deployment. INDI improves GR00T-N1.7 from 64.3% to 84.7% on SimplerEnv-Bridge and from 64.1% to 70.3% on RoboCasa Kitchen, with improvements on across both benchmarks. Across standard scenes, held-out objects, and distractor conditions, average real-world success increases from 62.0% to 68.7%. Representation probes and interventions further show that the recovered states encode task and progress information and influence downstream decoding.

Primary area:
applications to robotics, autonomy, planning
Keywords:
VLAs, Vision-Language-Action Models, Robotics, World Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jR8mOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses a limitation of behavior cloning: an action decoder may not learn or use the local purpose of a demonstrated behavior. INDI uses a frozen vision-language teacher to extract features from executed action segments and their outcomes. The student predicts these features from current inputs. Context routing and mismatch training encourage the action decoder to use the resulting intermediate state. The main strength is the combination of execution-based supervision and explicit training for downstream use. The teacher is removed at deployment. However, training stability and one ablation's mismatch-update schedule remain unclear. The performance is good enough, for GR00T-N1.7, success rises from 64.3% to 84.7% on Bridge and from 64.1% to 70.3% on RoboCasa. Results on π0.5 and real robots provide additional support. Overall, the evidence supports practical gains, while the contribution of each training mechanism needs firmer validation.

Critical strengths
  1. Useful gains across settings. Matched backbone comparisons cover two policies, simulation, and real robots. A future-supervision control helps distinguish INDI from outcome prediction alone. The performance is good.

  2. Relevant mechanism controls. Separate routing, mismatch, and alignment ablations directly test whether the decoder benefits from learning and using the intermediate state.

  3. Informative target analysis. Prompt and feature-source comparisons support the target design. The paper also acknowledges that these features contain scene and motion information, beyond intent alone.

Critical weaknesses
  1. Training stability is unclear. Reported evaluation repetitions do not measure independent training variation. The full model and no-mismatch variant need focused training repeats to strengthen the mechanism claim.

  2. The no-bottleneck schedule is ambiguous. Direct context remains open in this variant, but the mismatch-update frequency is unspecified. This makes it harder to isolate the effect of routing.

  3. Total training cost is incomplete. Policy costs are reported, but one-time teacher-target generation time and cache storage are missing. These costs matter when applying INDI to a new dataset.

Questions for authors

It is a good paper, but I still have three questions:

  1. How stable is the full model's gain over the no-mismatch variant across independent training seeds?

  2. Is the mismatch-update frequency matched between the full model and the no-bottleneck variant?

3.What are the teacher-target generation time and cache storage requirements for each dataset?

If the authors can address my concerns well, I will raise my score.

2 comments

We thank Reviewer jR8m for the focused review, which pinpoints the evidence needed to validate each training mechanism. We also appreciate the recognition of INDI’s consistent gains across backbones and real robots, the targeted mechanism controls, and the informative target analysis. We address the three concerns directly with training-seed repeats (W1), a frequency-matched rerun of the no-bottleneck variant (W2), and per-dataset target-generation time and storage (W3). We uploaded a revised version of the paper, with all changes highlighted in blue.

[W1, Q1] Training Stability

The full model’s gain over the no-mismatch variant is stable across independent training seeds. We trained the full model, the no-mismatch variant, and the baseline with two additional seeds each and evaluated all runs with a fixed evaluation seed (GR00T-N1.7, Bridge).

Model (3 training seeds)SpoonCarrotStackEP-BasketAvg.
GR00T-N1.782.7 ± 3.173.3 ± 5.061.3 ± 5.035.3 ± 4.663.2 ± 2.5
+ INDI w/o mismatch91.3 ± 3.180.0 ± 4.037.3 ± 7.088.7 ± 9.274.3 ± 3.0
+ INDI90.0 ± 2.080.0 ± 2.067.3 ± 7.095.3 ± 4.283.2 ± 2.8

The full model outperforms the no-mismatch variant for every seed (85.5%, 80.0%, and 84.0% vs. 74.0%, 71.5%, and 77.5%), and its lowest-scoring seed still exceeds the best no-mismatch seed.

We added the seed results to Appendix B.2 and clarified the evaluation protocol in Appendix A.2.

[W2, Q2] Mismatch-Update Frequency

The mismatch-update frequency was not matched in the original no-bottleneck variant, but the routing effect does not depend on it. In the full model, mismatch training is applied only to context-blocked updates (about half of all updates), because a mismatched intent can be ignored when the decoder has direct access to the VLA context. The no-bottleneck variant, which keeps direct access open, applied mismatch training to every update. We stated this schedule in Appendix B.2 and separated the two factors in two ways.

Comparison without mismatch training. Comparing the two variants without mismatch training removes the frequency difference altogether. Routing alone raises success from 65.5% to 74.0% (Table 10).

Frequency-matched rerun. We retrained the no-bottleneck variant with mismatch training applied to a random half of the updates, matching the full model. It reaches 68.0%, close to the original no-bottleneck variant (70.0%), so routing still adds 17.5 pp (85.5% vs. 68.0%).

Variant (Bridge)RoutingMismatch updatesSpoonCarrotStackEP-BasketAvg.
INDIYesContext-blocked (about 50%)90.078.074.0100.085.5
No bottleneck (original)NoAll (100%)88.076.060.056.070.0
No bottleneck (frequency-matched)NoRandom 50%86.080.056.050.068.0
w/o mismatchYesNone92.080.030.094.074.0
w/o bothNoNone86.072.058.046.065.5

[W3, Q3] Total Training Cost

The total training cost stays moderate: target generation is a one-time cost per dataset, and the stored targets are smaller than the decoded training frames. We measured the generation time and storage for each dataset.

BridgeRoboCasaReal robot (SO-101)
Samples1,520,923615,195267,502
Teacher-target generation (GPU-h)26.416.38.9
Visual-target generation (GPU-h)1.31.40.6
INDI target storage (GiB)28419787
Decoded training frames (GiB)347379180

Most of the time goes to generating the purpose statement (47 to 84 ms per sample), while feature extraction takes 15 to 35 ms. Storage is 128 KiB per sample for teacher targets and 64 KiB per endpoint frame and camera for visual targets, so the cost for a new dataset can be estimated directly. End to end, the first run, including target generation, takes about 85 GPU-h on Bridge and 75 GPU-h on RoboCasa Kitchen with GR00T-N1.7, compared with 51 GPU-h for the baseline (17 h on three GPUs, Table 7). The targets are reused by later runs with the same teacher, prompt, and target settings, so each additional run takes 57 GPU-h, 1.12x the baseline, and the teacher is not used at deployment. We added these measurements to Table 7 and Appendix A.7.

AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer jR8m,

Thank you again for your focused review, which pinpointed the evidence needed to validate each training mechanism.

Since the discussion period is about to end, we wanted to follow up on our rebuttal. We retrained the full model, the no-mismatch variant, and the baseline with two additional seeds, stated the mismatch-update schedule and reran the no-bottleneck variant with a matched frequency, and reported the target-generation time and storage for each dataset.

We would appreciate any feedback on whether our response addresses your concerns, and would be happy to clarify any remaining points.

Best, Authors

Reviewer_9TEhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes INDI to improve vision-language-action (VLA) models through behavior-level intent supervision. A frozen teacher VLM processes the current observation, instruction, execution video, and coarse action summary under a functional-purpose prompt; its input-span hidden states provide the intent target. The student recovers this representation from current inputs, while context routing and intent-mismatch training encourage the action decoder to use it. Visual and textual groundings provide auxiliary supervision, and the teacher is no longer used at deployment. The authors validate performance improvements on GR00T-N1.7, \pi_{0.5}, two simulation benchmarks, and real robot tasks, and analyze the role of each component through ablations, representation probes, and state intervention analyses.

Critical strengths

1.Converts local purpose information in demonstrations into training supervision. Compared with FLARE-style future supervision, this paper adds functional-purpose-oriented teacher supervision. The innovation mainly lies in the construction of the supervision target rather than in adding query tokens. The experiment showing that functional-purpose prompts outperform neutral descriptions supports this design. It provides an effective approach for exploiting high-level behavioral information in demonstrations.

2.Training mechanisms encourage the action decoder to use intermediate representations. The paper trains both intent alignment and the downstream use of intent. Context routing restricts direct information access, and mismatch training encourages lower downstream loss with matched intent than with swapped intent. This targets the problem that auxiliary representations may be ignored by the action decoder. Ablation experiments support the role of these mechanisms, and the teacher is used only in the offline stage, which has practical deployment value.

Critical weaknesses

1.Ambiguity in deterministic intent recovery is not explicitly modeled. The teacher sees the executed segment; the student sees only the current observation, instruction, proprioceptive state, and frozen VLA context. When the same current inputs are compatible with different future behaviors or history-dependent local objectives, the teacher target may not be uniquely predictable. INDI does not explicitly model this ambiguity or maintain multiple candidate objectives. The consequences of this ambiguity for partially observable tasks and failure recovery remain untested.

2.The method is sensitive to teacher choice and complicates training. Teacher target caches, multiple auxiliary objectives, context routing, and the mismatch branch add compute and configuration cost. In the teacher-swap experiments, Qwen3.5-9B performs worse than the action-only baseline, showing sensitivity to the teacher representation. Table 7 reports policy-training time, but offline target-generation cost is not quantified. Although the paper largely shares the INDI configuration across benchmarks, the additional tuning burden when transferring to new tasks remains unclear.

Questions for authors
  1. Can the intent and state interventions distinguish the effects of local-purpose information from those of object and scene information or general disruption of execution state? Tables 12 and 16 provide useful controls, but cross-goal donors in Table 16 do not redirect the policy, Table 6b jointly edits intent and groundings, and the phase-matched controls in Appendix B.3 also substantially reduce success. Please keep receiver inputs fixed and use donors matched as closely as feasible on object, scene, and execution stage while varying local purpose, with random controls matched in representation norm and perturbation magnitude. Report both original- and alternative-goal success for these added controls.

  2. Do context routing and intent-mismatch training depend on specific settings? Cyclic-shift donors may share the same purpose, and the context-dropout probability changes mismatch frequency. Please compare cyclic-shift pairing with random pairing and semantically different donor selection, control donor-state norms, and test sensitivity to the context-dropout probability and \lambda_{\mathrm{mis}}.

  3. How much do teacher-target construction choices matter? The approximately 1.6-second window and fixed teacher feature-extraction layer may not suit all tasks. Please compare window lengths and teacher feature-extraction layers and report per-task results for these comparisons.

  4. Do key-component gains hold in other settings and across independent training runs? Main component ablations cover only GR00T-N1.7 on SimplerEnv-Bridge, and the reported evaluation runs do not establish variability across independently trained models. Please clarify whether independent training seeds were used and add key ablations on \pi_{0.5} or RoboCasa Kitchen across independent training seeds. In addition to the policy-training times in Table 7, please report teacher-cache generation and end-to-end training costs.

5 comments

We thank Reviewer 9TEh for the rigorous review and the probing questions on how INDI uses the recovered intent, which have helped us strengthen the paper. We also appreciate the recognition of INDI’s purpose-oriented supervision from demonstrations, its explicit training for downstream use, and its teacher-free deployment. We address each concern and add new experiments, including a stochastic-intent variant (W1), matched random controls with the requested donor experiment (Q1), and sensitivity, training-seed, window, and layer studies (Q2 to Q4). We uploaded a revised version of the paper, with all changes highlighted in blue.

[W1] Ambiguity in Deterministic Intent Recovery

Modeling the ambiguity. In our tests, adding explicit uncertainty to the intent does not improve success. We evaluate a scene with two identical towels, where “put the spoon on the towel” has two valid goals. Deterministic INDI succeeds in 96.0% of episodes (baseline 92.0%), placing the spoon on the two towels in 53% and 43% of episodes. A variant that predicts a unimodal Gaussian intent and samples from it during training and inference reaches 94.0% in this scene and 84.5% on Bridge, compared with 96.0% and 85.5% for deterministic INDI.

ModelBridge Avg.Two towels: SuccessTowel ATowel B
GR00T-N1.761.592.044.048.0
+ INDI (deterministic intent)85.596.053.043.0
+ INDI (stochastic intent)84.594.045.049.0

Bridge results use the fixed evaluation run of Tables 9 to 11, and two-towel results average two evaluation seeds with 50 episodes each. We added this analysis to Appendix B.3.

Partial observability and failure recovery. Our benchmarks do not test these cases for INDI or for the baseline. Both backbones observe only the current frame, so ambiguities that only history can resolve, such as counting repetitions in “lift and place three times”, are not covered, and our datasets contain only successful demonstrations. Appendix D discusses extensions that track candidate objectives over time and incorporate failed and corrected executions.

[W2] Teacher Choice and Training Complexity

INDI’s sensitivity to teacher choice and its added training components remain manageable in practice, for three reasons.

Teacher-target geometry provides a diagnostic of the alignment signal. Among the three tested teachers, angular budget increases in the same order as policy success (Tables 20 and 21). Qwen3.5-9B has an angular budget of 0.0027 and reaches 54.5% success, below the 61.5% action-only baseline. This suggests inspecting target variation before training with a new teacher, but the three-checkpoint comparison does not establish a validated threshold or a general predictor of downstream performance.

TeacherAngular budgetBridge Avg.
None (action-only baseline)-61.5
Qwen3.5-9B0.002754.5
Qwen3-VL-8B-Instruct0.009873.5
Cosmos-Reason2-8B0.032785.5

No per-task tuning. A single configuration, including loss weights, margin, context dropout, warmup, prompt, and teacher, is used for all five settings without per-task retuning: two backbones on two simulation benchmarks and a real robot (Section 4.1). Its hyperparameter defaults were chosen once with a small sweep on Bridge and RoboCasa Kitchen (Appendix A.6). The sensitivity runs in [Q2] show that average success changes by at most 3.5 pp from the default (85.5%) across context-dropout probabilities and mismatch weights.

Most of the extra cost is paid once. Teacher targets are generated once per dataset and reused by every later run with the same teacher, prompt, and target settings, and the teacher is not used at deployment. With GR00T-N1.7 on Bridge, the first run takes about 85 GPU-h including target generation, and each later run takes 57 GPU-h, compared with 51 GPU-h for the baseline (17 h on three GPUs, Table 7). Across backbones, policy training takes 12% to 26% longer than the baseline, and inference takes 1.10x (GR00T-N1.7) and 1.21x (π0.5) the baseline time (Table 7). [Q4] reports the full per-dataset costs.

[Q1] Distinguishing Local Purpose from Scene Information

The interventions distinguish stage content from general disruption, but they do not isolate local purpose from object and scene information. Joint stage edits are far more disruptive than size-matched random edits, and editing only the intent changes which failures predominate. In contrast, the requested same-stage donor does not clearly redirect the policy to the other goal (3 of 50 plate placements vs. 1 of 50 for its random control), consistent with Table 16.

We first clarify what the intent encodes. In INDI, the intent is the local objective that an action segment serves under the instruction, such as securing an object for transport or aligning it for release at the destination (Introduction). It also carries information about the instruction’s target (Table 14). However, the decoder continues to condition on the unchanged instruction and current observation, which may keep a donor intent from redirecting the policy, as with the cross-goal donors in Table 16 (Appendix B.3).

Changing the stage content changes the behavior. With receiver inputs fixed, we replace only the coordinates of the intent and grounding state on the phase plane, a projection fit with elapsed-progress labels (Appendix B.3), and use same-stage edits as the matched control. This plane also retains some task information (task purity 36.6%, chance 25%). In selected rollouts (Figs. 8 to 10), early-stage values lead to approaching without grasping, and late-stage values lead to moving an empty gripper to the goal.

Random changes of the same size disrupt far less. Unlike the Gaussian-noise condition in Table 6b, whose magnitude was unconstrained, we now match a random control to each joint edit in norm and perturbation magnitude (Appendix B.3). Matched random edits retain 42.5% and 39.5% success, whereas the stage edits drop to 7.5% and 5.5%, consistently across all four tasks. In contrast, the same-stage control noted by the reviewer retains 53.0% success, against 52.5% for its random control.

Editing the intent alone. With the grounding state left unchanged, replacing only the stage coordinates of the intent lowers success to 13.0% (early) and 7.5% (late), compared with 7.5% (early) and 5.5% (late) for the joint edits. Matched random edits of the intent alone retain 64.5% and 58.0% success, higher than the intent-only stage edits on all four tasks. The predominant failure follows the inserted stage. Of the 200 episodes in each condition (50 per task), early-stage edits lead to approaching without grasping in 42.0%, against 18.5% for moving an empty gripper to the goal, and late-stage edits reverse this pattern (52.5% against 19.5%). Failure types are labeled automatically from the simulator state (Appendix B.3).

Condition (Bridge, 50 episodes each)SpoonCarrotStackEP-BasketAvg.
No edit90.078.074.0100.085.5
Early-stage edit20.06.04.00.07.5
Matched random42.052.022.054.042.5
Late-stage edit0.018.04.00.05.5
Matched random38.048.026.046.039.5
Same-stage edit50.070.032.060.053.0
Matched random56.062.036.056.052.5
Early-stage edit, intent only46.06.00.00.013.0
Matched random78.068.052.060.064.5
Late-stage edit, intent only0.030.00.00.07.5
Matched random46.078.036.072.058.0

Requested donor experiment. In a new run separate from Table 16, we place a spoon, a towel, and a plate on the table and fix the instruction “put the spoon on the towel”, so the towel is the original goal and the plate is the alternative goal. Mid-execution, we replace the intent (decoder block 16) with that of another episode in this scene at the same stage, whose goal is either the towel or the plate (details in Appendix B.3). Neither donor clearly redirects the policy, and both stay close to their random controls.

Replaced intent (50 episodes each)Towel (original goal)Plate (alternative goal)
None (own intent)82.00.0
Donor episode, goal towel82.00.0
Donor episode, goal plate78.06.0
Random, matched to plate donor82.02.0
Random, matched to towel donor86.00.0

All conditions use 50 episodes in this run, so the unedited 82.0% differs from the 82.4% of the 51-episode pilot in Table 16.

[Q1] Distinguishing Local Purpose from Scene Information (continued)

Purpose beyond progress. In progress-matched pairs, which are not additionally matched on object or scene, recovered intents are more similar for the same objective with dissimilar action sequences than for different objectives with similar ones (gap 0.167, 95% CI [0.089, 0.245], Appendix B.3). Purpose-oriented supervision also yields a better policy. Prompting the teacher for the segment’s purpose instead of a neutral description changes only the intent target, with the video, actions, and visual and textual targets unchanged, and raises success from 76.0% to 85.5% (Table 22). Every functional-prompt seed outperforms every neutral-prompt seed (Appendix B.2).

These results support the claim in Section 4.4 that the decoder uses the task and stage content of the joint intent-grounding state. We added both tables to Appendix B.3.

[Q2] Sensitivity to Routing and Mismatch Settings

Context routing and mismatch training do not depend on specific settings. Average success changes by at most 3.5 pp across the tested hyperparameters and donor choices.

Sensitivity. We vary the context-dropout probability (0.25, 0.75) and λ_mis (0.05, 0.2) around the defaults (0.5, 0.1). Because mismatch is applied only to context-blocked updates, changing the dropout probability also changes the mismatch frequency, so the sweep covers both.

Donor selection and norm. Cyclic-shift pairing is close to random pairing because each batch mixes samples from many episodes: 98.1% of donor pairs have different instructions, and only 1.34% come from the same episode. Donors that happen to share a purpose would only weaken the mismatch signal rather than inflate the reported gains. Selecting only semantically different donors (different tasks) gives 85.0%. Donors are recovered intents from the same decoder block (block 16), so their norms already match the originals (median ratio 1.002) while their directions differ (cosine similarity 0.47).

SettingSpoonCarrotStackEP-BasketAvg.
INDI (dropout 0.5, λ_mis 0.1, cyclic shift)90.078.074.0100.085.5
Dropout 0.2592.082.062.096.083.0
Dropout 0.7586.084.066.092.082.0
λ_mis 0.0590.082.068.098.084.5
λ_mis 0.288.080.072.096.084.0
Different-task donors90.082.070.098.085.0

[Q3] Teacher-Target Construction Choices

Teacher-target construction choices keep most of INDI’s gain over the baseline across window lengths and teacher layers. The window is also not tuned per task: it matches the segment supervised by GR00T-N1.7’s action loss (8 steps on Bridge, 32 on RoboCasa Kitchen, Appendix A.6), and layer 18 is a middle layer of the teacher. We varied the window from the default 1.6 s to 0.8 s and 3.2 s, and the extraction layer from 18 to 12 and 24, keeping everything else fixed. Average success stays within 6.5 pp of the default.

Teacher target (Bridge)SpoonCarrotStackEP-BasketAvg.
1.6 s, layer 18 (default)90.078.074.0100.085.5
0.8 s window92.082.052.090.079.0
3.2 s window88.090.070.098.086.5
Layer 1286.086.054.096.080.5
Layer 2494.078.072.092.084.0

Because intent targets come from the teacher's encoding of the inputs, a new window or layer only requires re-encoding, not regenerating text, so adapting them to a new task is cheap. These results complement our existing controls on what the teacher sees, what it is asked, and which hidden states supervise the student, each of which favors the default choice (Tables 22 to 24).

Reviewer_5sJvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Intention Distillation (INDI), which uses purpose-conditioned supervision from demonstrated execution segments to improve vision-language-action models. A frozen teacher processes the current observation, task instruction, action summary, and execution video to provide intermediate input representations and textual-purpose targets. A separate frozen visual encoder provides endpoint-image features. Using only currently available inputs, the student recovers the teacher representation at an intermediate action-decoder layer. Context routing and intent-mismatch training encourage downstream predictions to use this representation. Experiments on two VLA backbones, two simulation benchmarks, and real-world manipulation tasks demonstrate performance improvements. Component ablations, target-construction controls, representation probes, and behavioral interventions further examine the method.

Critical strengths
  1. The method addresses both representation supervision and downstream use, with targeted empirical support. Beyond aligning intermediate states with teacher features, the paper introduces context routing and mismatch training to discourage the decoder from bypassing these states. The factorial ablation of intent alignment and mismatch training in Table 11 supports their individual and combined contributions. Comparisons between functional-purpose and neutral-description prompts under matched execution evidence, together with target-source ablations, provide meaningful evidence for the specific supervision design.
  2. The evaluation is broad and accompanied by informative analyses. Both backbones improve on both simulation benchmarks, while real-world experiments cover standard scenes, held-out objects, and distractors. The paper also reports teacher substitutions, representation probes, closed-loop interventions, and computational costs. Reporting regressions and unsuccessful goal-redirection interventions helps clarify the method’s limitations rather than presenting only favorable evidence.
Critical weaknesses
  1. Controlled comparisons with closely related methods are insufficient to establish the incremental contribution. Comparisons against the original backbones and internal variants are reasonably comprehensive. However, methods such as Fast-ThinkAct, LaRA-VLA, and DualCoT-VLA already investigate teacher-guided latent planning, multimodal intermediate representations, or query-token supervision. The paper does not provide a direct comparison with a representative related approach under matched backbone, data, and training conditions. The future-supervision control is useful for comparing supervision targets, but it does not constitute a full reproduction of a related method. Likewise, published results obtained under different settings cannot establish a clear comparative advantage. The current evidence therefore supports improvements over the base policies and tested internal controls more strongly than it establishes additional value over existing latent-reasoning and representation-supervision approaches.
  2. Sensitivity to the multi-objective training configuration remains insufficiently characterized. The method combines action prediction, intent alignment, visual and textual supervision, and mismatch training, together with a mismatch margin, context-dropout probability, and auxiliary-loss warmup schedule. While component ablations are provided, the paper does not systematically examine sensitivity to the loss weights and related hyperparameters. Removing a component establishes its contribution under the selected configuration, but does not demonstrate robustness to reasonable parameter choices. Furthermore, most mechanism analyses are conducted with GR00T-N1.7 on a single fixed SimplerEnv-Bridge evaluation run. The three evaluation runs in the main tables do not substitute for uncertainty estimates across independent training runs. Consequently, the robustness of the training recipe to hyperparameter choices, training randomness, and task settings remains unclear.
Questions for authors
  1. Could the authors provide a controlled comparison with a representative latent-reasoning or teacher-representation-distillation approach using the same backbone, training data, and optimization settings? It would be particularly useful to isolate the additional benefits of execution-evidence targets and routing/mismatch training relative to such a baseline.
  2. Could the authors report sensitivity analyses for the principal loss weights, especially intent alignment, visual/textual supervision, and mismatch training? How stable is performance under reasonable changes to the mismatch margin and context-dropout probability, and how were the default values selected?
  3. Could the authors report results across independent training seeds for the key component ablations and prompt comparison, and validate the most important findings on the second backbone or RoboCasa? Please also clarify whether the three evaluation runs in the main tables use the same trained checkpoint.
5 comments

We thank Reviewer 5sJv for the constructive review and the concrete suggestions for controlled comparisons, which have helped us strengthen the paper. We also appreciate the recognition of INDI’s joint treatment of supervision and downstream use, the broad evaluation with informative analyses, and the transparent reporting of limitations. Following these suggestions, we add a matched reproduction of DualCoT-VLA and a teacher-distillation baseline that isolates the gains of execution evidence and routing/mismatch training (W1, Q1), together with sensitivity and training-seed results (W2, Q2, Q3). We uploaded a revised version of the paper, with all changes highlighted in blue.

[W1, Q1] Controlled Comparison with Related Methods

INDI outperforms both a reproduction of DualCoT-VLA and a teacher-representation distillation baseline on the fixed evaluation run of Tables 9 to 11, and the second comparison isolates the gains from execution evidence in the intent target and from routing and mismatch training. Both comparisons share INDI’s backbone (GR00T-N1.7), data, optimization budget (20,000 steps, global batch size 1,536), and checkpoint selection (Appendix A.6).

Reproduction of DualCoT-VLA. Among the three methods, we reproduce DualCoT-VLA because it is trained in a single stage, whereas Fast-ThinkAct and LaRA-VLA require multi-stage VLM training. Fast-ThinkAct trains a teacher with GRPO reinforcement learning and distills it into a student, and LaRA-VLA trains on constructed CoT datasets through a three-stage curriculum. We implement DualCoT-VLA on GR00T-N1.7, keeping all its main components, including VLM training and DA3 feature supervision (Appendix A.3). INDI exceeds this reproduction by 9.0 pp on average, while the reproduction is higher on Carrot (80.0% vs. 78.0%). Over the three evaluation runs of Table 1, the reproduction averages 75.0%, against 84.7% for INDI.

Method (Bridge)SpoonCarrotStackEP-BasketAvg.
GR00T-N1.782.068.066.030.061.5
+ DualCoT-VLA (reproduced)90.080.064.072.076.5
+ INDI90.078.074.0100.085.5

Teacher-distillation baseline. As suggested in Q1, we start from a baseline that distills the same frozen teacher’s representation of the current scene into the decoder’s intermediate states, the principle shared by teacher-representation distillation methods. As in Appendix B.7, all four variants keep INDI’s visual and textual targets, so the comparison changes only the evidence used for the intent target. We then add INDI’s two components one at a time. Execution evidence in the intent target adds 2.0 pp without routing and 13.0 pp with it, and routing with mismatch training adds 9.0 pp with current-scene targets and 20.0 pp with execution evidence. The two components are complementary, and together they give the largest gain.

Evidence used for the intent target (Bridge)Without routing and mismatchWith routing and mismatch
Current scene only63.5 (distillation baseline)72.5
Current scene + executed segment65.585.5 (INDI)

Source of the additional value. Existing methods, including DualCoT-VLA, train the VLM to produce intermediate outputs such as CoT text or geometric features. INDI keeps the VLM frozen and instead recovers, within the action decoder, the representation that a teacher forms after observing the executed segment, with routing and mismatch training making the decoder rely on it.

We added the reproduction to Table 1, its details to Appendix A.3, and the 2×2 control to Appendix B.7.

[W2, Q2] Sensitivity to the Training Configuration

Performance is stable under reasonable changes to the loss weights and hyperparameters. We use one configuration for all five settings, two backbones on two simulation benchmarks and a real robot, without per-task tuning (Section 4.1). Moderate changes to the main hyperparameters change average success by at most 4.0 pp.

Setting (Bridge)Avg.
Default (λ_I = λ_V = λ_R = 0.5, λ_mis = 0.1, margin 0.05, context dropout 0.5)85.5
Context dropout 0.25 / 0.7583.0 / 82.0
λ_mis 0.05 / 0.284.5 / 84.0
Margin 0.025 / 0.185.0 / 82.5
λ_I 0.25 / 1.081.5 / 86.5
λ_V, λ_R 0.25 / 1.083.0 / 84.5

In contrast, removing each term lowers success (intent alignment 63.0, mismatch training 74.0, visual and textual supervision 76.0, Tables 9 to 11). Each term matters, while its exact weight does not.

We selected the defaults once with a small sweep on Bridge and RoboCasa Kitchen and then fixed them for every backbone and setting. The alignment weights follow the default of REPA (0.5), the 1,000-step grounding warmup matches the backbones’ learning-rate warmup, and a context-dropout probability of 0.5 splits updates evenly between the two routing states. λ_mis and the margin were set once without per-task tuning, and the results above show stable performance around them. We added these results and the selection procedure to Appendix A.6.

[W2, Q3] Robustness of Key-Component Gains

The key gains hold across independent training seeds. Each configuration in the main tables was trained once, and its three evaluation runs come from the same checkpoint, so we added independent training seeds.

Training seeds. We retrained INDI, the no-mismatch variant, the neutral-prompt variant, and the baseline with two additional seeds and evaluated all runs with a fixed evaluation seed on Bridge.

Model (3 training seeds)SpoonCarrotStackEP-BasketAvg.
GR00T-N1.782.7 ± 3.173.3 ± 5.061.3 ± 5.035.3 ± 4.663.2 ± 2.5
+ INDI w/o mismatch91.3 ± 3.180.0 ± 4.037.3 ± 7.088.7 ± 9.274.3 ± 3.0
+ INDI, neutral prompt87.3 ± 3.176.7 ± 5.057.3 ± 3.182.0 ± 2.075.8 ± 3.3
+ INDI90.0 ± 2.080.0 ± 2.067.3 ± 7.095.3 ± 4.283.2 ± 2.8

Every INDI seed outperforms every seed of the other three models, including the neutral-prompt variant (lowest INDI seed 80.0% vs. highest neutral-prompt seed 79.0%). The gains from INDI, from mismatch training, and from the functional prompt are therefore not explained by training randomness.

Second backbone. On π0.5 (Bridge, one training run, averaged over three evaluation runs), removing mismatch training lowers success from 58.8% to 55.8%, giving up about half of INDI’s gain over π0.5 (3.0 of 6.5 pp), matching the trend on GR00T-N1.7 (85.5% to 74.0%, baseline 61.5%). INDI’s overall gains already transfer to π0.5 and RoboCasa Kitchen with the same configuration (Tables 1 and 2).

We clarified the evaluation protocol in Appendix A.2 and added the seed results to Appendix B.2.

AuthorsOpenReview ↗

A Gentle Reminder

Dear Reviewer 5sJv,

Thank you again for your constructive review and concrete suggestions for controlled comparisons, which have helped us strengthen the paper.

As the discussion period closes soon, we would like to confirm that our response reached you. We reproduced DualCoT-VLA with the same backbone, data, and optimization budget, added a teacher-representation distillation baseline that separates the gains of execution evidence and of routing with mismatch training, and reported sensitivity to the loss weights and hyperparameters together with results across independent training seeds.

We would appreciate any feedback on whether our response addresses your concerns, and would be happy to clarify any remaining points.

Best, Authors

Comments on the paper

2 comments
AuthorsOpenReview ↗

Discussion Update and Request for Follow-Up

Dear Area Chair and Reviewers,

We thank the reviewers for their constructive feedback. The reviewers highlighted the following strengths of INDI:

  • Purpose-oriented supervision: "an effective approach for exploiting high-level behavioral information in demonstrations" (9TEh), with "the combination of execution-based supervision and explicit training for downstream use" as its main strength (jR8m).
  • Mechanisms backed by ablations: targeted ablations support context routing and mismatch training (5sJv, 9TEh, jR8m).
  • Consistent gains: improvements on two backbones, two simulation benchmarks, and real robots (5sJv, 9TEh, jR8m), without the teacher at deployment (9TEh, jR8m).
  • Thorough evaluation: "broad and accompanied by informative analyses" (5sJv).

During the rebuttal, we ran new experiments and added them to the revised paper, with all changes highlighted in blue. We have provided point-by-point responses, fully supported by the experiments and clarifications summarized below.

  1. Independent training seeds (5sJv W2/Q3, 9TEh Q4, jR8m W1/Q1): We retrained INDI, the baseline, and two ablations with two more seeds. Every INDI seed outperforms every seed of the other three models (Appendix B.2). On π0.5\pi_{0.5}, a new run without mismatch training gives up about half of INDI's gain, as on GR00T-N1.7.

  2. Hyperparameter sensitivity (5sJv W2/Q2, 9TEh W2/Q2): We varied the main hyperparameters and donor selection. Average success changes by at most 4.0 pp (Appendix A.6). The defaults were set once and are shared by all five settings without per-task tuning.

  3. Teacher-target construction (9TEh Q3): We varied the teacher window and the extraction layer. Average success stays within 6.5 pp of the default (Appendix A.6).

  4. Comparison with related methods (5sJv W1/Q1): We reproduced DualCoT-VLA under the same setup. INDI outperforms it (85.5% vs. 76.5%, Appendix A.3). We also built a teacher-representation distillation baseline. Execution evidence and routing with mismatch training are complementary and together raise its success from 63.5% to 85.5% (Appendix B.7).

  5. Mismatch-update frequency (jR8m W2/Q2): We reran the no-bottleneck variant with a matched mismatch frequency. Routing still adds 17.5 pp (Appendix B.2).

  6. Intervention controls (9TEh Q1): We added size-matched random controls. Stage edits reduce success far more than the random controls (7.5% and 5.5% vs. 42.5% and 39.5%). Editing the intent alone changes which failures predominate. We also ran the requested same-stage donor experiment (Appendix B.3). These controls support the behavioral relevance of stage-related information, but do not isolate local purpose from object and scene information or demonstrate reliable redirection to an alternative goal.

  7. Ambiguous goals (9TEh W1): We tested a scene with two valid goals. Deterministic INDI succeeds in 96.0% of episodes and uses both goals. A stochastic-intent variant does not improve success (Appendix B.3).

  8. Training cost (9TEh W2/Q4, jR8m W3/Q3): We measured target-generation time and storage for each dataset. Targets are generated once and are smaller than the decoded training frames. On Bridge, the first run takes about 85 GPU-h including target generation. Later runs take 57 GPU-h, against 51 GPU-h for the baseline (Table 7, Appendix A.7).

We thank all reviewers for the suggestions that guided these additional experiments and revisions. We have not yet received follow-up feedback on our rebuttal. As the discussion period is nearing its close, we would appreciate follow-up from Reviewers 5sJv, 9TEh, and jR8m on whether our responses and revised manuscript address their concerns. We would also be grateful for the Area Chair's help in facilitating this follow-up and clarifying any remaining substantive concerns.

Best regards,

Authors

AuthorsOpenReview ↗

Brief Follow-Up Before the Discussion Closes

Dear Area Chair and Reviewers,

Thank you for your constructive comments, which have helped us improve the paper. We have made every effort to address each concern through detailed, point-by-point responses supported by additional experiments and evidence. These additions and clarifications are incorporated into the revised manuscript, with changes highlighted in blue.

We thank Reviewer 5sJv for the follow-up and for maintaining a positive assessment. We would be grateful if Reviewers 9TEh and jR8m could read our responses and revisions and briefly acknowledge that they have considered the new evidence. If any original concerns remain unresolved, we would appreciate knowing which points require further clarification.

Best regards, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.