acceptodds
Under review as a conference paper at ICLR 2027

RAFT: Reasoning Aligned Fine Tuning Improves Robustness of VLA Post Training

Abstract

Adapting a generalist vision-language-action (VLA) policy to a target task with limited demonstrations can improve nominal task success while reducing robustness to variations in appearance, objects, and instructions. We introduce reasoning- aligned fine-tuning (RAFT), a two-stage post-training procedure: RAFT first trains low-rank adapters on task-relevant reasoning annotations, then freezes them while fine-tuning the policy on action demonstrations. On a physical DROID robot, RAFT achieves about 75% out-of-distribution success, compared with about 28% for plain fine-tuning; on a multi-stage bimanual YAM task, it reaches 70% mean success, compared with 36% for the strongest baseline. RAFT also improves success under visual, object, and instruction shifts in simulated pick-and-place and drawer-opening tasks. Compared with plain fine-tuning, RAFT policies place more attention on annotated task regions and yield higher linear-probe accuracy for object identity in later layers, and a broader reasoning-data mixture further improves downstream success across demonstration budgets.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Post Training, Robotics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7oueOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RAFT, a post-training framework to improve the robustness of generalist VLAs when adapting to nominal tasks. RAFT is a two-stage post-training method. Specifically, in Stage 1, RAFT freezes the pretrained model and trains LoRA adapters, with the goal of mimicking reasoning annotations generated by a proprietary VLM. In Stage 2, RAFT freezes the LoRA adapters and fine-tunes the VLM backbone and action head under the SFT paradigm. Experimental results on both simulated and physical robot platforms show that RAFT can outperform existing methods.


As 'Questions For Authors' is for some Q's that answer can 'materially affect your assessment', I might just leave some of my suggestions here.

  1. The authors argue that "reasoning supervision has not been studied" in the context of low-data VLA post-training. However, I would like to point out that "not having been studied" is not a sufficient motivation. There could be many things in the world that have not been studied; some are valuable, while others are not. To make the paper better motivated, I would suggest that the authors focus on why reasoning supervision is particularly valuable for robust VLA adaptation, rather than simply highlighting a gap in the literature.

Meanwhile, I would like to point out that ECoT-Lite shares a very similar idea with RAFT, particularly in its reasoning pre-training strategy, which separates reasoning supervision from subsequent action training. The authors should better clarify the novelty of RAFT over this existing work, particularly the contribution of freezing reasoning adapters during action fine-tuning.

Chen, William, et al. "Training Strategies for Efficient Embodied Reasoning." Proceedings of the Conference on Robot Learning (CoRL), 2025.

Critical strengths

Good empirical performance, particularly on physical robot platforms.

Critical weaknesses
  • Distilling knowledge from a strong proprietary model and aligning the policy through supervised fine-tuning are strategies that have been widely explored over the past few years. Although RAFT introduces a two-stage training strategy with complementary parameter freezing, the paper does not sufficiently establish why this particular design leads to better robustness, beyond the empirical improvements.

  • As shown in Figure 6, RAFT achieves higher success rates under several OOD conditions than under the corresponding ID condition. Performing better on OOD than on ID is counterintuitive and needs analysis. In the menahwile, RAFT is proposed to address the low-data adaptation problem, as shown in Figure 11 (right). However, RAFT performs similarly to Plain FT when only 25% of the demonstrations are provided, with relatively modest gains at intermediate demonstration budgets.

Questions for authors
  1. Further analysis is required for why RAFT perform better on OOD than on ID. Is it a data issue, a setting issue, or some hidden gem to be explored in the algorithm
13 comments

Thank you for the review and for the pointer to ECoT-Lite! We start with the question you marked as material to your assessment, why RAFT can score higher OOD than ID, and answer it with new data: object start positions, per-colour success on both robots, and confidence intervals. We then address the low-data budgets in Fig. 11, and finally the motivation, why the design improves robustness, and the relation to ECoT-Lite. Details follow below.

Please let us know if there are additional questions we could address.

AuthorsOpenReview ↗

Why OOD success can exceed ID success (Fig. 6)

"Further analysis is required for why RAFT perform better on OOD than on ID. Is it a data issue, a setting issue, or some hidden gem to be explored in the algorithm"

It is mainly a property of the evaluation design, and the same analysis shows where RAFT's gain comes from.

1. ID episodes probe the edge of the demonstrated distribution; OOD episodes probe interpolation under shift. ID episodes place objects at the periphery of the placement region, the hardest positions represented in the demonstrations. The OOD conditions ask a different question: when appearance, objects, clutter or pose change, can the policy still act within the motions it was trained on? They therefore use unseen configurations inside the demonstrated range. We will add the start-position distributions for both robots to the revision.

2. This test is what separates the methods. On identical episodes, the baselines fail once the scene changes: Plain FT falls from 60% ID to 28% OOD on DROID, and the baselines succeed in 13–33% of YAM OOD-pose trials, against 70% for RAFT. Their remaining successes concentrate on the red toy, one of the novel-colour objects shown in Fig. 20:

  • DROID OOD-object: the fine-tuned baselines (Plain FT, RETAIN, DeLock) succeed in 50–88% of red-toy trials but only 13–25% of trials with other colours; RAFT reaches 100% and 75%.
  • YAM (clutter, pose and background pooled): the baselines succeed in 41–52% of red-toy trials and 18–25% with other colours, a gap of 23–27 points; RAFT's gap is 9 points (73% vs. 64%).
  • When the red toy is not the target, the baselines collapse: 23–30% on the YAM colour shift (RAFT 80%), and 17–33% on DROID OOD-distractor, where the red toy appears only as a distractor (RAFT 75%).

RAFT follows the instruction across colours and copes with unseen configurations; the baselines largely succeed only when the target is the red toy.

  1. The remaining ID–OOD difference is within evaluation noise. On DROID, RAFT's ID success is 19/30 (63%; 95% Wilson interval 45.5–78.1%) and its OOD-object success 20/24 (83%; 64.1–93.3%); the intervals overlap. OOD success above ID success has also been reported in prior VLA post-training work (Yadav et al., 2026). We will add this discussion and the intervals to the revision.

(Yadav et al., 2026) Robust Fine-tuning of Vision-Language-Action Robot Policies via Parameter Merging

AuthorsOpenReview ↗

Low-data budgets (Fig. 11)

"RAFT performs similarly to Plain FT when only 25% of the demonstrations are provided, with relatively modest gains at intermediate demonstration budgets."

With roughly 25% of the demonstrations, the two methods tie (49 of 106 ID successes each); at that budget neither policy has learned the task, and many required motions lie outside the demonstrated range. From there RAFT's lead grows with data: about 2 points at roughly 50% of the demonstrations, and 101 vs. 96 of 106 ID episodes (+4.7 points) with all demonstrations, so RAFT makes better use of additional data.

Reviewer_c7p4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to fine-tune pretrained VLA policies on limited demonstrations while keeping them robust to scene changes. RAFT first trains LoRA adapters using object boxes, keypoints, motion traces, and language descriptions. It then freezes the adapters and trains the language model, action expert, and projection layers on action demonstrations. The pretrained SigLIP weights stay frozen in both stages, while its adapters are trained only in Stage I. Experiments cover two simulated tasks and two real-robot platforms, with the largest gains on real robots. Further analysis suggests that RAFT focuses more on relevant objects and locations. Its internal representations also make it easier to identify objects and estimate how far an episode has progressed.

Critical strengths
  1. S1: On DROID, RAFT achieves 75% OOD success, compared with about 28% for plain fine-tuning, while ID success stays close (63% versus 60%). This suggests that RAFT handles changes in the scene better than plain fine-tuning. On YAM, it reaches 70% mean success across ID and OOD conditions, compared with 36% for the strongest baseline. These results cover two robot platforms using different policy backbones.

  2. S2: The method is straightforward and does not require generating reasoning text when controlling the robot. On simulated pick and place, freezing the adapter during action training works better than continuing to update it. The frozen-vision baseline is also a useful control. Its mean success on simulated pick and place is lower than RAFT's, so freezing the pretrained vision weights alone does not explain the full improvement.

Critical weaknesses
  1. W1: The analysis does not fully explain why RAFT improves robustness. Even with the adapter frozen, updating the language backbone can change how the model uses what the adapter learned. The probes are tested on episodes that the policy has already seen during training, although the probes themselves were not trained on those episodes. Also, the task-progress labels divide each episode into four time intervals; they do not label actual action stages. The results show that RAFT learns different representations, but do not show that RAFT is more robust because it keeps using what it learned in the reasoning stage. This concern is about the explanation, not the reported success rates.

  2. W2: The comparison with joint training needs stronger controls. The authors test only a 50/50 sampling ratio and equal loss weights. Equal weights do not necessarily give text and action learning equal influence. It's also unclear whether the methods see the same number of examples from each objective. On simulated pick and place, RAFT reaches 83.21% mean success versus 79.62% for plain fine-tuning. The Wilson intervals show uncertainty over evaluation trials, but the paper does not report variation across independent training runs. Thus, it's not clear how consistently this smaller gain would repeat. The real-robot gains are larger, but they do not resolve these questions about the training comparison.

Questions for authors
  1. Q1: Is LoRA dropout turned off during Stage II? On reasoning or grounding examples not used for training, how does performance change after action fine-tuning? This would help check whether the model can still perform the reasoning or grounding tasks learned in Stage I.

  2. Q2: Do joint training and RAFT use the same number of action and reasoning examples during training? Does joint training perform better with different loss weights or sampling ratios?

  3. Q3: Do the simulation gains hold across multiple training seeds? Since the methods are evaluated on the same episodes, could you use these matched results to estimate the success-rate differences and their uncertainty against plain fine-tuning and the frozen-vision baseline?

  4. Q4: Do the linear probes still perform better with RAFT on episodes that the policy has never seen during training, especially under OOD shifts?

7 comments

Thank you for the positive assessment and the precise questions! To address them, we (1) reran the probes on episodes never used in policy training, including OOD-object episodes, with planner-annotated task phases, and tested whether RAFT can still perform the Stage-I grounding task after action fine-tuning; (2) strengthened the co-training comparison with matched action exposure, re-weighted losses and a frozen vision encoder; and (3) trained a second independent seed and report paired differences, with uncertainty, against Plain FT and the frozen-vision baseline. Details follow below.

Please let us know if there are additional questions we could address.

AuthorsOpenReview ↗

Explanation, reasoning after Stage II and probes (W1, Q1, Q4)

"The results show that RAFT learns different representations, but do not show that RAFT is more robust because it keeps using what it learned in the reasoning stage."

We tested this directly with two new analyses.

Q1: LoRA dropout and grounding after action fine-tuning. LoRA dropout is off in Stage II; the adapters are frozen and act deterministically. Prompted for bounding boxes on 128 held-out frames, RAFT reaches mean IoU 0.563 at the end of Stage I and 0.381 after action fine-tuning (shuffled control: 0.04). RAFT retains about two thirds of its Stage-I grounding accuracy after action fine-tuning, nearly ten times the shuffled control, so the model can still perform the Stage-I task.

Task-progress labels. Following your comment, we re-annotated every frame with the manipulation phases emitted by the data-generating planner and reran the analysis; all task-phase results below use these planner phases.

Q4: Probes on episodes the policy never saw. We reran the probes on 10 new episodes from unseen seeds (60 frames each) and on 10 further OOD-object episodes (balanced accuracy, layers 14–17):

TaskProbeSplitBasePlain FTRAFT
Pick and placeObject (chance 1/7)Held-out0.380.300.64
Pick and placeObject (chance 1/7)OOD-object0.380.300.63
Pick and placeTask phase (chance 1/4)Held-out0.480.500.65
Pick and placeTask phase (chance 1/4)OOD-object0.380.440.46
Open drawerTask phase (chance 1/4)Held-out0.440.390.47

Plain fine-tuning lowers object decodability below the pretrained policy, while RAFT raises it, and this holds on OOD-object episodes, where every reasoning-modality variant (0.43–0.74) stays above both Base and Plain FT. RAFT also remains highest on task phase under OOD-object shift (0.46, vs. 0.44 for Plain FT and 0.38 for Base).

Together with the adapter ablation (keeping the adapters trainable in Stage II drops mean success from 83.2% to 74.0%), these results link what is learned in Stage I to the policy's behaviour after Stage II.

AuthorsOpenReview ↗

Joint-training controls (W2, Q2)

"Do joint training and RAFT use the same number of action and reasoning examples during training? Does joint training perform better with different loss weights or sampling ratios?"

Example counts. We re-ran co-training for 10,000 steps so that it sees exactly as many action examples as Plain FT and RAFT (2.56M), all else unchanged. Co-training also sees far more reasoning examples than RAFT (2.56M vs. 48k in RAFT's Stage I), yet it still falls well below RAFT, so the gap does not come from limited reasoning exposure.

Loss weights and frozen vision. We also re-weighted the losses (0.9 action / 0.1 reasoning) and froze SigLIP as in RAFT. Pick and place, success (%), the same 106 episodes per split:

MethodIDLightTextureObjectPromptMean
Plain FT90.5787.7492.4535.8591.5179.62
Co-train, 5k steps (submitted)79.2570.7572.6417.9275.4763.21
Co-train, 10k steps (matched)83.0277.3671.7018.8784.9167.17
Co-train, SigLIP frozen80.1975.4775.4720.7573.5865.09
Co-train, 0.9 action / 0.1 reasoning79.2573.5872.6416.0478.3063.96
RAFT95.2893.4092.4543.4091.5183.21

Co-training stays at 63–67% in every setting, below Plain FT and 16–20 points below RAFT. Matched exposure raises ID, lighting and instruction success, but not texture or object shift.

Reviewer_5KCgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Fine-tuning a vision-language-action model (VLA) on a small set of demonstrations can improve success on the current task, but performance may drop when objects, appearance, or instructions change. This paper proposes RAFT, which splits training into two stages. It first trains vision and language LoRA adapters using task-related reasoning annotations. It then freezes these adapters and fine-tunes the language backbone, action module, and projection layers using action demonstrations. The experiments cover simulated manipulation, DROID robot tasks with MolmoB0t, and a YAM bimanual task with π0.5. The paper also studies different reasoning annotations, adapter freezing, joint training, reasoning data size, and learned features. Presentation: Generally clear and well organized, but some implementation details and model-selection choices need more explanation. Informativeness: The experiments, especially the physical robot results, help assess whether staged reasoning training improves VLA fine-tuning. Soundness: 2: I have some concerns that could be addressed with clearer explanations or additional experiments.

Critical strengths

The paper addresses a practical problem in fine-tuning with limited data and reports large gains on two physical robot platforms using different models. On DROID, out-of-distribution (OOD) success rises from about 28% to 75%, while in-distribution (ID) performance remains similar. On YAM, average success reaches 70%, compared with 36% for the strongest baseline. The appendix includes useful comparisons of joint training, adapter freezing, vision-encoder freezing, reasoning annotations, and combining RAFT with RETAIN. The methods use the same simulation test episodes, and physical trials are interleaved, making the comparisons more reliable.

Critical weaknesses

The results support the overall training method, but they do not fully explain how much each design choice contributes.

  1. The joint-training baseline samples reasoning and action data at a 50/50 ratio and gives the two losses equal weights. However, equal weights for text cross-entropy and action loss do not mean they have equal influence on training. Joint training also updates the vision encoder, while RAFT freezes the pretrained vision weights and freezes the vision adapters in the second stage. The paper adds a Plain FT baseline with a frozen vision encoder, but only for simulated pick and place. In the OOD-object test, freezing the vision encoder alone raises success from 35.85% to 40.57%, compared with 43.40% for RAFT. Please compare joint training under the same training conditions and test vision and language freezing separately, especially on the physical tasks with the largest gains.
  2. Appendix A.4 states that the confidence intervals are Wilson intervals based on evaluation trials. It does not report the number of independent training runs or the variation between them. This matters for the smaller gains in simulation: Table 4 reports 83.21% for RAFT and 79.62% for Plain FT, with identical results under texture and instruction changes. The large gains on physical robots are still convincing, but testing the same model more times does not show that retraining will consistently reproduce those gains. Please include independent training seeds and report paired differences on the same simulation test episodes.
  3. In Appendix B.3, the 414 frames used for the linear probes come from 46 policy-training episodes. The test episodes are held out only when training the probes; they were already used to train the policy. Also, the task-phase labels divide each episode into four equal time intervals, rather than actual manipulation phases. Higher probe accuracy could therefore reflect familiarity with these trajectories or their timing. Since other language-model weights continue to change, freezing the adapters does not guarantee that their effect on the model stays the same. After Stage II, please test reasoning directly on episodes excluded from policy training and under new distribution shifts.
  4. Table 1 and Appendix A describe the MolmoB0t setup and explicitly state that these settings do not cover the π0.5 model used on YAM. However, the full YAM setup is not provided. The choice of reasoning annotations is also unclear: pick and place uses full reasoning annotations, while drawer opening uses bounding boxes. Appendix B.1.1 shows that the best annotation type varies across conditions. Please explain whether these choices were made using a separate validation set, and provide the sources and sampling ratios of the mixed reasoning data.
Questions for authors

Q1. Does RAFT still outperform joint training when both methods update the same modules, see the same number of action examples, use the same training budget, and select loss weights on a separate validation set? Q2. How many independent training runs produced the reported results? Can you report variation across runs and paired comparisons on the same simulation test episodes? Q3. How does reasoning performance on held-out data change from the end of Stage I to the end of Stage II? Do the gains in the feature analysis remain when the test images have not been used for policy training? Q4. What are the full π0.5/YAM training settings? How were the reasoning annotation types for each task and the composition and sampling ratios of the mixed reasoning data chosen?

7 comments

Thank you for your time and the detailed review! To address your comments, we (1) ran matched co-training controls and freezing ablations that isolate each design choice, including a frozen-language-model baseline; (2) trained a second independent seed and report paired comparisons on the same 530 simulation episodes; (3) reran the probes on episodes never used in policy training, with planner-annotated task phases, and tested Stage-I grounding directly after Stage II; and (4) give the full π0.5/YAM settings, how annotation types were chosen, and the reasoning-data mixture. Details follow below.

Please let us know if there are additional questions we could address.

AuthorsOpenReview ↗

Joint-training controls and freezing (W1, Q1)

"Please compare joint training under the same training conditions and test vision and language freezing separately, especially on the physical tasks with the largest gains."

Matched action exposure. The submitted co-training control split each 512-sample batch 256/256 between reasoning and actions over 5,000 steps. We re-ran it for 10,000 steps so that it sees exactly as many action examples as Plain FT and RAFT (2.56M), all else unchanged; it also sees far more reasoning examples than RAFT (2.56M vs. 48k in RAFT's Stage I), so any gap does not come from limited reasoning exposure. We also re-weighted the losses (0.9 action / 0.1 reasoning) and froze SigLIP as in RAFT. Pick and place, success (%), the same 106 episodes per split:

MethodIDLightTextureObjectPromptMean
Plain FT90.5787.7492.4535.8591.5179.62
Plain FT, SigLIP frozen91.5186.7987.7440.5791.5179.62
Plain FT, language model frozen92.4584.9185.8536.7992.4578.49
Co-train, 5k steps (submitted)79.2570.7572.6417.9275.4763.21
Co-train, 10k steps (matched)83.0277.3671.7018.8784.9167.17
Co-train, SigLIP frozen80.1975.4775.4720.7573.5865.09
Co-train, 0.9 action / 0.1 reasoning79.2573.5872.6416.0478.3063.96
RAFT95.2893.4092.4543.4091.5183.21

We find that:

  1. Co-training stays at 63–67% in every setting, below Plain FT and 16–20 points below RAFT. Matched exposure raises ID, lighting and instruction success, but not texture or object shift.
  2. Freezing a backbone alone does not explain RAFT's gain. Freezing SigLIP helps on unseen objects (40.6% vs. 35.9%) but leaves the mean at 79.6%; freezing the language model gives 78.5%.
  3. The same holds on the physical robots. Preliminary Plain FT runs with a frozen vision encoder reach 40% on DROID and 42% on YAM, against 71% and 70% for RAFT (all conditions pooled).
  4. The reasoning stage helps only when its adapters stay frozen. Keeping the adapters trainable in Stage II gives 74.0%, below Plain FT. Freezing only the vision or only the language adapters recovers most of the gain (82.5% and 80.9%), and freezing both (RAFT) is best at 83.2%.

Loss weighting does not close the gap: shifting weight to the action loss (0.9 / 0.1) gives 63.96%, no better than equal weights (63.21%). Across every setting, co-training stays well below both Plain FT and RAFT. We will include a broader loss-weight sweep in the revision.

AuthorsOpenReview ↗

Independent training seeds (W2, Q2)

"Please include independent training seeds and report paired differences on the same simulation test episodes."

We trained a second independent seed of both methods. Evaluation is deterministic, and all methods are evaluated on identical episodes (same evaluation seeds and initial conditions), so every comparison is paired.

Training seedPlain FTRAFTΔ
179.6283.21+3.59
278.8783.40+4.53

RAFT is stable across seeds (441 and 442 of 530 episodes) and outperforms Plain FT in both. Pooled over the two seeds, RAFT solves 883 of 1,060 episodes against 840 for Plain FT (+4.1 points). In both seeds the gain comes from ID (+4.7 / +7.5 points), lighting (+5.7 / +5.7) and unseen objects (+7.5 / +8.5), while both methods are near ceiling under instruction shift.

The gain is consistent across seeds in simulation and much larger on the physical robots: on DROID, 75% vs. 28% , and on YAM, 70% vs. 36% for DeLock, the strongest baseline . Every policy was evaluated from the same set of start states, with trials interleaved across resets.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.