- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ReFlowTTA, a test-time adaptation framework for flow-based VLA models that optimizes a low-dimensional initial-noise distribution while keeping the pretrained policy frozen. Experiments across three simulation benchmarks and two real-world robot platforms report improved task success rates under various out-of-distribution conditions.
Critical strengths
-
The paper presents an interesting perspective on VLA adaptation by modifying the source distribution of action generation rather than updating pretrained model parameters.
-
The method provides a unified adaptation mechanism for different distribution shifts, with optional human corrections incorporated through inverse flow.
-
The experimental evaluation is relatively comprehensive, covering multiple simulation benchmarks, cross-embodiment transfer, and real-world robotic experiments, with consistent improvements over the evaluated baselines.
Critical weaknesses
-
Dependence on Dense Task Feedback. The method relies on dense chunk-level rewards, using privileged geometric information in simulation and an external reward model in real-world experiments. Its effectiveness under sparse, delayed, or noisy feedback remains unclear, limiting the practicality of the proposed test-time adaptation framework.
-
Insufficient Mechanism Ablations. While the paper investigates source-space dimensionality and perturbation directionality, it lacks comparisons with simpler source-distribution optimization strategies under matched interaction budgets. It is therefore difficult to determine whether the gains specifically arise from the proposed PPO-based optimization.
-
Limited Generalizability. All experiments rely on the same π0.5 backbone, leaving the applicability to other flow-based VLA architectures unverified. Moreover, the shared, state-independent source mean may limit adaptation to heterogeneous states or changing deployment conditions.
-
Limited Analysis of Human Corrections. The proposed inverse-flow mechanism is an interesting component, but its reconstruction accuracy, sensitivity to numerical integration, and individual contribution to adaptation performance are not sufficiently evaluated.
Questions for authors
-
How does ReFlowTTA perform with sparse binary rewards or noisy reward-model feedback instead of dense task-progress signals?
-
How accurately does inverse flow recover source states from human corrections, and how sensitive are the resulting improvements to inversion error and the number of correction samples?
5 comments
Responses to Reviewer p1XA (2/3)
W3-2: the shared, state-independent source mean may limit adaptation to heterogeneous states or changing deployment conditions.
A3-2: The shared mean pools feedback across states to enable efficient adaptation under a limited interaction budget. Since the frozen VLA conditions action generation on the current observation and robot state, the same source mean still allows state-dependent behaviors. We compare this design with a state-conditioned mean variant following DSRL’s conditioning architecture, under otherwise matched adaptation settings. Results on ManiSkill3 (§4.2) are as follows:
| Agent | Online SR@15 (%) ↑ | Final SR (%) ↑ |
|---|---|---|
| Base VLA | – | 33.8 |
| DSRL | 34.3 | 38.1 |
| ReFlowTTA (state-conditioned mean) | 38.9$_{\pm 0.86}$ | 42.6$_{\pm 0.94}$ |
| ReFlowTTA (shared mean) | 49.2$_{\pm 0.21}$ | 53.6_{\pm 0.15}\ |
The shared mean achieves higher success rates with lower variability across runs, supporting more effective and stable adaptation than the state-conditioned variant under the tested interaction budget.
We have added this comparison and analysis to Appendix D.1 (Lines 1315–1323).
W4: "reconstruction accuracy, sensitivity to numerical integration, and individual contribution to adaptation performance are not sufficiently evaluated."
A4: i) Reconstruction accuracy and inversion cost. We evaluate 400 correction chunks from the real-world experiments (10 per task, 20 tasks per platform), passing the recovered source noise through the actual four-step forward decoder and comparing the reconstructed actions with the original corrections. As a reference, we additionally evaluate a computationally more expensive fixed-point inversion method [5] for the same decoder, comparing both reconstruction accuracy and inversion time:
| Method | MAE on PiPER (deg) ↓ | MAE on ARX X5 (deg) ↓ | Inversion time (ms) ↓ |
|---|---|---|---|
| Fixed-point inversion | 2.2 | 2.4 | 1748.5 |
| Ten-step reverse Euler (ours) | 2.7 | 3.1 | 252.4 |
Ten-step reverse Euler yields MAEs of 2.7° and 3.1° on PiPER and ARX X5, respectively, compared with 2.2° and 2.4° for fixed-point inversion, while requiring 252.4 ms rather than 1748.5 ms (6.9× speedup). These results quantify the reconstruction accuracy and computational cost of the reverse-flow approximation.
We have added this analysis to Appendix D.3 (Lines 1340–1351).
ii) Sensitivity to numerical integration. Recovered source coordinates provide auxiliary supervision for shared-mean learning alongside execution feedback, without requiring exact reconstruction of every corrective action. We compare -step reverse Euler with fixed-point inversion [5] under the same human-correction budget on PiPER. Results are as follows:
| Inversion setting | Reconstruction MAE (deg) ↓ | Online SR@15 (%) ↑ | Final SR (%) ↑ |
|---|---|---|---|
| Without human corrections | – | 51.6 | 58.6 |
| Reverse Euler, | 5.8 | 56.8 | 62.4 |
| Reverse Euler, | 3.5 | 57.0 | 62.5 |
| Reverse Euler, (ours) | 2.7 | 57.3 | 62.9 |
| Fixed-point inversion [5] | 2.2 | 57.4 | 62.5 |
Across the tested inversion settings, online and final SR differ by at most 0.6% and 0.5%, respectively, indicating low sensitivity to numerical integration.
We have added this analysis to Appendix D.8 (Lines 1418–1428).
iii) Contribution of human corrections. Table 4 already shows that removing human corrections reduces Online SR@15 and Final SR on PiPER by 5.7% and 4.3%, respectively, demonstrating the additional gains from correction supervision (Lines 378–392, 413–417). We further evaluate sensitivity to correction quantity by capping the number of corrections at 40%, 60%, and 80% of the original count for each task. Results on PiPER are as follows:
| Correction budget (% of original count) | Online SR@15 (%) ↑ | Final SR (%) ↑ |
|---|---|---|
| 0% | 51.6 | 58.6 |
| 40% | 53.5 | 59.2 |
| 60% | 54.9 | 61.1 |
| 80% | 56.6 | 61.8 |
| 100% (original setting) | 57.3 | 62.9 |
Compared with the full correction budget, using 40% lowers Online SR@15 and Final SR by 3.8% and 3.7%, respectively; both metrics increase with the correction budget. This demonstrates the contribution of human corrections to adaptation efficiency and final performance.
We have added this analysis to Appendix D.9 (Lines 1431–1443).
Responses to Reviewer p1XA (3/3)
Q1: "How does ReFlowTTA perform with sparse binary rewards or noisy reward-model feedback instead of dense task-progress signals?"
A1: Please see W1.
Q2-1: "How accurately does inverse flow recover source states from human corrections?"
A2-1: Please see W4(i).
Q2-2: "how sensitive are the resulting improvements to inversion error and the number of correction samples?"
A2-2: Please see W4(ii) for sensitivity to inversion error and W4(iii) for sensitivity to the number of correction samples.
[1] Black et al., Training Diffusion Models with Reinforcement Learning. In ICLR 2024.
[2] Pinneri et al., Sample-efficient Cross-Entropy Method for Real-time Planning. In CoRL 2020.
[3] NVIDIA Isaac GR00T (N1.7) 2026.
[4] Wei Wu et al. A Pragmatic VLA Foundation Model. arXiv:2601.18692, 2026.
[5] Lu et al. UniSteer: Unified Noise Steering for Efficient Human-Guided VLA Adaptation. arXiv:2605.10821.
Responses to Reviewer p1XA (1/3)
We thank Reviewer p1XA for the insightful feedback and valuable suggestions. We address each comment below and highlight the corresponding revisions in red in the updated manuscript. All references to lines, tables, and sections refer to this revised version.
W1: Its effectiveness under sparse, delayed, or noisy feedback remains unclear.
A1: We test three feedback settings:
i) Sparse: binary rewards, with 1 only for the chunk that completes the task and 0 otherwise;
ii) Delayed: rewards delivered Δ chunks later, remaining paired with their original chunks and used for updates only after arrival;
iii) Noisy: r~t=rt+ϵt, where ϵt∼N(0,(ασr)2) and σr is the standard deviation of the unperturbed rewards within each rollout batch. Results on ManiSkill3 (§4.2) are as follows:
i) Sparse: Online and final SR decrease by 7.5% and 8.4%, respectively, while Final SR remains above Base VLA (45.2% vs. 33.8%), demonstrating that dense feedback is an important design choice for rapid adaptation.
ii) Delayed: Final performance remains stable, but postponed updates slightly slow online adaptation, with Online SR@15 dropping by up to 2.3%.
iii) Noisy: Online and final SR remain stable under reward noise, deviating by at most 0.2% and 0.5%, respectively.
We have added this comparison to Appendix D.4 and discussed the associated limitation in Appendix G (Lines 1355–1370, 1488–1490).
W2: it lacks comparisons with simpler source-distribution optimization strategies under matched interaction budgets.
A2: We adopt PPO to balance reward improvement and update stability under limited deployment feedback. Its clipped objective discourages overly aggressive updates to the source distribution. We compare two simpler strategies under the same setting:
i) REINFORCE [1], which updates the mean through reward-weighted log-likelihood maximization;
ii) a CEM-style update [2], which moves the mean toward the average of high-reward source samples. Results on ManiSkill3 (§4.2) are as follows:
Compared with REINFORCE, clipped PPO improves Online SR@15 and Final SR by 3.3% and 4.4%, respectively; compared with the CEM-style update, the gains are 6.7% and 7.2%. These results show that clipped PPO is more effective for source-mean learning under the matched interaction budget.
We have added this comparison to Appendix D.6 (Lines 1386–1397).
W3-1: leaving the applicability to other flow-based VLA architectures unverified.
A3-1: Because ReFlowTTA updates only the source-noise mean while keeping the pretrained VLA frozen, its adaptation rule is not tied to a particular flow-based VLA backbone. To test its applicability to other flow-based VLA architectures, we evaluate it on GR00T N1.7 [3] and LingBot-VLA-4B [4], comparing the frozen base policy, a DSRL-based variant, and ReFlowTTA.
Compared with DSRL-based adaptation, ReFlowTTA improves Online SR@15 and Final SR by 10.2% and 11.7% on GR00T N1.7, and by 6.3% and 8.3% on LingBot-VLA. These results demonstrate the applicability of source-mean adaptation across different flow-based VLA backbones.
We have added this comparison to Appendix D.7 (Lines 1401–1414).