- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Adjoint Guidance Flow (AGF), which amortizes trajectory-aware critic guidance into a lightweight guidance network while keeping the pretrained flow-based VLA policy frozen.
Critical strengths
1.AGF formulates critic-guided flow generation as a deterministic optimal control problem and characterizes the optimal guidance as a costate that propagates the terminal critic gradient through the remaining flow trajectory.
2.By learning a separate lightweight guidance network, AGF preserves the pretrained VLA and requires only one guidance-network forward pass per flow step, eliminating the critic ensemble, back-propagation, and adjoint computation during inference.
3.Experiments across LIBERO, RoboCasa, LIBERO-Pro, three flow-based VLAs, and real-robot tasks show that AGF consistently improves pretrained policies while achieving competitive performance and substantially lower inference cost than existing critic-guidance methods.
Critical weaknesses
-
The paper lacks a ablation comparing the same guidance network trained with trajectory-wise costates versus one-step QGF gradients, making the benefit of trajectory-aware supervision unclear.
-
AGF requires a separate critic ensemble and guidance network for every task, limiting its scalability to large-scale multi-task deployment.
-
Since the performance difference between AGF and QGF is not substantial, the paper should analyze when the inference-cost savings offset the additional cost of training a task-specific guidance network.
Questions for authors
-
Can the authors compare the same guidance network trained with trajectory-wise costates versus one-step QGF gradients to isolate the benefit of trajectory-aware supervision?
-
Since AGF trains a separate critic ensemble and guidance network for each task, can the authors quantify the resulting cost and scalability limitations in multi-task deployment, and, if possible, evaluate whether either component can be shared across tasks?
-
Can the authors report the complete offline training cost and the break-even number of action chunks or deployment episodes required for AGF’s inference savings to offset its additional guidance-network training cost?
3 comments
Thank you for the detailed response. I believe a score of 3 is appropriate for this paper, and I will therefore maintain my rating. Good luck!
Thank you for your follow-up response. We wish you all the best as well!
Response by authors
W1. Missing comparison with QGF-style guidance network
We thank the reviewer for pointing out our missing comparison. Accordingly, we train a small guidance network of AGF by changing only the training target to QGF gradient, by preserving all the other settings such as network architecture, training recipes, and critic functions.
As shown in Tables G1a–c of the General Comments, Distilled QGF falls below live QGF on average, while AGF (M=4) performs on par on the other suites and avoids the drop on LIBERO-Spatial. AGF variants also achieve higher mean success than Distilled QGF at almost all tested weights, and AGF (M=4) shows the smallest drop under LOTO (Table G1b). Moreover, both the QGF target and Distilled QGF are less aligned with the exact adjoint, with the gap growing at noisier flow steps (Table G1c).
These findings support the importance of trajectory-aware adjoint targets for preserving effective guidance under amortization, beyond merely replacing online critic-gradient computation with a lightweight network.
W2. Limited scalability due to task-specific critic and guidance and effect of shared guidance
Per your request, we first test whether the trained guidance network can be shared across different tasks. Specifically, for a given target task, we apply the critic and guidance network trained on each of the other LIBERO-Goal tasks with a fixed guidance weight. As shown in Table R1, cross-task guidance falls significantly below own-task guidance (−4.2, 95% CI [−7.4, −1.0]) and stays at the base level without degrading it. This indicates that the gain of AGF comes from task-specific critic information.
Table R1. Cross-task transfer on LIBERO-Goal (SmolVLA, AGF M=4, w=4), averaged over the other nine source tasks.
We agree that per-task training limits scalability. As described in Section 4, we follow prior works [1,2] that train a critic per task, and the critic-guidance methods we compare against share this setting. We consider a shared task-conditioned critic beyond the scope of this work since our focus is on amortizing critic guidance given the same critic function.
Regarding the resulting cost, in our real-robot setting, collecting 50 rollout episodes takes approximately 2–3 hours including resets, critic training takes 2 hours 40 minutes, and guidance-network training takes 2 hours 15 minutes, about 7–8 hours per task. This cost and the number of stored guidance networks grow linearly with the number of tasks.
Importantly, the critic ensemble is needed only during training, not deployment. Deployment requires only the corresponding lightweight guidance network, so per-task inference computation does not increase with the total number of supported tasks.
Our current method is therefore most directly applicable to a fixed set of repeatedly executed tasks, while extending AGF toward shared multi-task guidance remains an important direction for future work.
W3. Training-inference cost comparison
Offline training and real-time inference operate under different computational constraints. AGF removes the critic ensemble and backward computation from deployment, reducing the guidance overhead of every action chunk, while its one-time training cost can be amortized across multiple robots performing the same task.
Importantly, AGF also achieves 84.0% real-robot success versus 68.0% for QGF (p = 0.004), using simulation-selected guidance strengths without on-robot tuning. This demonstrates a significant performance advantage under our deployment protocol, beyond inference efficiency.
For reference, the complete offline cost in our real-robot setting is about 7–8 hours per task (see W2), of which AGF requires approximately 2 hours 15 minutes of guidance-network training. Since AGF and QGF share the same critic and rollout dataset, their critic-training and data-collection costs cancel in the comparison. AGF reduces guidance overhead from 31.2 to 11.2 ms/chunk, yielding a time-equivalent break-even point of 405k chunks, or approximately 56 hours of cumulative active operation assuming two chunks per second, excluding resets and downtime.
This calculation compares computational time rather than end-to-end task completion time or monetary cost, which also depend on hardware and deployment conditions. Importantly, AGF additionally improves real-robot success from 68.0% with QGF to 84.0% (p = 0.004) under our deployment protocol, using simulation-selected guidance strengths without on-robot tuning. Thus, AGF offers both reduced online guidance overhead and an observed success-rate improvement in this real-robot setting.
Reference
[1] Test-Time Gradient Guidance of Flow Policies in Reinforcement Learning, arXiv, 2026
[2] Q-Flow: Stable and Expressive Reinforcement Learning with Flow-based Policy, ICML, 2026