- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper evaluates different methods for value guided flow policies. The paper starts with a pretrained BC policy and a critic and varies guiding time step, guiding iterations, and guiding signal. Experiments are conducted in OG bench and Robomimic. The main takeaways are: guidance is best applied after t=0.4, a small number of guidance steps is sufficient to boost performance, and different value guidance signals do differ. Furthermore, further training the value function on sampled actions improve performance.
Critical strengths
- This paper is well written and easy to understand the main takeaways. The figures presented the results very clearly.
- The experiments studies 3 main dimensions to a flow policy design which is pretty comprehensive. The environments are sufficiently diverse and representative to demonstrate the properties of different policies. The guidance budget is a clever mechanism to ensure fair comparisons across environments.
- The results are informative and provide practical guidance for users.
Critical weaknesses
- The only minor concern I have for the paper is hyperparameter sensitivity. The results suggest guidance scale need to be tuned to specific task (which is true for all guided sampling methods) but also one needs to pick a late enough guidance time step.
Questions for authors
NA
2 comments
Thank the authors for their response. I have no other questions.
We thank the reviewer for this important point. We agree that our findings do not remove the need to tune the guidance scale and timestep, which is shared by value guidance methods in general. Rather, we believe that our analysis and findings suggest how to substantially reduce the cost of setting them.
For example, since the effective guidance timesteps are defined by how each guidance signal is constructed (Section 4.2), practitioners can identify where to apply guidance from the construction of their signal rather than through a per-task search. Moreover, since our analysis shows that within this effective range what matters is applying a sufficient number of guidance steps rather than the exact timestep, the timestep within the range does not need to be tuned precisely. Building on this, Section 5.3 considers such a practical choice, where window schedules apply guidance at a fixed set of late timesteps, the same for every task and signal, instead of manually tuning them, and still largely retain the performance of dense guidance with fewer guidance steps. Nonetheless, we agree that reducing the sensitivity of value guidance to its hyperparameters is an important matter, and we leave it as future work. We will add this discussion to the revised version of the paper.
We thank the reviewer again for the positive assessment and the valuable feedback. We hope our response has addressed your concern, and please let us know if you have any further questions.