- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents RoMAN-Flow, an offline reinforcement learning framework using autoregressive normalizing flows to model continuous robotic actions with exact likelihoods. Its NF-IQL procedure applies advantage-weighted likelihood optimization to dataset action chunks, avoiding expensive autoregressive sampling during policy updates. The optimized policy is then distilled into a one-step actor by matching intermediate flow states and final actions, enabling parallel action generation. Experiments across MetaWorld, LIBERO, RoboMimic, and real-world manipulation report improved performance over imitation learning. Notably, distillation achieves an 8.55× inference speedup on LIBERO-Long.
Critical strengths
- Introducing AR-NF architectures for robotic manipulation, which have not been widely used previously.
- The proposed algorithm focuses on both task performance and efficiency.
- Reported inference efficiency is promising for real-time execution of robot policies.
Critical weaknesses
- The methodological novelty appears primarily integrative. The architecture adapts SimFlow, the policy update instantiates IQL, and the deployment method adapts an existing one-step inverse distillation.
- Many design choices (Please refer to Q.1., Q.2., Q.3.) are possible within the proposed algorithm, but ablation studies investigating on different choices are lacking
- Writing could have been better. It was initially hard to follow the motivations and details of the approach since everything is introduced directly in the Method section. Between the Related Work and Method sections, it would be nice if a preliminary section existed to connect prior work to the base algorithm RoMAN-Flow is building on.
Questions for authors
- Q.1. How would RoMAN-Flow perform when attached to backbones other than SimFlow, e.g., TarFlow, STARFlow, FARMER, and BiFlow? I am especially curious about FARMER and BiFlow, since they further improve efficiency through one-step inversion.
- Q.2. Why should we perturb with noise during data-induced distillation? Does it empirically lead to better performance, or is it based on some theoretical insight? How does the performance change with different noise variance values?
- Q.3. Why do we need both data-induced and prior-sampled distillations? Adding on Table 8, please provide an ablation study on one more task environment that compares three methods: (1) only using the data-induced distillation loss, (2) only using the prior-sampled distillation loss, and (3) using both.
2 comments
Thank you for the follow-up.
I maintain my score for the following reason:
- The Introduction part of the latest submission makes it clear to me that the authors investigate AR-NF because there is a lack of prior work that applies AR-NF. This is itself good, but in this case, the authors should have substantially investigated the full possible variants of AR-NF categories that have been applied to other fields, e.g., LLMs. The authors' rebuttal addresses some of it due to my questions (Section 5.5), but I think the paper in its current form still lacks in this aspect. After extensive analysis, I hope the authors find some key observations that can lead to good methodological novelty, which I think is also currently lacking.
Thank you, and I hope my review further improves the quality of your paper.
Thank you for the clarification.
However, I have remaining concerns, which are largely related to substantial reorganization of the writing.
Why is tracking the exact likelihood important? The main motivation of the research starts from the limitation that flow and diffusion policies cannot exactly model the likelihood. However, the work lacks clarity on why modeling the exact likelihood is crucial.
Lack of Clear Analysis on the AR-NF family. In my perspective, the proposed method lacks novelty since the approach is to combine various pre-existing techniques. However, the authors study a new class of policy architecture called AR-NF, and I value this aspect highly. In this case, I think the authors should have presented a substantial amount of controlled ablation and analysis to reveal when and why AR-NF performs well.
Although some of the related content was prepared during the rebuttal, I think the paper requires substantial reorganization of the presentation based on this content. Therefore, I remain tilted towards weak rejection.