- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies PPO fine-tuning of flow policies using the change-of-variables density. For general velocity fields, it shares Hutchinson probes across the old and new policies to reduce noise in the log likelihood-ratio estimate, and uses stiffness-graded integration nodes to improve numerical stability. It also introduces EDC, where the Jacobian diagonal is a bounded network output, making divergence computation inexpensive. The split-EDC variant provides an invertible discrete map with exact density computation. Experiments on Robomimic state and pixel tasks and MimicGen state tasks show competitive success rates and sample efficiency against several on-policy baselines.
Critical strengths
- Sharing probes across the PPO ratio is a useful idea. The analysis explains how this makes the estimation noise depend on the Jacobian difference between the old and new policies, giving a clear reason why sharing probes can help during PPO updates.
- The paper provides both a general approach and an EDC parameterization that makes divergence computation cheaper. The experiments cover several manipulation tasks with state and pixel observations. The parameterization ablations and wall-clock comparison are also useful for assessing the learning performance and computational cost of the method.
Critical weaknesses
None.
Questions for authors
- In Section 4.1, what metric and evaluation setting did the authors use to select the BC checkpoints? How close were the initial success rates across methods?
- Figure 3 compares shared probes with the exact trace only on Square, and Figure 4 uses one seed per setting. Could the authors clarify whether the Figure 4 results hold across seeds and whether a single shared probe also performs comparably to the exact trace on other tasks?
- In Figure 8, split-EDC is labeled “n=1–3,” but the caption says three seeds for each method. Could the authors report how many seeds were used for each task?
- Is the stiffness-graded mesh in Appendix E fixed during RL fine-tuning? If so, did the authors check whether it remains suitable as the policy changes?
- As an additional validation, could the authors add experiments on real-world robot data to assess the numerical stability of log-density computation? Ask only questions whose answers could materially affect your assessment. Distinguish clarification requests from suggestions for future work.
Kindly Reminder to Reviewer T3Cs
Dear Reviewer,
Due to your busy schedule, we understand that you may not have had sufficient time to look at our previous message yet.
We would like to kindly follow up on the possible review mismatch we raised earlier. As mentioned, the review discusses PPO fine-tuning of flow policies, Hutchinson probes, EDC, and split-EDC, whereas our submission focuses on dual-arm Kalman filtering for non-stationary dynamics.
Could you kindly double-check whether the review was correctly assigned to our submission and let us know?
We sincerely appreciate your time and understanding.
Best regards,
The Authors