- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes using LLM to do code level policy improvement and parameter level search for multi-drone transport. The team conducted comprehensive simulation experiments for evaluation.
Critical strengths
- It is a novel idea to apply code revision using LLM to improve policy performance for robotics (drones).
- The team carried out comprehensive simulation experiments for evaluation
Critical weaknesses
-
(this comment stays true, with code as policy in the literature, this paper does not provide totally novel ideas or methodology) It is not a complete novel idea using LLM to do iterative code revision. For example, here are two papers on this topic https://arxiv.org/abs/2603.23613 https://arxiv.org/abs/2606.17514
-
(addressed) The ablation did not study how the initial handwritten program will impact the framework's performance. Multiple "random seeds" (initial handwritten programs) could be explored.
-
(addressed) The experiments did not track or report the 16 iterative rounds if the policy performance keep increasing or some code revisions will make the model become worse.
-
(addressed) For a code level revision paper, the authors could list an example of an initial handwritten code and a revised/optimized code, then compare them side to side for code differences, highlighting the differences and why the code revision worked.
Questions for authors
- If the authors would like to call this framework "policy learning", I think it's helpful to try different initial handwritten codes and see how it will impact the framework's performance. If this is "learning" instead of searching, it would be great to show the learning curve starts to converge along the 16 rounds.
- For a code level work, the authors should show the initial handwritten code and the revised code in the appendix. Then compare them side to side, section by section or line by line. Give discussions and analysis on which parts the LLM revised, and why it made sense or improved the performance.
4 comments
Response to reviewer o85q Part 2
Q4: Can the original and optimized programs be shown side by side, with an explanation of the changes and their effects? A4: We compare the complete original I1 controller with the frozen endpoint of I1 / Run 1, the first listed run of that initialization rather than the highest-scoring run. This is an additional experiment, distinct from the original manuscript's 38.8% policy. The original and optimized sources contain 29 and 297 lines, respectively, and are reproduced unmodified with line numbers.
The original program follows geometric slots using position error and switches from lift to transport by elapsed time. The optimized program retains this scaffold and adds payload-motion feedback and near-goal braking, obstacle-normal constraints and tangent steering with a remembered bypass side, and a setdown branch with a descent reference, contact memory, and unloading logic. It also adds a gated error integral for persistent offsets and attachment-motion feedback using cable tension and available thrust.
Paired interventions support obstacle handling and transport-velocity feedback in this program. The error integral shows no benefit in this run, while the individual effects of setdown and attachment feedback have not been isolated. The comparison therefore distinguishes the intended control role of each addition from its measured effects.
We added the complete source listings and a side-by-side functional comparison to Appendix F, “Original and Optimized Policy Sources,” in the supplementary material. The comparison includes source-line references, the purpose of each addition, and supporting intervention results.
The authors have addressed my comments in a very thorough manner. I have increased my score.
Thanks!
Thank you so much for the constructive comments and endorsement!
Response to reviewer o85q Part 1
We thank the reviewer for the careful review and constructive feedback. All revisions addressing the comments below are marked in blue in the revised manuscript, including the main text and supplementary material.
Q1: How does the contribution relate to prior work on iterative LLM code revision, including the two suggested papers? A1: Our goal is to improve control policies for cooperative multi-drone transport. Executable code represents the policy, and revision improves its formation, obstacle-avoidance, and payload-stabilization logic using task performance and execution feedback. We agree that iterative LLM code refinement has been studied previously; our focus is feedback-policy optimization for physically coupled transport.
We have added the suggested papers to Section 2 to acknowledge prior work on iterative code refinement using compiler and test feedback and clarify our focus on policy optimization for cooperative transport.
Q2: How sensitive is the framework to its initial control program, and why describe it as policy learning? A2: By policy learning, we mean improving a reusable feedback policy from task-execution experience. The learned object is the executable program, including its decision rules and parameters; the pretrained LLM serves as a fixed optimizer.
We evaluated three initializations with the same observation and action interfaces: I0, the original handwritten controller; I1, geometric slot following with LIFT/CARRY phases; and I2, slot following augmented with vertical cable-tension feedback and neighbor repulsion. Each initialization has three independent optimization runs with the same settings, up to 16 rounds, and a 40-million-team-physical-step cap. Final policies were frozen before evaluation on the same 250 test cases, without test-based optimization or selection. Each unchanged initial program provides the reference for its three runs.
Each row averages three runs. Across all nine runs, mean final success is 41.9%. Initialization substantially affects the outcome: I1 achieves higher and more consistent gains, although I2 starts with higher success. Initial performance alone therefore does not predict how well a program responds to optimization. All nine runs finish above their own starting programs, supporting improvement across the tested initializations while leaving room to improve reliability. Future work will examine initialization design and selection using training or validation feedback, with selection costs included in the total budget.
We clarified the policy-learning formulation in Sections 1 and 3, including the method title “Programmatic Policy Learning.” We added the initialization-sensitivity table (Table 7) and discussion to Section 4.3, with protocol details in Appendix C.
Q3: How does performance evolve over the optimization rounds, including regressions and convergence? A3: The table below summarizes key rounds for all nine runs. Success is evaluated on the same 125 validation cases, which were not used for training or checkpoint selection in these runs.
All nine endpoints exceed their starting performance. I1 improves rapidly in early rounds and then fluctuates, while I0 and I2 show larger differences among runs. These trajectories show net improvement alongside regressions and continued late gains, without establishing consistent convergence within 16 rounds.
We have added the complete round-by-round curves and a brief interpretation to Appendix E of the supplementary material.