acceptodds
Under review as a conference paper at ICLR 2027

Large Language Models as Policy Optimizers for Cooperative Multi-Drone Payload Transport

Abstract

Cooperative transport of a cable-suspended payload requires multiple drones to coordinate formation, obstacle avoidance, and stabilization under coupled dynamics. Learning and adapting these behaviors is challenging when control decisions cannot be readily inspected or revised. We develop a framework that uses pretrained large language models (LLMs) to optimize a shared control program from feedback on simulated transport tasks. The optimizer alternates code revision with local parameter search and compares candidates on the same task instances. The simulator and low-level flight controller remain fixed, and deployment executes the frozen program without online language-model inference. We construct a benchmark covering standard transport, commanded emergency setdown, and stress conditions. Our method achieves higher overall task success than proximal policy optimization (PPO) on this benchmark under the reported training budgets. Optimization-flow ablations support combining diagnostic execution feedback with local parameter search on the main benchmark.

Primary area:
applications to robotics, autonomy, planning
Keywords:
LLM agents, policy optimization, program synthesis, multi-robot cooperative transport, aerial swarms, evolutionary search, decentralized control

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_o85qOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes using LLM to do code level policy improvement and parameter level search for multi-drone transport. The team conducted comprehensive simulation experiments for evaluation.

Critical strengths
  • It is a novel idea to apply code revision using LLM to improve policy performance for robotics (drones).
  • The team carried out comprehensive simulation experiments for evaluation
Critical weaknesses
  • (this comment stays true, with code as policy in the literature, this paper does not provide totally novel ideas or methodology) It is not a complete novel idea using LLM to do iterative code revision. For example, here are two papers on this topic https://arxiv.org/abs/2603.23613 https://arxiv.org/abs/2606.17514

  • (addressed) The ablation did not study how the initial handwritten program will impact the framework's performance. Multiple "random seeds" (initial handwritten programs) could be explored.

  • (addressed) The experiments did not track or report the 16 iterative rounds if the policy performance keep increasing or some code revisions will make the model become worse.

  • (addressed) For a code level revision paper, the authors could list an example of an initial handwritten code and a revised/optimized code, then compare them side to side for code differences, highlighting the differences and why the code revision worked.

Questions for authors
  1. If the authors would like to call this framework "policy learning", I think it's helpful to try different initial handwritten codes and see how it will impact the framework's performance. If this is "learning" instead of searching, it would be great to show the learning curve starts to converge along the 16 rounds.
  2. For a code level work, the authors should show the initial handwritten code and the revised code in the appendix. Then compare them side to side, section by section or line by line. Give discussions and analysis on which parts the LLM revised, and why it made sense or improved the performance.
4 comments
AuthorsOpenReview ↗

Response to reviewer o85q Part 1

We thank the reviewer for the careful review and constructive feedback. All revisions addressing the comments below are marked in blue in the revised manuscript, including the main text and supplementary material.

Q1: How does the contribution relate to prior work on iterative LLM code revision, including the two suggested papers? A1: Our goal is to improve control policies for cooperative multi-drone transport. Executable code represents the policy, and revision improves its formation, obstacle-avoidance, and payload-stabilization logic using task performance and execution feedback. We agree that iterative LLM code refinement has been studied previously; our focus is feedback-policy optimization for physically coupled transport.

We have added the suggested papers to Section 2 to acknowledge prior work on iterative code refinement using compiler and test feedback and clarify our focus on policy optimization for cooperative transport.

Q2: How sensitive is the framework to its initial control program, and why describe it as policy learning? A2: By policy learning, we mean improving a reusable feedback policy from task-execution experience. The learned object is the executable program, including its decision rules and parameters; the pretrained LLM serves as a fixed optimizer.

We evaluated three initializations with the same observation and action interfaces: I0, the original handwritten controller; I1, geometric slot following with LIFT/CARRY phases; and I2, slot following augmented with vertical cable-tension feedback and neighbor repulsion. Each initialization has three independent optimization runs with the same settings, up to 16 rounds, and a 40-million-team-physical-step cap. Final policies were frozen before evaluation on the same 250 test cases, without test-based optimization or selection. Each unchanged initial program provides the reference for its three runs.

MethodInitial program (%)Final mean (%)Mean gain (pp)
Full-I020.825.1+4.3
Full-I125.657.3+31.7
Full-I229.243.3+14.1

Each row averages three runs. Across all nine runs, mean final success is 41.9%. Initialization substantially affects the outcome: I1 achieves higher and more consistent gains, although I2 starts with higher success. Initial performance alone therefore does not predict how well a program responds to optimization. All nine runs finish above their own starting programs, supporting improvement across the tested initializations while leaving room to improve reliability. Future work will examine initialization design and selection using training or validation feedback, with selection costs included in the total budget.

We clarified the policy-learning formulation in Sections 1 and 3, including the method title “Programmatic Policy Learning.” We added the initialization-sensitivity table (Table 7) and discussion to Section 4.3, with protocol details in Appendix C.

Q3: How does performance evolve over the optimization rounds, including regressions and convergence? A3: The table below summarizes key rounds for all nine runs. Success is evaluated on the same 125 validation cases, which were not used for training or checkpoint selection in these runs.

Initialization / runRound 0 (%)Round 4 (%)Round 8 (%)Round 12 (%)Round 16 (%)
I0 / 122.424.024.824.837.6
I0 / 222.422.425.624.824.8
I0 / 322.422.423.224.023.2
I1 / 125.646.459.258.451.2
I1 / 225.651.255.257.656.0
I1 / 325.644.051.249.649.6
I2 / 125.626.427.228.027.2
I2 / 225.627.228.026.427.2
I2 / 325.645.650.455.257.6

All nine endpoints exceed their starting performance. I1 improves rapidly in early rounds and then fluctuates, while I0 and I2 show larger differences among runs. These trajectories show net improvement alongside regressions and continued late gains, without establishing consistent convergence within 16 rounds.

We have added the complete round-by-round curves and a brief interpretation to Appendix E of the supplementary material.

AuthorsOpenReview ↗

Response to reviewer o85q Part 2

Q4: Can the original and optimized programs be shown side by side, with an explanation of the changes and their effects? A4: We compare the complete original I1 controller with the frozen endpoint of I1 / Run 1, the first listed run of that initialization rather than the highest-scoring run. This is an additional experiment, distinct from the original manuscript's 38.8% policy. The original and optimized sources contain 29 and 297 lines, respectively, and are reproduced unmodified with line numbers.

The original program follows geometric slots using position error and switches from lift to transport by elapsed time. The optimized program retains this scaffold and adds payload-motion feedback and near-goal braking, obstacle-normal constraints and tangent steering with a remembered bypass side, and a setdown branch with a descent reference, contact memory, and unloading logic. It also adds a gated error integral for persistent offsets and attachment-motion feedback using cable tension and available thrust.

Paired interventions support obstacle handling and transport-velocity feedback in this program. The error integral shows no benefit in this run, while the individual effects of setdown and attachment feedback have not been isolated. The comparison therefore distinguishes the intended control role of each addition from its measured effects.

We added the complete source listings and a side-by-side functional comparison to Appendix F, “Original and Optimized Policy Sources,” in the supplementary material. The comparison includes source-line references, the purpose of each addition, and supporting intervention results.

The authors have addressed my comments in a very thorough manner. I have increased my score.

Thank you so much for the constructive comments and endorsement!

Reviewer_MtyTOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes offline optimization of a shared multi-drone control program using LLM-based code revision, local parameter search, and paired execution feedback. It introduces a 1,000-instance simulation benchmark for cable-suspended payload transport with standard, setdown, and stress tasks. The full method achieves 38.8% overall test success, compared with 34.0% for PPO, although PPO performs better on commanded setdown.

Critical strengths
  1. The paper studies a challenging coordination problem and presents an inspectable shared controller that runs without online LLM inference. The benchmark includes variations in teams, payloads, disturbances, and emergency setdown, making it useful for studying this setting systematically.

  2. The full method improves overall success over the reported handwritten, independent code-generation, and PPO baselines. Ablations show that diagnostic feedback and combining code revision with parameter search improve performance on the main benchmark. These results provide initial support for the optimization procedure.

Critical weaknesses
  1. The comparison does not establish improvement under equivalent computational budgets (Sec. 4.2, 4.3). The overall difference from PPO is 4.8 percentage points, while PPO performs better on commanded setdown. This limits the strength of the claimed advantage over neural-policy learning.

  2. Generalization and robustness remain uncertain given the current evaluation results. The train and test splits share scenario families, and the setdown and stress categories each contain only 16 test cases. The combined-disturbance results are also mixed, while physical deployment and dynamics mismatch remain untested.

Questions for authors
  1. Would the improvement over PPO and the code-generation baselines persist under a matched simulation or computational budget?

  2. How well does the learned program transfer to entirely unseen scenario families or disturbance combinations?

  3. How consistent are the results across independent optimization runs, given the 16-round search and the relatively small differences among methods?

3 comments
AuthorsOpenReview ↗

Response to reviewer MtyT Part 1

We thank the reviewer for the careful review and constructive feedback. All revisions addressing the comments below are marked in blue in the revised manuscript, including the main text and supplementary material.

Q1: Are the simulation budgets comparable, and does the evidence establish an advantage over PPO and code-generation baselines under matched budgets? A1: For the original Table 2 runs, one team physical step advances the entire team by 0.005 s; a complete 20 s episode contains 4,000 steps.

Method (original Table 2)Training-and-selection team physical steps
Handwritten0
One-shot0
Best-of-N32,500,000
PPO (velocity)60,000,000
Full procedure40,000,000

Counts exclude final validation/test evaluation and cache hits. Full includes code evaluation, parameter search, and selection; PPO includes training and checkpoint monitoring. Handwritten has no optimization. One-shot uses the predesignated first generated program without simulation-based selection; its shared 125-episode screen (0.5M steps) is charged to Best-of-N only. Zero denotes optimization simulation, not evaluation or LLM cost.

The budgets are comparable in scale, and Full uses fewer simulation steps than PPO. They are not identical, so these figures do not establish strictly matched-budget or equal-total-compute superiority. PPO remains stronger on commanded setdown in the original comparison.

We added the team-physical-step column and accounting notes for all five original methods to Table 2 in Section 4.3.

Q2: How well do the learned programs transfer to unseen scenario families and disturbance combinations? A2: We evaluated frozen policies on 30 unseen scenario families with 10 cases each (300 cases), without retraining or adaptation. These combine task category, team size, payload shape, attachments, and layout in configurations absent from the audited original data; individual factors need not be new.

We also tested 12 unseen disturbance combinations (eight pairs and four triples), with 10 base scenarios each (120 full-combination cases). Constituent events appeared individually in training, but the selected combinations were absent from the audited training and original evaluation data.

MethodUnseen scenario families (300 cases)Unseen disturbance combinations (120 cases)
Handwritten-I017.6730.83
One-shot16.6739.17
Best-of-N21.5641.67
PPO11.2213.89
Full-I0 (ours)23.6739.72

Entries are mean success percentages over three runs, except for the fixed handwritten reference. Full-I0, PPO, and Best-of-N are additional runs trained under a 40-million-team-physical-step cap. One-shot uses the predesignated first default candidate of each Best-of-N run. Full-I0 exceeds PPO and handwritten control on both panels, supporting transfer to the tested configurations. Performance is comparable to Best-of-N overall, while emergency setdown remains weaker than PPO.

We added this table and a brief description of both transfer experiments to Section 4.3 (Table 9).

Q3: How consistent are the results across independent optimization runs and different initial programs? A3: We evaluated three initializations: I0, the original handwritten controller; I1, geometric slot following with LIFT/CARRY phases; and I2, slot following with vertical cable-tension feedback and neighbor repulsion. Each has three independent runs with the same optimization settings, up to 16 rounds, and a 40-million-team-physical-step cap. Initial code and parameters are fixed within each group; model calls and parameter-search samples are independent. Policies are frozen before evaluation on the same 250 test cases, without test-based selection.

MethodInitial program (%)Final mean (%)Mean gain (pp)
Full-I020.825.1+4.3
Full-I125.657.3+31.7
Full-I229.243.3+14.1

Each row averages three runs; the mean final success across all nine is 41.9%. Outcomes vary with both initialization and optimization run. I1 achieves higher and more consistent gains even though I2 starts with higher success. Initial score alone therefore does not predict optimized performance. All nine runs improve over their own starting programs, but variation and small gains in some runs leave room to improve reliability. Future work will examine initialization design and selection using training or validation feedback, including selection costs in the budget.

We added the initialization-sensitivity table and discussion to Section 4.3, with protocol details in Appendix C.

AuthorsOpenReview ↗

Response to reviewer MtyT Part 2

Q4: Does a larger evaluation strengthen the evidence for emergency setdown and stress transport? A4: We agree that transfer to fresh emergency-setdown and stress-transport cases remains limited under the original criteria. Some requirements are demanding, particularly terminal stable support and sustained airborne operation. To examine this, we augmented training with 256 cases per category, optimized three runs from the original handwritten initialization, and evaluated the frozen policies on a separate test set of 256 fresh cases per category.

TaskOriginal criteria (same policy)Diagnostic conditionDiagnostic result
Emergency setdown31/256 (12.11%)Remove terminal stable-support requirement; retain contact speed <0.5 m/s and all other criteria112/256 (43.75%)
Stress transport64/256 (25.00%)Condition on episodes meeting the airborne requirement; retain all other criteria64/154 (41.56%)

With a targeted relaxation of the terminal-support requirement, setdown performance becomes more reasonable. Stress transport also shows a more reasonable conditional success rate among episodes meeting the airborne requirement. Each row reports the best observed diagnostic result across three runs, selected after evaluation; the rows use different policies. For stress transport, 41.56% is conditional on 154 retained episodes, while full-panel success remains 25.00%. These diagnostics rescore existing trajectories; they identify terminal support and sustained airborne operation as improvement priorities rather than demonstrate improved full-criteria success.

We added the evaluation and diagnostic table to the supplementary subsection “Expanded Emergency Setdown and Stress Transport,” clarifying both the limited transfer and the effects of the evaluation conditions.

Q5: How robust are the policies to dynamics and observation mismatch, and has physical deployment been validated? A5: We evaluated our frozen policies with perturbed actual dynamics or high-level observations, without retraining or supplying corrected parameters. Entries are mean success percentages over three independent runs from the original handwritten initialization, with 60 episodes per run and condition on matched scenarios. A / B corresponds to the two settings in each row. Noise levels are Gaussian standard deviations; joint conditions combine physical and observation offsets.

FactorSettings A / BResult (%) A / B
Nominal—16.1
Payload mass−10% / +10%15.0 / 15.0
Motor capability−10% / +10%15.0 / 16.7
Cable stiffness−25% / +25%11.7 / 15.0
Cable damping−25% / +25%15.0 / 11.7
Position noise0.01 / 0.02 m17.2 / 15.0
Velocity noise0.02 / 0.05 m/s17.8 / 13.9
Observation delay50 / 100 ms12.8 / 11.7
Frame loss5% / 10%15.0 / 15.0
Joint mismatchLight / Moderate12.8 / 9.4

Our policies are less sensitive to mass and motor-capability offsets within the tested ranges, while cable parameters, observation delay, and combined mismatch cause larger degradation. This identifies priorities for improving robustness; physical hardware performance remains unvalidated. We added these results and the protocol to the supplementary subsection “Sensitivity to Dynamics and Observation Mismatch.”

Thank you for your detailed response and additional experiments. The response addresses my questions. However, some concerns remain regarding the limited advantage over Best-of-N, sensitivity to initialization, and low success rates on certain tasks. I appreciate the additional analyses and will maintain my original rating.

Reviewer_LTzKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies programmatic policy optimization for cooperative transport of a cable-suspended payload by multiple drones. A pretrained LLM uses execution feedback to revise a shared control program, alternating code revision with local parameter search and paired selection. The frozen program executes above a fixed flight controller without online LLM inference. The authors construct a synthetic simulation benchmark comprising 1,000 task instances and report higher overall task success than handwritten control and PPO under the reported training budgets, supported by feedback and optimization-stage ablations and mechanism removals.

Critical strengths
  1. Cooperative aerial transport requires coordinating formation, obstacle avoidance, and payload stabilization under physically coupled dynamics. This provides a meaningful setting for studying LLM-based program optimization.
  2. The paper formulates policy optimization using a pretrained language model as offline improvement of a readable, shared control program. Cooperative behavior is learned through code revision and parameter search without training an additional task-specific neural control policy.
  3. A structured benchmark covering diverse operating conditions. The benchmark varies team size, payload geometry, attachment arrangements, obstacles, and disturbances across standard transport, commanded setdown, and stress tasks.
  4. Policy comparisons, optimization ablations, and interventions on the frozen controller provide initial evidence of improvement and help identify beneficial and potentially redundant components of the generated program.
Critical weaknesses
  1. Code revision and local parameter search require repeated simulator interaction. The comparisons do not establish whether the full procedure offers advantages over PPO in task success and simulation efficiency under comparable simulation budgets.

  2. The framework offers limited methodological adaptation of execution-guided code revision and local parameter search to cable-coupled transport. Mechanism removals on one frozen policy provide limited generalizable insight into when and why the LLM-based optimization procedure succeeds or fails than the other training-based algorithms.

  3. The scenario distribution, physical settings, and success thresholds need stronger physical or empirical justification. Static screening also leaves dynamic feasibility unresolved, limiting confidence that the benchmark provides representative tasks and meaningful measures of cooperative transport performance.

  4. The demonstrated application value is not sufficiently compelling. The experiments do not clearly establish which practically important difficulties the method resolves or how the reported gains address concrete requirements of cooperative aerial transport.

  5. The full procedure uses simulation experience for code revision and local parameter search. A plausible alternative is a hybrid approach in which the LLM also coordinates the training of a task-specific neural control policy using the collected training trajectories. This could potentially improve the reuse of simulation experience, but the paper does not explore this trade-off. Moreover, the PPO comparison uses different computational budgets, leaving unclear how the chosen procedure compares with neural policy training or a hybrid approach in task success and simulation efficiency.

Questions for authors

Please refer to the previous Weaknesses.

7 comments
AuthorsOpenReview ↗

Response to reviewer LTzK Part 1

We thank the reviewer for the careful review and constructive feedback. All revisions addressing the comments below are marked in blue in the revised manuscript, including the main text and supplementary material.

Q1: Are the simulation budgets comparable, and does the full procedure improve success and simulation efficiency over PPO? A1: We have added the training-and-selection team physical steps for the original Table 2 runs to the paper. One step advances the whole team by 0.005 s; a complete 20 s episode has 4,000 steps.

Method (original Table 2)Training-and-selection team physical steps
Handwritten0
One-shot0
Best-of-N32,500,000
PPO (velocity)60,000,000
Full procedure40,000,000

Counts exclude final validation/test evaluation and cache hits. Full includes code evaluation, parameter search, and selection; PPO includes training and monitoring. Handwritten has no optimization. One-shot uses the first generated program without simulation-based selection; its shared screen (0.5M steps) is charged to Best-of-N only. Zero denotes optimization simulation, not evaluation or LLM cost.

The budgets are comparable in scale: Full uses fewer simulation steps than PPO, although this is not a strictly matched-budget or equal-total-compute comparison.

We added the team-physical-step column and accounting notes for all five original methods to Table 2 in Section 4.3.

Q2: What does the framework contribute to cable-coupled transport, and what evidence beyond a single frozen policy explains its successes and failures? A2: Our objective is policy performance: code represents the policy, and execution feedback identifies behavioral problems. We study whether a general-purpose pretrained LLM can improve cable-coupled multi-drone transport through policy revision and parameter search.

In original Table 3, fixing the code causes the largest loss; reducing diagnostic feedback also lowers success; removing numerical search has a smaller effect. This supports structural revision, diagnostic feedback, and complementary parameter refinement under the reported budgets.

We added cross-program interventions to supplementary Appendix D (Table 11), with a brief summary in Section 4.3: three independent runs per initialization (I0: original script; I1: geometric slot following; I2: added cable-tension feedback and neighbor repulsion). All use the same 250 test cases, fixed task criteria and low-level safety filter, without retraining or retuning. Values are success (%) and changes from each full program (pp). N/A denotes an absent mechanism.

Initialization–runFull programNo obstacle-normal guardNo tangent steeringNeither guard nor steeringNo transport velocity feedbackNo error integral
I0–132.432.0 (−0.4)30.8 (−1.6)25.6 (−6.8)22.8 (−9.6)N/A
I0–221.6N/AN/AN/A19.6 (−2.0)N/A
I0–321.2N/AN/AN/AN/AN/A
I1–156.039.6 (−16.4)46.0 (−10.0)32.0 (−24.0)49.2 (−6.8)56.4 (+0.4)
I1–259.646.4 (−13.2)48.4 (−11.2)30.0 (−29.6)52.4 (−7.2)60.8 (+1.2)
I1–356.439.6 (−16.8)46.0 (−10.4)30.0 (−26.4)51.6 (−4.8)N/A
I2–132.4N/AN/AN/A31.6 (−0.8)N/A
I2–230.4N/AN/AN/AN/AN/A
I2–367.239.2 (−28.0)65.2 (−2.0)35.6 (−31.6)62.0 (−5.2)57.2 (−10.0)

Jointly removing obstacle-normal guarding and tangent steering causes larger drops than either removal alone in every applicable program, with more obstacle collisions. Removing transport velocity feedback also lowers success across applicable programs. Integral removal has little effect in some programs but causes a substantial decline in another. These post-hoc interventions support recurring, complementary roles for obstacle handling and motion feedback, while showing that integral feedback depends on the surrounding policy.

The full protocol, Table 11, and analysis are in supplementary Appendix D; Section 4.3 provides a concise summary and cross-reference.

AuthorsOpenReview ↗

Response to reviewer LTzK Part 2

Q3: What justifies the scenario distribution, physical settings, and success thresholds, and does static screening establish dynamic feasibility? A3: We agree. We have revised the main-text benchmark description and added this table to the supplementary material, clarifying the evidence and rationale for previously specified settings.

Settings combine model parameters, literature or engineering precedents, and empirical choices. Empirical values depend on the payload and task and can be adjusted; there is no universal fixed answer. References support models, numerical precedents, or physical scales, not apparatus-specific calibration. All methods use the same frozen criteria.

ItemBenchmark settingEvidence
Vehicle mass / motor noise0.028 kg / 0.05QuadSwarm model
Cable length1 mSun et al., 2025
Elastic cable / dampingSpring-damper / 6 N·s/mKotaru et al., 2017; Ru et al., 2024
Delivery dwell time3 sWard et al., 2025
Stable-support linear speed≤0.1 m/sXiong et al., 2025
Terminal stable support≥1 sPX4 v1.15
Setdown horizontal radius1 mKownacki et al., 2023
Delivery goal tolerance≤0.3 mRen et al., 2026
Downward contact-point speed<0.5 m/sEmpirical
Thrust-to-weight ratio3.0Pretorius and Boje, 2014
Stable-support angular speed≤0.2 rad/sEmpirical
Static capacity screenGeneration ≤68%; acceptance ≤70%; stress >70% and ≤95%Empirical
Cable stiffness200 N/mEmpirical
Ground contact300 N/m; 6 N·s/mEmpirical
Team / episode / scene2–8 drones; 20 s; 14 × 14 × 6 mEmpirical
Payload / disturbances / allocationMass 0.02912–0.38080 kg; 872 / 64 / 64 task categoriesEmpirical
Neighbor / wall clearance0.6 / 0.4 mEmpirical
Cable breakingTension-triggered breaks off; prescribed cuts onEmpirical

Scenario proportions define synthetic coverage, not deployment frequencies. Static screening checks initial overlap and phase-wise thrust capacity, not dynamic reachability within the episode horizon. Successful executions demonstrate feasibility for those instances under the simulator's criteria; failures alone do not distinguish controller limitations from task infeasibility.

We clarified the setting sources and task-dependent choices in Section 4.1, and added the consolidated evidence table in Appendix G.1 (Table 14).

Q4: Which practical challenges in cooperative transport does the method address, and how do the reported gains meet concrete task requirements? A4: Our motivation is better task completion and easier policy diagnosis. Success requires coordinated obstacle avoidance, formation, payload motion, and delivery or setdown under cable coupling. Higher success means more tasks meet these combined demands; A2 connects gains to obstacle responses and payload-velocity feedback.

Readable programs expose formation rules, braking conditions, and phase transitions. Designers can connect conflicting avoidance actions or insufficient braking to specific control logic and test targeted revisions in simulation.

We revised Section 1 to explain why task success measures the combined transport requirements and how inspectable policies support targeted corrections.

AuthorsOpenReview ↗

Response to reviewer LTzK Part 3

Q5: Could program-generated trajectories support neural-policy training, and how would this hybrid approach compare under comparable budgets? A5: We tested trajectory reuse through behavioral cloning from an intermediate program, followed by PPO. The reported run achieved 28.4% success (71/250), versus 5.6% for random-initialized PPO with the corresponding run label. Both have a 40M-team-physical-step ceiling. The reuse run consumes 38M steps: 14M for program optimization and demonstration collection, plus 24M for PPO and monitoring; the reference consumes 40M. Teacher generation is included. This reference differs from the original Table 2 PPO run at 60M steps. The pipelines also differ in sampling, so the gain cannot be attributed solely to initialization. Complete three-run results and the LLM-curriculum variant are in the supplementary material.

The hybrid outperforms the original handwritten controller but remains below the optimized program. It uses an intermediate program as its teacher; stronger teachers and better distillation may help, although the causes of the gap remain unverified. Future work will examine these factors and budget allocation between program optimization and neural fine-tuning. We added the hybrid result to Table 3 in Section 4.3, with the full three-run results and budget protocol in Appendix B.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.