- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes MILD-MPC, a multi-agent imitation learning framework based on distributed model predictive control. The framework learns a policy from demonstrations provided by a distributed MPC expert, which also serves as a backup policy during deployment. By combining distributed MPC with imitation learning, it reduces online solve times and uses a safety verifier to assess the validity of predicted trajectories, providing an approach to accelerating real-time multi-agent control. The framework’s effectiveness is evaluated through comparative simulations and hardware experiments.
Critical strengths
-
The proposed method offers an effective approach to accelerating real-time multi-agent control. Comparisons with the evaluated baselines demonstrate substantial reductions in online computation time, highlighting its practical potential.
-
The paper evaluates the proposed framework through diverse simulation scenarios and real-robot experiments, providing evidence of its computational efficiency, transferability to larger teams, and robustness under the tested agent-failure scenarios.
Critical weaknesses
-
In Fig. 2, the proposed MPD-DDP and learning augmented methods are compared against a centralized SQP baseline. However, distributed methods are generally intended to improve computational efficiency over centralized approaches, so this comparison does not sufficiently establish the proposed framework’s computational advantages over other representative distributed methods. The evaluation should include comparisons with existing distributed approaches, such as ADMM-DDP, which is discussed repeatedly in the paper.
-
In the workflow shown in Fig. 1, trajectory validation is applied to the candidate trajectories generated by the learned policy and refined by the safety filter. However, no further validation is shown for the output of the MPD-DDP backup policy.
Questions for authors
-
Fig. 2 shows that MPD-DDP has a higher average solve time with 12 agents than with 15 agents. What accounts for this non-monotonic trend? The paper does not provide an explanation.
-
The maximum numbers of ADMM iterations for MPD-DDP are set to 5, 30, 20, 45, and 60 for the 8, 12, 15, 20, and 30 agent settings, respectively. How were these limits selected? In particular, why is the iteration limit higher for 12 agents than for 15 agents? The rationale for varying these limits is not sufficiently explained.
-
Could the average solve time axis in Fig. 2 use a logarithmic scale? The smaller values are difficult to distinguish on the current scale, which limits the readability of the comparison.
Update: After reading the responses, I decided to keep the original score.
5 comments
Author response (2/2)
2. (Weakness 2): Trajectory validation is applied to the candidate trajectories generated by the learned policy and refined by the safety filter. However, no further validation is shown for the output of the MPD-DDP backup policy.
Response:
Indeed, it is possible that MPD-DDP returns an infeasible solution due to the time constraint or when the optimization problem in the intermediate step is infeasible. Checking the feasibility of the final trajectory is straightforward, and it boils down to the solution of the final backup policy or specification of the emergency behavior, e.g., emergency stop (e-stop). In our context, we chose to use MPD-DDP as a final backup policy, which handles constraints by construction. In terms of emergency behavior specification, we tested the quadrotor software-in-the-loop (SITL) experiment with e-stop enabled and the Robotarium experiment under the platform-wide e-stop guard. We have not included these engineering techniques in our methodology as they are out of scope for this work.
3. (Question 1): Fig. 2 shows that MPD-DDP has a higher average solve time with 12 agents than with 15 agents. What accounts for this non-monotonic trend? The paper does not provide an explanation. The maximum numbers of ADMM iterations for MPD-DDP are set to 5, 30, 20, 45, and 60 for the 8, 12, 15, 20, and 30 agent settings, respectively. How were these limits selected? In particular, why is the iteration limit higher for 12 agents than for 15 agents? The rationale for varying these limits is not sufficiently explained.
Response:
Thanks for sharing the astute observations! We wish to make a point that the non-monotonic trends of solve times and ADMM iteration count with respect to the number of agents are due to the nonconvexity of the problem. Fig. 2 task involves two moving obstacles. When these time-varying per-agent state constraints () are combined with the inter-agent state constraints () in a receding horizon scheme, then the gap between the DDP trajectories () and the safe trajectories () each distributed optimal control problem at each replanning step is not necessarily monotonic with respect to . Since ADMM enforces the consensus between the DDP trajectories and the safe trajectories, the non-monotonic gap between these trajectories implies that the ADMM iteration count is not necessarily monotonic, i.e., a larger gap needs more ADMM iterations to close. Since the solve time of MPD-DDP mostly depends on the ADMM iteration count, the non-monotonic ADMM iteration count with respect to implies the non-monotonicity of the MPD-DDP solve time.
The median values of the primal residual () at ADMM iteration 1 across all replanning steps for in Fig. 2 are , which show the non-monotonic gap between the DDP trajectories and the safe trajectories with respect to . This result also aligns with the trends of the maximum number of ADMM iterations and the solve times s of MPD-DDP with respect to .
4. (Question 2): Could the average solve time axis in Fig. 2 use a logarithmic scale? The smaller values are difficult to distinguish on the current scale, which limits the readability of the comparison.
Response:
Thanks for pointing out the readability of Fig. 2! We have added another version of Fig. 2 in which the y-axis is in a log scale.
Author response
We appreciate the reviewer for the constructive feedback. We updated our manuscript to address the reviewer's points.
Author response (1/n)
We thank the reviewer for acknowledging the practical potential of the proposed framework. We hope that our revised manuscript and our responses address the reviewer's questions and concerns.
Summary
We have clarified the following points.
Our Response
Response:
We appreciate the reviewer's point that the comparison with existing distributed approaches, e.g., ADMM-DDP, should be included. Indeed, we agree that distributed methods generally improve the computational cost over centralized methods. In this context, we have compared our method with ADMM-DDP (labelled as Trajectory optimization) in Appendix I: ``What is the effect of dynamic neighborhood set reconstruction in time-varying problem structure?", where ADMM-DDP fails to complete the task safely in 8 and 12-agent settings due to the lack of dynamic neighborhood reconstruction (ADMM-DDP assumes fixed neighborhood across time). Furthermore, we wish to point that we have included the comparison of MILD-MPC with our proposed distributed MPC baseline, MPD-DDP, in Fig. 2.