- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper considers a linear time-invariant (LTI) dynamical system and assumes that a library of linear state feedback controller for this LTI system is available. The paper then aims at learning the distribution for switching among the available feedback gain to minimize a typical discounted quadratic cost. The authors propose a policy gradient framework to find learn the switching distribution in two regimes: known and unknown dynamics.
While the paper is well-written and theoretical my recommendation is reject. The problem in this paper is not relevant. To be more specific, at high level, this paper considers the simplest linear time varying system. Why would one study switching controllers for this system? There is no motivation for this. It is well known that a single controller is enough for controlling a linear system. The paper could have some motivation if at least linear parametric systems were considered but it is not the case. Again compare the simplest system of linear time invariant noise-free dynamical system with more general dynamical systems (nonlinear, uncertain, domain randomized, etc..) currently in the literature. There is no motivation for this setup, there is no gap and the theoretical results bring no value.
Critical strengths
Paper is well-written and easy to follow. One can start reading the paper and follow it.
Critical weaknesses
The setup is too simple: linear time-invariant dynamical system, without any noise and uncertainty. There is no research gap in the paper rather what is sold as research gap is a made up problem that brings no value to the literature.
The problem is not scientifically sound: there is no need to have switching for the linear time-invariant systems. Why would someone consider switching controllers for a linear system? There is no need for that. Design a single optimal controller and never switch! This can be the optimal switching. Then there is no need for all the theoretical developments here and caring about stability and this and that.
Questions for authors
The authors give the controller selection probability in (2). And they say that (2) is used to sample the switching signal and (2) is the distribution over controllers. You cannot use a distribution over controllers when controlling. What you are trying to say is that you use (2) as soft max.
There is a half-sentence after Lemma 1. Please take care of that.
After Lemma 2, the authors discuss regret with respect to the optimal switching. There is no optimal switching. The optimal switching is “no-switching”. I understand that you mean optimal with respect to a given set of controllers but I would argue that why would one consider such a set? You have a time invariant linear system without any uncertainty. You have quadratic costs. So just design the optimal controller for this system from theories developed almost a century ago.
Response to the Reviewer's comments
We thank the reviewer for the thoughtful comments regarding the motivation for controller switching and its relationship to classical LQR. It appears there may be a misunderstanding: the reviewer notes that because the problem is an LQR formulation, a new controller can be designed directly by solving the Riccati equation, and thus interprets our work as an attempt to provide an alternative solution to this well-established result. However, this is not the case.
Motivation of the Work
The reviewer is absolutely correct that for an LTI system with LQR objective, the optimal gain can be found by solving the Algebraic Riccati Equation. In this setting, the objective is not to synthesize another feedback gain, but to determine how the available controllers should be selected to improve performance under the new objective.
For example, autonomous UAVs and NASA aircrafts may use onboard autopilots containing prevalidated flight-control laws. Adding new controllers or modifying the gains of the approved ones may require additional review. Similarly, aircraft electronic engine control systems (often known as FADEC) may include previously approved selectable control options, while introducing new control software may require further validation and approval. These examples motivate the use of existing controller libraries, where performance must be improved through supervisory controller selection without modifying the individual feedback gains. To the best of our knowledge our work appears to be the first one to address this challenge. We agree that the scope is limited due to the LTI and LQR nature of the problem. Nonetheless, the proposed method can still be adapted to nonlinear systems by leveraging iterative LQR (iLQR) framework.
We understand that the proposed idea goes against the traditional thinking for an LQR problem. We hope the above examples clarifies the need for the proposed study. We don't claim that our method to be implemented for every LQR systems, but we respectfully emphasize that certain systems would benefit from the from the results presented in this paper, and such systems are often most sensitive and safety critical.