- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies excitation collapse in in-context control trained through privileged oracle imitation. The authors show that, under certain symmetry assumptions, MSE-optimal imitation can lead to zero control inputs and prevent further system identification. They propose symmetry-broken imitation (SBI), which trains a controller to commit to a system hypothesis and revise it based on observations. Experiments on linear systems and a nonlinear pendulum show improvements over standard imitation.
Critical strengths
Strengths
-
The paper makes an interesting connection between symmetry, imitation learning, and the lack of excitation in adaptive control. The theoretical analysis provides useful insights into this failure mode.
-
The empirical results are encouraging, particularly for unknown actuator polarity. I also appreciate the comparisons with classical adaptive control methods and the discussion of cases where SBI does not work well.
Critical weaknesses
Weaknesses
-
The paper focuses on rather specific symmetry structures, such as unknown actuator polarity, permutations, and rotations. While these settings allow for a clean theoretical analysis, I am not fully convinced how common or important this particular failure mode is in practical in-context control. Moreover, SBI requires knowledge of the symmetry group and privileged system parameters during training, which further limits its applicability.
-
The proposed method requires constructing a symmetry-specific teacher, maintaining multiple hypotheses, and performing additional training with DAgger. However, the underlying issue appears to be a lack of excitation? One possible simpler alternative might be to add a small probing input at the beginning of deployment while keeping the original imitation-trained controller. Since the training data already contain excitation, the controller may be able to use the resulting history to identify the system and adapt its actions. Of course, this may not always work, especially for unstable systems, but it seems like a very simple and natural baseline worth considering. The paper includes some probing and closed-loop fine-tuning experiments, which is appreciated, but a more direct comparison would help clarify the advantages of SBI. In particular, ordinary imitation with closed-loop fine-tuning already achieves 0–2% instability in the simple linear setting.
-
Theorem 1 establishes equivariance almost everywhere under the training distribution, while Theorem 2 claims exact freezing along closed-loop trajectories. I wonder whether additional assumptions are needed to ensure that the equivariance property holds along these trajectories. The scalability results are also concerning. With four and six unknown actuator polarities, SBI+DAgger has 74% and 93% instability, despite the teacher achieving zero instability. More importantly, even on all-positive plants, where the teacher's initial polarity commitment is correct, the student still has 68% and 88% instability. This suggests that the student struggles with the underlying in-context identification and control task, beyond the symmetry-breaking issue addressed by SBI.
Questions for authors
-
Could the authors comment on how often the assumed symmetry structures arise in practical in-context control problems?
-
Would it be possible to compare SBI with ordinary imitation plus simple initial probing at deployment? This seems like a straightforward baseline that could help demonstrate the benefits of the proposed approach.
-
Does Theorem 2 require additional assumptions to extend the almost-everywhere result of Theorem 1 to closed-loop trajectories?