acceptodds
Under review as a conference paper at ICLR 2027

Excitation Collapse in In-Context Control: Symmetric Freezing and Symmetry-Broken Imitation

Abstract

In-context controllers are sequence models pretrained on families of dynamical systems so that they can control an unseen system from its interaction history. They are usually trained to imitate an oracle that knows the system, or to regress the oracle's privileged parameters. It is well known in adaptive control that controllers which hedge against parameter uncertainty can turn off: they stop exciting the system and so stop learning about it. We show that imitation pretraining produces this failure exactly whenever the family has a symmetry that the history cannot resolve before the controller acts; unknown actuator polarity is the standard example. The imitation optimum is equivariant, so from a cold start its closed loop stays on the symmetry-fixed actions for all time, which we call symmetric freezing. Because the frozen controller minimises the imitation loss over all history-measurable controllers, changing the architecture or adding capacity and data does not avoid it, and better optimisation moves the learner closer to it. The theory predicts degenerate latents in privileged latent distillation, a default direction of predictable sign under asymmetric priors, and exact freezing of equivariant architectures. We confirm each prediction for discrete and continuous symmetry groups and introduce two black-box diagnostics. Since the failure comes from the training target, we change the target. Symmetry-broken imitation (SBI) trains an unmodified network on a teacher that commits to one orbit element and revises this choice using exact privileged likelihoods. The teacher is provably consistent, and one round of on-policy relabelling transfers its revisions to the student. With one plain transformer and no knowledge of the symmetry at deployment, SBI removes all instability in a linear family (two seeds, 180 paired runs each), matching an explicit-copy quotient controller. In RMA-style latent distillation on a nonlinear pendulum with unknown motor polarity, changing only the regression target and adding one round of on-policy relabelling reduces falls from 32–43% to 10–13% across two seeds (p < 10^-12 in each), on par with classical multiple-model control. In a six-state family whose prior no model bank can cover, SBI beats multiple-model control with 1024 models on paired runs across three training seeds (p = 8 × 10^-12 pooled).

Primary area:
applications to robotics, autonomy, planning
Keywords:
in-context learning, in-context control, imitation learning, excitation collapse, symmetry breaking, equivariance, adaptive control, dual control, privileged teacher-student distillation, learning-based control

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_pRADOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies excitation collapse in in-context control trained through privileged oracle imitation. The authors show that, under certain symmetry assumptions, MSE-optimal imitation can lead to zero control inputs and prevent further system identification. They propose symmetry-broken imitation (SBI), which trains a controller to commit to a system hypothesis and revise it based on observations. Experiments on linear systems and a nonlinear pendulum show improvements over standard imitation.

Critical strengths

Strengths

  1. The paper makes an interesting connection between symmetry, imitation learning, and the lack of excitation in adaptive control. The theoretical analysis provides useful insights into this failure mode.

  2. The empirical results are encouraging, particularly for unknown actuator polarity. I also appreciate the comparisons with classical adaptive control methods and the discussion of cases where SBI does not work well.

Critical weaknesses

Weaknesses

  1. The paper focuses on rather specific symmetry structures, such as unknown actuator polarity, permutations, and rotations. While these settings allow for a clean theoretical analysis, I am not fully convinced how common or important this particular failure mode is in practical in-context control. Moreover, SBI requires knowledge of the symmetry group and privileged system parameters during training, which further limits its applicability.

  2. The proposed method requires constructing a symmetry-specific teacher, maintaining multiple hypotheses, and performing additional training with DAgger. However, the underlying issue appears to be a lack of excitation? One possible simpler alternative might be to add a small probing input at the beginning of deployment while keeping the original imitation-trained controller. Since the training data already contain excitation, the controller may be able to use the resulting history to identify the system and adapt its actions. Of course, this may not always work, especially for unstable systems, but it seems like a very simple and natural baseline worth considering. The paper includes some probing and closed-loop fine-tuning experiments, which is appreciated, but a more direct comparison would help clarify the advantages of SBI. In particular, ordinary imitation with closed-loop fine-tuning already achieves 0–2% instability in the simple linear setting.

  3. Theorem 1 establishes equivariance almost everywhere under the training distribution, while Theorem 2 claims exact freezing along closed-loop trajectories. I wonder whether additional assumptions are needed to ensure that the equivariance property holds along these trajectories. The scalability results are also concerning. With four and six unknown actuator polarities, SBI+DAgger has 74% and 93% instability, despite the teacher achieving zero instability. More importantly, even on all-positive plants, where the teacher's initial polarity commitment is correct, the student still has 68% and 88% instability. This suggests that the student struggles with the underlying in-context identification and control task, beyond the symmetry-breaking issue addressed by SBI.

Questions for authors
  1. Could the authors comment on how often the assumed symmetry structures arise in practical in-context control problems?

  2. Would it be possible to compare SBI with ordinary imitation plus simple initial probing at deployment? This seems like a straightforward baseline that could help demonstrate the benefits of the proposed approach.

  3. Does Theorem 2 require additional assumptions to extend the almost-everywhere result of Theorem 1 to closed-loop trajectories?

Reviewer_BMNVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the training of in-context controllers. It first reveals that training by imitating an oracle which knows the system parameter θ\theta will make the controller freeze if the system family has a specific symmetry. Then it proposes a fix (SBI). Here the teacher knows the system up to the symmetry. The teacher commits to one hypothesis at the start and switches only when exact likelihoods favour another hypothesis. The student learns these targets, which no longer require averaging opposite actions. The authors show that SBI outperforms the imitation in experiments on a two-state linear family and a nonlinear pendulum.

Critical strengths
  1. This paper studies a realistic and interesting phenomenon in control, namely the "turning off/freezing" of the controllers and explains that its root is the training objective (imitation). This papers offers theoretical explanations to this "turning-off" problem.

  2. This paper offers a counterintuitive but interesting insight about equivariance. People often believe that having symmetry in a neural network is good. However, this paper shows that an unknown system with symmetry can result in failure. Bridging control and symmetry-breaking in equivariant networks is novel.

Critical weaknesses
  1. The presentation of the paper should be improved. Several quantities and variables, which are much involved in the assumptions and the theoretical results are not clearly defined. Details can be found in Questions For Authors.

  2. There are mistakes in the core theoretical contributions of the paper (the lower bound in Proposition 4 and Proposition 5 (c)). Details can be found in Questions For Authors.

Questions for authors
  1. I assume θ\theta is the system's parameter. Does θ∼p(θ)\theta \sim p(\theta) in Section 2 mean θ\theta follows a distribution pp, which is a function of θ\theta or pp is merely a distribution independent of θ\theta?

  2. In Section 3, does "σ\sigma is trivial" mean "do not flip the sign"?

  3. Does "cold start" mean the controller is only given only y0y_0? Please explain what is "warm start" explicitly. My current guess is the evaluated controller is given the controls uu and observations yy generated by an oracle controller up to a given number of time steps.

  4. What is "information about g" in Proposition 4? Is it the KL divergence between the two hypotheses’ one-step predictive distributions?

  5. The lower bound in Proposition 4 is wrong. Please check it. The correct predictive variance StS_t contains 3 parts, namely the sensor/measurement noise s2s^2, the process/transition noise (non-negative) and the remaining state uncertainty (non-negative). The correct quantity should be 2(CBuˉ)2/St22(CB\bar{u})^2/S_t^2, which is not larger than 2(CBuˉ)2/s22(CB\bar{u})^2/s^2, since St2≥s2≥0S_t^2 \geq s^2 \geq 0. In Proposition 4 it is stated as "at least". The lower bound does not hold.

  6. The mistake of Proposition 4 goes into Proposition 5 (c) and makes it incorrect.

Reviewer_ox22OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper identifies symmetric freezing, a fundamental failure mode in imitation-pretrained in-context controllers where unresolved system symmetries cause the controller to stop exploration and learning, a problem that cannot be fixed simply by increasing model capacity, data, or optimization. It proposes symmetry-broken imitation (SBI), which modifies the training target to break the symmetry and substantially improves adaptive control performance in both linear and nonlinear systems.

Critical strengths
  1. The theory seems to be solid.
Critical weaknesses
  1. I am familiar with imitation learning, symmetry, and classical adaptive control individually, but I found the connection among these three concepts somewhat difficult to follow. It would be helpful if the authors could provide a clearer high-level explanation of what brings these ideas together in the present setting. More importantly, the paper could better motivate why this relatively specific problem is of broader interest to the ML community.

  2. The experiments consider relatively simple linear and nonlinear dynamical systems. Could the authors clarify what specific symmetries (θ=gθˉ\theta=g\bar{\theta}) arise in these models and how they relate to the theoretical symmetry assumptions?

  3. The abstract is quite long and dense, making the main message difficult to follow. Also, a clear problem statement is missing.

  4. The experimental systems appear to be relatively low-dimensional. How well does the proposed method scale to high-dimensional dynamical systems?

Questions for authors

see the weakness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.