acceptodds
Under review as a conference paper at ICLR 2027

Taming the Edge of Chaos: The Dynamical Origin of Prompt Robustness in LLMs

Abstract

Large Language Models (LLMs) exhibit fragility to microscopic, semantic-preserving prompt perturbations. To uncover the physical origin of this vulnerability and its mitigation, we propose a paradigm shift: modeling the LLM residual stream as a discrete-time non-linear dynamical system. Within this framework, we mathematically unify a disparate ”zoo” of robust alignment strategies, ranging from Sharpness-Aware Minimization (SAM) and Muon to noise injection and Weighted Instruction Tuning (WIT). We prove that despite their distinct engineering heuristics, these optimizers converge on a singular physical mechanism: they act as dynamical dampers that drive the network's Quasi-Lyapunov Exponent into a strictly dissipative regime (). Furthermore, we establish the first causal link between continuous-time neural flows and the geometric phenomenon of Progressive Neural Collapse (NC). Rather than adopting the Unconstrained Feature Model (UFM) as an axiomatic assumption, we rigorously prove that localized dynamical contraction () physically ”unlocks” the UFM. This continuous contraction acts as a physical engine that exponentially dampens intra-class variance (), sculpting the terminal representations into a highly symmetric Simplex Equiangular Tight Frame (). During inference, these geometric vertices act as asymptotically stable attractor basins (). Extensive empirical evaluations confirm that this topological defense intrinsically absorbs adversarial lexical noise and structural jailbreaks, achieving state-of-the-art zero-shot prompt robustness without sacrificing foundational model expressivity.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language models, prompt engineering, chaos theory, dynamics

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_21szOpenReview ↗
Rating
3(+1)
Confidence
3(+1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper explains LLM prompt robustness through layerwise dynamics. It interprets several fine-tuning methods as dampers that reduce a quasi-Lyapunov exponent. It further claims that contraction produces neural collapse, ETF geometry, and robust decoding.

Critical strengths

Sections 2–5 connect prompt robustness, dynamics, and representation geometry through a worthwhile research question. Table 1 reports complementary dynamical, geometric, and attack metrics across three models. Appendices D.3–D.4 examine contraction boundaries and capability preservation, helping characterize the empirical findings.

Critical weaknesses
  1. Theorem 0 and Appendix B.1, Eq. (16), rely on unproved singular-direction alignment: a positive quasi-exponent does not guarantee exploding gradients. Alternating diag(2,1/2) and diag(1/2,2) gives a quasi-exponent of log 2, but every two-layer Jacobian product is the identity.
  2. Theorem 5 and Appendix B.7 leave centroid surjectivity and the replacement of parameter regularization unproved, undermining the claimed UFM equivalence.
Questions for authors
  1. Which explicit assumptions of Theorem 0 exclude the alternating-Jacobian counterexample?
  2. Does Appendix B.1 provide a verifiable quantitative lower bound supporting the alignment asserted in Eq. (16)?
  3. How does Appendix B.7 establish centroid realizability and equivalence between parameter and centroid regularization?
Reviewer_VJ1DOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper models prompt robustness in large language models (LLMs) as a nonlinear dynamical systems problem. It proposes that various fine-tuning methods, particularly SAM + Muon, can induce negative quasi-Lyapunov exponents, promote neural collapse, and establish stable representation geometries, thereby reducing sensitivity to prompt perturbations and jailbreak attacks. Experiments on Llama and Qwen models show substantial reductions in attack success rates while preserving foundational capabilities.

Critical strengths
  1. An insightful theoretical framework. The paper attempts to unify several robust fine-tuning methods and connect internal model dynamics, representation geometry, and prompt robustness, offering a novel perspective on the origins of robustness.

  2. A relatively comprehensive empirical evaluation. The authors evaluate multiple model scales and various prompt perturbation and attack scenarios, examining metrics such as attack success rate, representation variance, and foundational task performance.

Critical weaknesses
  1. Strong assumptions underlying the theoretical derivations. Several key conclusions rely on assumptions concerning Jacobian contraction, non-degenerate between-class covariance, and feature mapping properties, whose applicability to real-world LLMs requires further validation.

  2. Insufficient causal evidence. Although the experiments show a clear association between dynamical contraction and improved robustness, more rigorous intervention studies are needed to establish whether negative quasi-Lyapunov exponents directly cause robustness improvements.

Questions for authors
  1. Strengthen theoretical validation. The authors should explicitly state the applicability conditions of each theorem and use more rigorous derivations or counterexample analyses to verify whether the key assumptions hold in practical Transformer architectures.

  2. Expand causal and generalization experiments. The authors should conduct controlled experiments and ablation studies to investigate the causal relationship between dynamical metrics and robustness, and evaluate the method across more tasks, models, and adaptive attack scenarios.

Reviewer_gcjfOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper models the LLM residual stream as a discrete-time dynamical system. It argues that WIT, SAM, Muon, NEFTune, and FreeLB improve robustness by reducing an empirical Quasi-Lyapunov Exponent. The theory links this contraction to neural collapse and resistance to prompt changes. On Llama-3-8B, SAM+Muon reduces attack success from 38.5% under standard SFT to 4.2%. The appendices add cross-model tests, hard attacks, capability checks, and analyses of where excessive damping hurts performance.

Critical strengths
  • The paper connects optimizer behavior, hidden-state dynamics, and neural collapse in one framework. Appendix B gives explicit assumptions, derivations, and a toy example rather than only a conceptual analogy.
  • The appendices test other architectures, a 70B model, LoRA variants, and hard attacks (Tables 7–11). Capability and diversity checks help assess whether robustness comes from making outputs less useful.
  • The failure analysis is concrete and useful. Figures 11 and 12 show limits under larger perturbations and capability loss from excessive damping.
Critical weaknesses
  • The experiments do not isolate the exponent as the cause of robustness. Table 4 changes training recipes, while Figure 8 shows temporal ordering, so other effects of those recipes remain plausible explanations.
  • Theorem 0 appears to rely on unproved steps in Appendix B.1. Equation 16 assumes alignment of dominant singular directions, and the squeeze-principle argument asserts that SGD settles near zero; later proofs use this boundary condition rather than establish it.
  • The main robustness suite uses five curated intents with 200 variations each (Section 5.1). Table 7 adds hard attacks, but broader generalization beyond this custom prompt setting is still not established.
Questions for authors
  • Could you vary the exponent while keeping the training recipe fixed, or otherwise separate it from optimizer effects? This would help test the claimed causal link to robustness.
  • Section 5.2 gives a lower bound of +0.025 for unaligned bases, but Table 1 includes +0.018, -0.005, and +0.012. How should these statements be reconciled, and how stable are the estimates with more than 15 power iterations?
  • What exact subset does the AdvGLUE-Hard column in Table 7 use, and was PromptRobust also evaluated? Will the release include the exponent and neural-collapse measurement code?
Reviewer_UkjPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper treats the residual stream as the state of a dynamical system, with each layer acting as a single step in the system. The authors propose using the empirical Quasi-Lyapunov Exponent (λemp\lambda_{emp}) of the system to measure the model's robustness to input perturbations, and show that many disparate optimization schemes which improve robustnes share negative λemp\lambda_{emp}, which correspond to a more stable dynamical system. The authors then link negative values of λemp\lambda_{emp} to neural collapse in the final layers of the neural network. They empirically measure λemp\lambda_{emp}, model robustness, and neural collapse across different optimization schemes, including their proposed method combining Muon and SAM, on models at the 1-8B parameter scale. The paper's primary contribution is the theoretical analysis of LLMs as dynamical systems, and the proposed Muon + SAM robust fine-tuning method.

Critical strengths
  • The characterization of model robustness using Quasi-Lyapunov Exponents provides a strong theoretical justification for the effectiveness of varied optimization methods, and could inspire future robust fine-tuning methods.

  • The empirical results show a clear connection between negative values of λemp\lambda_{emp} and robustness measures such as zero-shot Attack Success Rate, aligning with the theoretical claims.

Critical weaknesses
  • The paper is quite dense and can be hard to follow at times; some sections, like 4.3, could benefit from additional explanation.
Questions for authors
  • Given that foundation models are trained with λemp≈0\lambda_{emp} \approx 0, it is not clear to me why an "infinitesimal adversarial perturbation" would necessarily have significant impact when propagated through the network, when to my understanding that sort of blow up is associated with systems that have greater values of λemp\lambda_{emp}?

  • Is there a reason for citing Muon via "Training Transformers with Enforced Lipschitz Constants" by Newhouse et al. versus the original blog post by Jordan et al. (which is itself cited in the Newhouse paper)?

  • In appendix section D.2.1, when discussing the critical transition point at t≈180t \approx 180 steps is claimed to be the point at which λemp\lambda_{emp} crosses zero, but when looking at both Figure 8 and Table 5, it appears that λemp<0\lambda_{emp} < 0 well before that point -- was it the case that before t=180t=180, the λemp\lambda_{emp} was merely near-zero but did not sharply diverge away, or something along those lines?

  • Is there a reason that a Muon-only line was not included in Figure 2a?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.