acceptodds
Under review as a conference paper at ICLR 2027

Hidden Sensitivity in LLM Pre-training: A Finite-Time Lyapunov View

Abstract

Subtle differences in numerical precision, parameters, or optimizer state are unavoidable in LLM pre-training, yet loss curves reveal little about whether training will suppress or amplify them. We model AdamW pre-training as an extended-state dynamical system and introduce a directional, parameter-seeded normalized Adam-state finite-time Lyapunov exponent (FTLE). Batch-order changes are evaluated separately as exogenous forcing. Across a 60M LLaMA-style model and OLMo-1B continual pre-training, sensitive windows identified by FTLE produce persistent separation in optimizer state, gradients, logits, token predictions, generations, and individual benchmark decisions, even when aggregate loss and accuracy remain nearly unchanged. FTLE complements learning rate and update norm by adding information about the rate of trajectory separation. The pattern recurs across independent runs and model scales. Offline FTLE profiling further identifies intervention windows that reduce perturbation separation at a measurable optimization cost. At matched scheduled budget, FTLE-selected windows reduce separation by 7.0% and 7.3% relative to gradient- and update-norm-selected windows, respectively, with a 0.002 loss difference from the gradient baseline and lower loss than the update baseline. Finite-time perturbation amplification therefore provides a practical framework for measuring and controlling hidden sensitivity in LLM pre-training.

Primary area:
optimization
Keywords:
large language model, pre-training, finite-time Lyapunov exponent, training dynamics, optimization stability, perturbation sensitivity

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_bPhZOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This review was entirely human-written, GPT-6 Astra was used after the fact to adversarially contradict my review. I manually corrected some of my statements that I judged to have been misrepresented based on this.

The authors look at how we can analyze pre-training trajectories through the lens of Finite-Time Lyapunov Exponent theory from dynamical systems. They use the AdamW state to get a proxy for the FTLE. They claim through experiments that when the proxy FTLE is high (high divergence in the dynamical system), the pre-training trajectory varies significantly with small perturbations. This divergence does not show up in the training loss. They use this insight to design an FTLE-aware optimizer. They show additional experiments which I am not sure I understand due to the extremely dense and jargon-heavy text that simply describes results rather than interpreting them. I cannot evaluate the paper properly when it comes to the quality of the results.

Critical strengths
  • I really like the main idea of looking at pre-training through the lens of dynamical systems theory: this leads to being able to reuse ideas that have been iterated on for a long time in the literature
  • The core finding that different pathways in optimizer trajectory can still lead to the same loss is interesting, and deserves further studying.
Critical weaknesses

Nevertheless, the presentation of the paper is poor which makes it below the acceptance threshold. The authors would need to rewrite the paper for it to pass what I would expect from a paper at ICLR. I encourage the authors to resubmit at a later date since the core of the paper is very exciting and interesting!

  • Presentation
    • Much of the paper is extremely hard to read. Section 3.1 is the only section that I was able to read easily. The rest of the text is filled with jargon and complicated sentence structures.
    • Un-introduced datasets, baselines, and metrics (e.g. Table 3), L200 lists metrics and datasets, but do not define them, give relevant citations for the datasets. The authors need to 1. fix the sentence L200 which is grammatically incorrect 2. add the relevant citations 3. explain why these datasets and metrics were chosen in the context of FTLE for pre-training.
    • Opaque methodology that is not justified
      • The normalized Adam-state FTLE is not clearly explained, it is not clear how this derives from the general form of the FTLE: the authors should either link an appendix to show how these are equivalent mathematically or give a qualitative explanation.
      • The Lyapunov-aware AdamW control equations are un-justified: the authors need to justify the choices they made for these.
    • Missing reference (L201, ?? reference), typos in notation (L159, uu is supposed to be mm? )
  • Experiment clarity: the choices for each experiment are poorly justified which leads to confusing the reader
    • Figure 1 is difficult to parse, the caption does not give any details and the main text simply describes the results.
    • Figure 2 does not show that high and low-FTLE respond differently: it simply implies that locations (in weight space) of high FTLE lead to high separation and this persists over time: the training time in Figure 2 is 200 steps but the checkpoints were taken 2000 apart. The experiment needs to be more carefully designed.
Questions for authors
  • Muon is more common for pre-training LLM's do you expect the findings to be transferrable?
  • I rationalize the findings by thinking of e.g. a 2d quadratic function (convex bowl) with noise. At certain points where noise is high, you would have high FTLE as the trajectory can diverge either side of the noise "bump", but the loss would then still go down similarly. In this frame, are the results that trajectories diverge while loss decreases similarly surprising?
Reviewer_NsKFOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents FTLE, an Adam/AdamW specific measure that can to a certain extent predict loss growths and "sensitive windows" of higher susceptibility to instabilities. It can be used to "dampen" instabilities perceiving the training as a control system - potentially mitigating larger spikes in training. While the idea is promising, the exact choice is not well-supported theoretically and the experimental evidence is not (yet) statistically significant.

Critical strengths

Defining proper measures quantify or predict variances in model training is an underexplored field, and this paper present such a measure even closes the control loop towards stabilizing the training.

Critical weaknesses

The theoretical motivation for the exact combination of terms for FTLE is weak - it is simply stated. The experimental evidence is weak (three runs at 60M scale, one at 1B scale) with outdated models (Olmo-1B), the predictive quality is just shown qualitatively, not quantitatively. There is no analysis on statistical significance.

Questions for authors

What is u exactly in formula (10)? Please define it properly. What is the undefined reference in line 201/202? How does your method expand to modern optimizers like Muon?

Reviewer_b2k4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

Small perturbations in training (data ordering, precision, etc) can cause very different training trajectories without changing the loss curve. This paper asks whether we can distinguish whether small perturbations will cause large or small differences in the training trajectory. They formulate training with Adam as a dynamical system that includes the parameters, optimizer state, learning rate schedule, and other information like the actual batches and precision / hardware behavior.

They propose a metric called Normalized Adam-state FTLE that is supposed to measure something about perturbation growth. They propose a controller (or two controllers? a "mild controller" and "internal controller"?) that clips the Adam first moment, damps the second moment and preconditioner, and clips the update-to-parameter ratio.

They perform experiments on a 60M LLaMA-style model and a OLMo-1B continual pre-training setting and identity "sensitive windows" where the aggregate loss and accuracy stay the same across perturbations but the optimizer state / gradients / logits etc are very different.

Critical strengths
  1. The core idea has some value: the majority of research on training stability looks at aggregate quantities like loss curves (and spikes) or gradient / parameter norms, but it could be useful to know when the training dynamics are sensitive vs robust to small perturbations.
  2. Their FTLE metric does show that there is info about the sensitivity to perturbations that is not visible in the loss curve.
  3. They provide code for reproducibility.
Critical weaknesses
  1. The writing sounds like AI and is hard to follow, particularly in the introduction and experiment sections. Here are some examples of writing I found difficult to understand:
This question separates three notions often conflated by aggregate evaluation. Two runs may have
similar validation loss and benchmark accuracy while their parameters and Adam moments follow
different trajectories, and while their logits, token choices, and generations cease to be reproducible.
We call these aggregate, dynamical, and behavioral stability, respectively. Measuring the latter two
requires a local diagnostic alongside scalar performance summaries. This pathwise notion differs
from algorithmic stability, which relates data changes to generalization (Hardt et al., 2016): here the
data stream may be fixed while we measure amplification along one realized optimization trajectory.
Our study connects measurement, consequence, generality, and control.
  1. The theoretical contributions, the FTLE metrics and control, are not justified well at all. I don't understand why these metrics or controller should be used rather than any other choice of metrics or controller. I don't even really understand what the definition of the metric is.
  2. The empirical evaluations are quite limited. I didn't understand what I should take away from the experiments.
Questions for authors
  1. Why did you choose these metrics and this design for the controller?
  2. How would your ideas generalize to other optimizers?
  3. How would you expect your methods to be used in practice?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.