Hidden Sensitivity in LLM Pre-training: A Finite-Time Lyapunov View

Subtle differences in numerical precision, parameters, or optimizer state are unavoidable in LLM pre-training, yet loss curves reveal little about whether training will suppress or amplify them. We model AdamW pre-training as an extended-state dynamical system and introduce a directional, parameter-seeded normalized Adam-state finite-time Lyapunov exponent (FTLE). Batch-order changes are evaluated separately as exogenous forcing. Across a 60M LLaMA-style model and OLMo-1B continual pre-training, sensitive windows identified by FTLE produce persistent separation in optimizer state, gradients, logits, token predictions, generations, and individual benchmark decisions, even when aggregate loss and accuracy remain nearly unchanged. FTLE complements learning rate and update norm by adding information about the rate of trajectory separation. The pattern recurs across independent runs and model scales. Offline FTLE profiling further identifies intervention windows that reduce perturbation separation at a measurable optimization cost. At matched scheduled budget, FTLE-selected windows reduce separation by 7.0% and 7.3% relative to gradient- and update-norm-selected windows, respectively, with a 0.002 loss difference from the gradient baseline and lower loss than the update baseline. Finite-time perturbation amplification therefore provides a practical framework for measuring and controlling hidden sensitivity in LLM pre-training.