Measuring the stability of autoregressive generation: Paired latent trajectories under benign input perturbation
Abstract
Replacing a single word in a factual question can change an instruction-tuned model's answer even when the question's meaning is preserved. Across six open-weight models, 8 to 34 % of our audited substitutions change the answer. We introduce the Latent Trajectory Assay (LTA) to measure how such changes propagate through generation. LTA pairs hidden-state trajectories for clean and perturbed prompts in two conditions. Letting the perturbed prompt generate its own continuation shows whether the answer changes. Under teacher forcing, the clean continuation is fixed through both prompts, which isolates the internal response to the input change. In all six models the internal divergence, and how well it predicts a changed answer, grow with depth. Most perturbed trajectories drift back toward their clean counterparts, but so do meaningless edits such as a change of letter case, and what a real substitution adds is a larger displacement that outlasts that baseline. Scored only on positions after the answer, so that it cannot read the flip itself, divergence predicts answer flips at AUROC 0.68 to 0.76 and beats an output-probability score from the same pass on every model by point estimate. A human audit of 360 substitutions rejects 14 percent as meaning-changing. The signal survives their removal (AUROC 0.68 to 0.78), and it flags the rejected substitutions on its own (AUROC 0.76). For screening fragile prompts before a rewording is known, black-box resampling beats every hidden-state statistic, and an erosion score from the teacher-forced pass improves that screen on five of six models without hidden-state access. LTA's contribution is therefore not a better detector but a controlled, depth-resolved measurement of the internal response to a specific input change. We release the instrument, capture archive, and tooling.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.