acceptodds
Withdrawn from ICLR 2027

Spectral Monitors Detect Change, Not Damage, in LLM Post-Training

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can hidden-state spectra distinguish harmful post-training updates from benign adaptation? A fixed-readout construction shows that identical spectra can coexist with arbitrarily different predictive losses—an identifiability limit, not an impossibility theorem for empirical alarms. We test the empirical question with pre-registered shared-prefix stress tests on three open models, run on a corrected training/probe split with fresh seeds. Extreme duplication damages held-out NLL by to nats while raw RankMe rises in 9/9 model–seeds, and the same high-LR damage moves RankMe down on OLMo-2-1B but up on Qwen3-1.7B: the sign of a spectral response tracks the (regime, architecture, layer) tuple, not damage. A trajectory-level split-conformal protocol then compares spectral and loss monitors at an equal false-alarm budget on the development model (Qwen3-0.6B): on abrupt damage every spectral family alarms at the same measurement as a plain loss detector; on a gradual learning-rate ramp spectra alarm 80–110 steps earlier, but CUSUM and Page–Hinkley detectors on the same loss stream recover about half (CUSUM) and a quarter (Page–Hinkley) of that lead, and all of these detectors also fire on a non-damaging duplication ramp and on a benign control that improves its target ( nats) with no significant task-accuracy change. Only a loss monitor with a non-degenerate threshold separates damage from benign change; a pre-registered matched-threshold control shows the sidedness of the detector is immaterial. An earlier campaign affected by a training/probe exposure defect is disclosed, repaired, and reported side by side; its structure replicates cleanly. In these settings, the tested spectral diagnostics measure change, not damage.

Primary area:
foundation or frontier models, including LLMs
Keywords:
representation monitoring, spectral statistics, effective rank, RankMe, continual post-training, conformal prediction, identifiability, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_auEXOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper examines common spectral-based metrics (RankMe, matrix entropies), which are often used as signal of degradation in LLM training. Using controlled experiments on 3 small models with a fixed alarm budget, the authors show that these monitors reliably detect that the representations changed, but not whether the change was harmful.

Critical strengths
    1. The paper performs an interesting analysis on the metrics often used, which may benefit practitioners in training.
    1. The authors's experimental design uses matched controlled comparison enhances the credibility of the findings.
Critical weaknesses
    1. The paper's setup is borderline circular: it defines "damage" as probe-NLL (+0.30 nats, which is also arbitrarily chosen) increases and then concludes that an then concludes that a probe-NLL monitor is the only detector that separates damage from benign change. The test results show that this is confounded with test results (extreme duplication is maximally "damaged" with HellaSwag flat), raising confusion about the definition of damage on capability or representation-based.
    1. The central claim that the monitors fire on benign adaptation while a loss monitor stay silent is backed by only 1 benign control, which is far too thin to support the conclusion. While the paper acknowledges this partly, it still strongly weakens the paper's central argument, and generalization to other common training settings (LoRA, RLHF updates).
    1. Finally, the paper is borderline unreadable with confusing organization, using terminology without sufficient introduction, with dense results without sufficient explanation.
Questions for authors
    1. The authors define the 'damage' threshold at 0.30 arbitrarily. Did you run a sensitivity sweep to see how this changes?
    1. Since damage is defined by an increase in probe NLL, how much of the loss monitor’s apparent advantage reflects its access to the signal defining the outcome?
    1. Table 5 shows extreme duplication is "damaged" by your gate (+1.34 nats) while HellaSwag accuracy is statistically flat. Does it mean that that spectral monitors fail to detect capability loss, that they fail to detect loss loss, or both?
    1. The specificity claim rests on one benign recipe (mild GSM8K domain shift, one model, three seeds). Given that the two-sided NLL detector also fires on 2/3 non-damaging duplication ramps, what justifies the general claim that a non-degenerately-thresholded loss statistic are better than spectral monitors at distinguishing damage from benign change?
    1. All models are ≤1.7B with full-parameter fine-tuning on a single Dolly prefix, and the damaging regimes are extreme by construction. How would the finding changes on different choices of datasets and/or training regimes?
Reviewer_m99nOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper shows that spectral monitoring may not correctly detect performance degradation, both theoretically and experimentally. The authors observe that spectral monitoring can raise alarms earlier than other methods, but it can also raise alarms for beneficial changes in the model.

Critical strengths

The paper provides good experimental evidence that supports its claims.

Critical weaknesses

Since this paper is a negative-result paper, it would be better to provide more solid theory that is closer to the last layer of LLMs. However, this paper only handles a binary setting with simple representations. To provide more solid theory, the paper would need an analysis of multiclass classification with softmax loss, where the output dimension is larger than the input dimension, under realistic assumptions about the input distribution that are close to LLM representations. I know that analyzing a full LLM theoretically is too hard, but analyzing the last layer may be necessary to provide more solid theoretical evidence.

Questions for authors

If the authors think their theoretical setting is sufficient to provide solid evidence supporting their claims about LLMs, or if they think it can be directly extended to the last layer of LLMs, an explanation would be helpful.

Reviewer_BPMfOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether hidden-representation spectral statistics can identify harmful LLM post-training updates. It combines a fixed-readout non-identifiability construction, controlled interventions on three models, and trajectory-level conformal calibration. Raw RankMe increases in all nine clean model–seed runs under extreme duplication despite substantial NLL degradation. The tested spectral monitors also alarm during the designated benign adaptation, revealing limited damage specificity under the chosen calibration protocol.

Critical strengths
  1. Useful empirical counterexamples. The clean cross-model experiments show that simple directional spectral heuristics are unreliable indicators of damage. The rank increase under extreme duplication and the model-dependent high-LR responses are informative findings for representation-based monitoring.

  2. Thoughtful experimental design and transparent reporting. Shared-prefix experiments improve comparability across interventions. Benign controls, sequential loss baselines, and the matched-threshold analysis strengthen the evaluation. The authors also disclose the historical probe-exposure issue and perform fresh experiments with the corrected split.

Critical weaknesses
  1. The equal-budget interpretation needs clarification. Figure 2 and Table 11 report the “earliest spectral family,” while individual families are calibrated separately. If this selects the earliest alarm across families, the resulting rule does not automatically retain the nominal 0.05 false-alarm guarantee. This distinction matters for the claimed timing advantage over individual loss detectors.

  2. Capability-based early warning remains insufficiently established. NLL degradation and task-accuracy degradation do not consistently coincide: extreme duplication leaves multiple-choice performance largely unchanged, although LAMBADA declines. The independent capability-event analysis in Appendix C is helpful, but uses historical endpoints. Sparse capability checkpoints prevent determining whether spectral alarms precede the onset of capability degradation.

  3. Monitoring performance has limited cross-model validation. The sign findings span three models, but calibrated specificity and gradual-onset timing are evaluated only on Qwen3-0.6B, with one benign adaptation recipe. Given the demonstrated model dependence, these monitoring conclusions have narrower empirical support than the cross-model directional findings.

Questions for authors
  1. Does “earliest spectral” denote the prespecified, jointly calibrated all-spectral union or the minimum alarm time across separately calibrated families? Please report clean per-seed timings for the calibrated union relative to CUSUM and Page–Hinkley, or explain how the selection rule retains the stated false-alarm guarantee.

  2. When the independent capability-event rule from Appendix C is applied to the clean checkpoints, do the specificity conclusions remain unchanged? Please distinguish demonstrated lead time relative to the NLL severity gate from evidence concerning the onset of capability loss.

  3. Is there calibrated monitoring evidence beyond Qwen3-0.6B? If not, please distinguish the cross-model sign findings from the development-model specificity and timing findings in the main conclusions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.