- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 3
Read review
Paper summary
The paper examines common spectral-based metrics (RankMe, matrix entropies), which are often used as signal of degradation in LLM training. Using controlled experiments on 3 small models with a fixed alarm budget, the authors show that these monitors reliably detect that the representations changed, but not whether the change was harmful.
Critical strengths
-
- The paper performs an interesting analysis on the metrics often used, which may benefit practitioners in training.
-
- The authors's experimental design uses matched controlled comparison enhances the credibility of the findings.
Critical weaknesses
-
- The paper's setup is borderline circular: it defines "damage" as probe-NLL (+0.30 nats, which is also arbitrarily chosen) increases and then concludes that an then concludes that a probe-NLL monitor is the only detector that separates damage from benign change. The test results show that this is confounded with test results (extreme duplication is maximally "damaged" with HellaSwag flat), raising confusion about the definition of damage on capability or representation-based.
-
- The central claim that the monitors fire on benign adaptation while a loss monitor stay silent is backed by only 1 benign control, which is far too thin to support the conclusion. While the paper acknowledges this partly, it still strongly weakens the paper's central argument, and generalization to other common training settings (LoRA, RLHF updates).
-
- Finally, the paper is borderline unreadable with confusing organization, using terminology without sufficient introduction, with dense results without sufficient explanation.
Questions for authors
-
- The authors define the 'damage' threshold at 0.30 arbitrarily. Did you run a sensitivity sweep to see how this changes?
-
- Since damage is defined by an increase in probe NLL, how much of the loss monitor’s apparent advantage reflects its access to the signal defining the outcome?
-
- Table 5 shows extreme duplication is "damaged" by your gate (+1.34 nats) while HellaSwag accuracy is statistically flat. Does it mean that that spectral monitors fail to detect capability loss, that they fail to detect loss loss, or both?
-
- The specificity claim rests on one benign recipe (mild GSM8K domain shift, one model, three seeds). Given that the two-sided NLL detector also fires on 2/3 non-damaging duplication ramps, what justifies the general claim that a non-degenerately-thresholded loss statistic are better than spectral monitors at distinguishing damage from benign change?
-
- All models are ≤1.7B with full-parameter fine-tuning on a single Dolly prefix, and the damaging regimes are extreme by construction. How would the finding changes on different choices of datasets and/or training regimes?