- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper considers the problem of how linear probes get affected by various training setups such as SFT, DOP and other RL methods. They find that SFT causes the most significant change in probe performance compared to DPO and RL. In the second half of the paper, the authors then propose methods to mitigate this drop in probing performance without having to obtain new activations and labels. Specifically, a label-free orthogonal drift correction mechanism is proposed in the paper.
Critical strengths
Given that probing is an established technique for checking for various useful properties of a model (although there are criticisms as to the validity of linear probing as a method for this such as https://arxiv.org/abs/2004.03061) the paper might be of interest to practitioners who would like to track a particular property of a model across multiple checkpoints without having to retrain the probe by extracting labelled activations every time.
Critical weaknesses
The paper is rather poorly written and the authors do not explain why it is important to study this issue and why one would expect a linear probe trained on one checkpoint to even generalise to another. Linear probes are trained for various purposes (as briefly mentioned in the first paragraph of the intro) for detecting various properties of a model such as its social biases. However, the idea is to study a specific model (checkpoint) and given that training (using any method) would substantially modify the weights of a model, there is no expectation for a linear probe to generalise (applicable from) one checkpoint to another. What benefits should we see if this happens? is something that the authors should have motivated well in the introduction. Although I can "guess" this because of my expertise of the domain, I think that should not be an excuse not to make this explicit in the introduction. I am not sure whether the paper has been written using LLMs, but it has not helped at all to make it more readable in this case.
Given that linear probing has been criticised and more complex non-linear probes have been recommended in prior work (https://arxiv.org/abs/2004.03061) I wonder what the value of the findings of this paper be. I have formulated this also as a question below.
The label-free methodology is effectively orthogonal procrustes, which has been widely applied in NLP/ML for cross-lingual or cross-domain feature alignment (see https://aclanthology.org/N15-1104/ for example). Can the authors explain the novelty of the method proposed in your work against this backdrop?
Questions for authors
Given that linear probing has been criticised and more complex non-linear probes have been recommended in prior work (https://arxiv.org/abs/2004.03061) I wonder what the value of the findings of this paper be.
The label-free methodology is effectively orthogonal procrustes, which has been widely applied in NLP/ML for cross-lingual or cross-domain feature alignment (see https://aclanthology.org/N15-1104/ for example). Can the authors explain the novelty of the method proposed in your work against this backdrop?
Thank you for taking the time to review our paper and recognizing the practical value of probe transfer. We address each of your concerns below:
The motivation is already explicitly provided in the opening paragraphs of our introduction, which explain that probes are used in LLM monitoring, model updates can negatively affect probe performance, and retraining probes “is limited by the cost of generating new responses, labeling them and, in some cases, extracting terabytes of per-token activations.”
Linear probes have a concrete application in LLM safeguards. Anthropic’s August 2026 Risk Report describes streaming linear probes being used as a component of deployed safeguards, linking this design to Cunningham et al.’s Constitutional Classifiers++, which our introduction already cites. We have edited our introduction to make the connection to safeguards clearer.
The paper you cited formulates probing as estimating mutual information between representations and linguistic features and recommends the best performing probe for that purpose. Our objective is reliable, lightweight detection and we do not use metrics for linear probes to establish causal relationships or to measure all information in representations.
The limitations of linear probes as tools for interpreting representations do not negate their practical use as lightweight detectors or the importance of studying their transfer across model updates (especially since safeguards should be kept effective as the model updates).
We do not assume that probes generalize from one checkpoint to another. Studying whether and when they generalize is one of the main empirical questions we aim to answer with this study. Empirically, we find that in some cases, linear probes generalize very well without intervention (such as between DPO and RL checkpoints as shown in our paper).
The practical benefit of probe generalization follows directly from the motivation provided in the introduction (that probe retraining is costly). To make this connection clearer, we have also explicitly added that successfully transferring probes can help avoid these retraining costs.
Our paper differentiates between label-free representation alignment in Section 5 and our supervised correction method that we propose in Section 6.
The orthogonal drift correction, which we propose in Section 6, is supervised and does not perform Procrustes or label-free alignment (though it can be used alongside Procrustes or any other label-free representation alignment method.)
We agree that label-free alignment has been well-studied and also have explicitly acknowledged that it has been used in prior probe transfer papers (Duan (2026) studies probe transfer using label-free alignment when the model undergoes quantization, LoRA, and QLoRA.) Our main contribution in Section 5 is to evaluate how well these methods perform in transferring probes across post-training checkpoints. Note that Duan (2026) also does not analyze probe repair performance as a function of the data budget. We also find that in some probe transfer cases label-free alignment can degrade performance.
Our results in Section 5 also motivate why our proposed orthogonal drift correction method in section 6 is useful. Our proposed orthogonal drift correction method in Section 6 can improve transfer beyond using label-free alignment alone.
We hope these clarifications address your concerns. If they do, we would appreciate reconsideration of your assessment of our paper.