acceptodds
Under review as a conference paper at ICLR 2027

A Transport Law for Linear Directions Between Language Models

Abstract

The Linear Representation Hypothesis argues that a concept corresponds to a direction in a model's representation space, while the Platonic Representation Hypothesis posits that independently trained models converge to a shared structure of representations. A central unresolved question is whether these two ideas imply that a behaviorally meaningful direction learned in one model should be recoverable and transferable in another. We address this question in a deliberately narrow setting: the **direction of a successful attack** (prompts for which a model produces a response judged unsafe by a guard) within a single modality, text decoders, and the linear case, where both read-out and transfer are linear. Under class imbalance and guard-dependent labels, we first derive an **admissibility criterion** for statistics used to measure transfer of the attack space. From these admissible statistics, we derive a **transport law**: the correlation between a probe's score in its source model and its carried score in the target model equals the root-mean-square canonical correlation of the model pair, weighted across directions by the probe's profile. We further prove an exact decomposition of transport into within-class transport and label transport. The decomposition reveals a striking asymmetry: labels transport without loss, and on some pairs with a gain (–), whereas the underlying direction is only partially recovered (–). We validate the transport law across 11 dense transformers ranging from B to B parameters, yielding 110 ordered model pairs. Their responses are labelled by ten diverse guards using a corpus of prompts, with representations compared at fourteen relative depths. We further show that the law holds beyond text decoders, on 8 alternative architectures ranging from B to B parameters. In classical decoder transformers we find that transfer is asymmetric in model size, where a probe transferred from a smaller model to a larger model loses markedly less AUC than transfer in the reverse direction, and the difference increases with the size gap. Together, these results provide a principled framework for testing whether a **shared linear direction of successful-attack behavior exists across independently trained models and transfers between them**. More broadly, our work turns two influential representation hypotheses from qualitative claims into testable, quantitative questions about cross-model representation transport, with direct implications for understanding and controlling model behavior in AI safety.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI safety, interpretability, linear probes, cross-model transfer, representation similarity, large language models, guardrail models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_xQwpOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper formalizes the idea that models have a shared linear structure of representations. The metric they introduce, which they term the “transport”, is the correlation of the scores of the same linear probe transferred across models using a “stitch”: an unsupervised method which simply finds the best-fit linear transformation from one space to the other. They decompose this correlation metric into a within-class and between-class component, where it turns out that the within-class component dominates for a probe that flags rarely. They also show that the more obvious metric, the AUC transfer gap, splits into a label transfer and geometry transfer component, and that the geometry part is negligible in practice.

They empirically validate their formula for the transport using the case of a probe which predicts whether a jailbreak will lead to an unsafe response. Moreover, they find that probes transport better from small to large models than large to small.

Critical strengths

The small-to-large transfer result could be useful, as you could imagine training activation monitors more cheaply on jailbroken transcripts from smaller models and then just applying a stitch to transfer to a larger model. Larger models can be harder to jailbreak and more expensive to generate from, so these would be significant savings.

Critical weaknesses

If you just read the main text, it seems like the main contribution is Theorem 4: the abstract says “we derive a transport law”, it is the only theorem (everything else is a lemma or corollary) and it is the subject of the headline figure. However, in B.7 the authors admit “The transport law is not new: canonical analysis goes back to Hotelling (1936), and the fact that the regression of one model’s representation on another’s decomposes into separate canonical components is set out in textbooks of multivariate statistics.” This reads to me as very disingenuous and causes me to conclude that a large fraction of the presented work is not novel.

The narrative for rejecting the AUC transfer gap is also unclear. Observation 3 says that the AUC transfer gap is zero “for every label”, but I think what it really mean is every label shared by both models? In that case, if two models have the same labels and the same geometry, what is wrong with the AUC transfer gap being zero? Why would you call that degenerate? The text following the observation only adds to the confusion, stating “a measured nonzero gap must then be explained not by a difference of spaces” but the the contrapositive of Observation 3 is that a non-zero gap implies that the two spaces are not linearly equivalent, so it actually does imply “a difference of spaces”? This is key to the story of the paper, where the AUC transfer gap is rejected in favor of the transport metric of Theorem 4, but is very poorly explained. In fact, the transport metric seems to suffer from the same issue as the AUC gap “function of the triple (representation, representation, label), not of the pair of representations” since it depends on the label via the probe w. Figure 2 presents an empirical argument against using the AUC transfer gap but this is discussed much later.

Questions for authors
  • Can you clearly explain why you reject the AUC gap in favor of the transport statistic? Is it for theoretical or empirical reasons?
  • How do you imagine these jailbreak-input probes being used in practice?
  • Figure 3B: Why is tau decreasing with depth (vs. r peaking ~.25), and why is it greater than 1 in early layers?
  • Could you please expand on the threat modeling? Why is the probe applied to the prompt rather than the response? If I wanted to actually use this to detect jailbreak inputs, I would want to know what kind of classification (AUC or recall or FPR) am I going to get depending on whether I train a probe directly or carry over from a smaller model. In Appendix I.3 I think it is saying the AUCs that you get even without transfer are <80%, which seems way too low to be useful? Having a small AUC gap doesn’t mean anything if the baseline AUC is so low. Moreover, is it not a major problem that the transfer “drops by half” on a different prompt source?
  • Could you please work to improve the clarity of the writing? There are lots of unnecessary clauses which appear LLM-generated. Lots of “it’s not X, it’s Y”, lots of writing out numbers rather than just referring to plots. The text could be improved structurally for instance by starting with a threat model which motivates the design choices in the paper. The phrase “guards’ verdicts” in particular was very confusing to me, it sounds like two guards disagreeing with each other, but in fact it is the same guards giving a verdict on two different responses.
  • The conclusion drops a bombshell that the transfer “drops by half” with distribution shift. What does this mean exactly? This should have been discussed properly in the results.
Reviewer_3XoeOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates linear-probe transfer across language models and derives a transport law: under optimal linear reconstruction, the correlation between original and transferred scores equals the root-mean-square canonical correlation between models, weighted by the probe’s directional profile. Experiments on 19 models show that attack-success label separability can largely survive or even improve despite only partial score recovery; smaller-to-larger transfer generally incurs less loss, although this asymmetry mainly appears in conventional Transformers and the theory overpredicts transfer for alternative architectures.

Critical strengths
  1. Clear theoretical contribution: The paper quantitatively characterizes linear-direction transfer using the canonical correlation spectrum and probe-specific weights, distinguishing score recovery from preserved label separability rather than relying solely on AUC.

  2. Broad empirical validation: Experiments span 19 models, multiple architectures, and 10 safety guards, with ablations and controlled structure-injection tests assessing robustness and detection sensitivity.

Critical weaknesses
  1. Overstated theoretical novelty and limited independence of the validation: The central "transport law" follows directly from optimal linear regression in canonical coordinates; Appendix B.7 explicitly acknowledges that the law is not new. Moreover, the paper states that its assumptions hold by construction on the rows used to fit the canonical analysis. Agreement between the spectrum-based prediction and the fitted stitch therefore partly checks consistency between closely related estimators, rather than independently establishing a new empirical law of language-model representations. The paper should distinguish its genuinely new empirical insights from this classical identity and more clearly establish the prediction's added value on held-out data.

  2. Insufficient care in figure presentation and manuscript verification: Figures 1-3 are embedded as raster images rather than vector graphics, and Figure 1 contains small, faint annotations that are difficult to inspect at normal reading size. Appendix B also contains an unresolved section reference ("§??"). These concrete presentation defects make a mathematically dense paper harder to audit and suggest insufficient final quality control. The authors should provide legible vector plots and resolve broken references; these defects alone do not establish automated authorship.

Questions for authors
  1. What is novel beyond the classical CCA identity? Appendix B.7 acknowledges that the transport law is not new, and its assumptions hold by construction on the rows used to fit the canonical analysis. Could you clarify the novel theoretical contribution and show whether the probe-weighted prediction outperforms simpler baselines, such as the leading canonical correlation and the unweighted RMS canonical correlation, when all predictions and mappings are estimated on training data and evaluated on held-out prompts?

  2. How should the discrepancy for alternative architectures be explained? Section 4.2 reports an OLS slope of 0.775 for alternative architectures, indicating systematic overprediction. Can you separate the contributions of finite-sample estimation, PCA truncation, and ridge regularization from architecture-related differences, and identify conditions under which the prediction remains well calibrated?

Reviewer_Q6HMOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies whether linear probes trained on one language model can transfer to another via a trained linear map. Therefore, it distinguishes recovering the probe’s original scores after transfer from preserving its ability to distinguish between classes.

First, the authors introduce a criterion for comparing representations that invertible differences in the geometry of representation space between models. They then derive a formula relating score recovery to the canonical correlations between the representations and the probe’s weighting of the corresponding directions. They also decompose score agreement into within-class agreement and separation between class means to explain how imperfect scores can still lead to largely the same classification performance.

The empirical evaluation focuses on the refusal direction as a case study. Across 19 models and ten safety guards, the authors find that classification performance largely survives transfer even when the original probe scores are only partly recovered.

Finally, the authors compare bidirectional transfer across model pairs. They find that transfer from smaller to larger transformers loses less AUC than transfer in the opposite direction. This pattern is, however, not observable in alternative architectures the authors tested.

Critical strengths
  • The separation of transport into preserving probe scores and preserving classification performance is useful. They also empirically show that probe scores can meaningfully change during transfer while classification performance remains largely unchanged.
  • The empirical evaluation covers a wide range of model families, sizes, and architectures. Testing results across different LLM-judges is also useful.
Critical weaknesses
  • The authors motivate the study by arguing that cross-model transfer of linear directions is an unexplored question but this is largely untrue. Previous work ([1, 2, 3]) already studies transferring steering vectors and linear probes between language models. The authors should state their own contribution more clearly.

[1] A. Chen, J. Merullo, A. Stolfo, and E. Pavlick, “Transferring Linear Features Across Language Models With Model Stitching,” 2025. [Online]. Available: http://arxiv.org/abs/2506.06609

[2] N. Oozeer, D. Nathawani, N. Prakash, M. Lan, A. Harrasse, and A. Abdullah, “Activation Space Interventions Can Be Transferred Between Large Language Models,” 2025. [Online]. Available: http://arxiv.org/abs/2503.04429

[3] Y. Huang, C. Huang, D. Feng, W. Lei, and J. Lv, “Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts,” 2025. [Online]. Available: http://arxiv.org/abs/2501.02009

Questions for authors

NA

1 comment

We thank you again for the references.

We have attached a revised version that adds the references you listed. It also includes the new results and a reworked narrative, since we agree that the pre-revision version was messy.

We would be grateful if you could take the time to look at the revised manuscript and consider revisiting your score. If anything in your assessment remains open, we would be glad to address it in the remaining window.

Comments on the paper

4 comments

Thank you for your response. We have submitted a revised version of the paper that incorporates your suggestions. In addition, the revised version adds the new results and reworks the narrative, since we agree that the pre-revision version was messy.

We would be grateful if you could take the time to look at the revised manuscript and consider revisiting your score. If anything in your assessment remains open, we would be glad to address it in the remaining window.

We thank the three reviewers for their time. We are grateful for this round regardless of its outcome. The reviews identified a real failure of exposition, and the revised paper is a better one for it. We will carry that version forward in any case. If the committee considers the work worth a second look, the revision is where the case now stands. Below, we provide a summary for the AC and SAC.

  • The common thread of the three reviews is readability. All three set Presentation to 1, and almost all of the objections concern material that was already in the submission (§1, §4.2, Appendix). We accept the criticism without reservation and have rewritten the structure of the paper in the revised submission.

  • During rebuttal we answered each review: Chen et al. (2025) was missing and is now cited, with §1 stating the delta instead of implying that cross-model transfer is unexplored (Q6HM); the baseline comparison and the two out-of-sample checks the review asked for are added, and the manuscript defects are fixed (3Xoe); the identity is now Lemma 1 (Transport law, Hotelling), the argument for setting the AUC transfer gap aside is rewritten, and the distribution-shift result has moved from the conclusion into the results (xQwp).

  • The study covers 19 models from 0.5B to 70B parameters in two architecture families, labelled by ten guard models, over 342 ordered model pairs, 3420 (pair × guard) cells, fourteen relative depths and two prompt sources on an identical corpus of 3306 prompts. Appendices A–L carry the controls, including six checks whose reading rules were written before the data; one returns an outcome against us and is reported as such.

  • The result is that the transfer of a linear direction between two models can be predicted before any map is fitted, from the canonical correlations of the pair weighted by how much the probe uses each of them: on classical dense transformers the prediction matches the measurement with Pearson 0.998, and it beats both pair-only predictors by an order of magnitude in error. We also report the conditions under which it fails (a change of prompt source, and the operating point of a stand-alone monitor with metrics and details).

  • Reviewer Q6HM reports no substantive soundness concerns (Soundness 3), and Reviewer xQwp has confirmed that the revisions address the points raised.

Thank you again for considering our work.

A couple of key clarifications:

  • I have not "confirmed that the revisions address the points raised". The comment you are referring to was made 3 days before you submitted the revision. This is another example of the authors not being consistently candid.
  • The writing has improved but still has a long way to go. Lots of claude-isms like “attack stratum”, “control slab” and “intrinsic-dimension contrast" remain undefined. Many of the sentences are incredibly dense and hard to read, like the one containing “keeps 0.21 of the residual variance of the measured transfer that it explained.”. There are still numbers without units like "the prediction drops (0.743 against 0.759 on average)".

We apologize for missing your comment before. Yes, your comment came before the revision, so it could not have been about it. We chose the wrong verb and only meant that you had called the proposed changes good. English is not our first language, and the responsibility for the sentence is ours. We apologize, and we had no intention of misrepresenting you.

On the three terms. We did not think of them as defined objects, or as terms that needed definitions. But we see now that a reader can read them differently, and that this matters here (two of the three appear inside quantitative statements). “Control slab” and “intrinsic-dimension contrast” are defined in the appendices but not where they first appear; “attack stratum” is not defined anywhere. We will define each at first use or replace it with the explicit wording. Thank you again!

None of this changes what the work is. It is fully reproducible, where the theoretical statements, including the bounds, are proved in Appendix B, and the empirical claims are backed by ablations and controls (Section 4 and the appendices). My coauthors and I stand behind the theoretical results.

Thank you for reading the text closely. This rebuttal has been a real lesson for us. We will keep working on the paper, during the remaining discussion period and after it.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.