acceptodds
Under review as a conference paper at ICLR 2027

Predictable Cross-Model Control Without Shared Coordinates

Abstract

Can information from one model guide a predictable intervention in another? We apply a source model's signed relation weights over paired examples to the receiver's own states, so no hidden vector crosses models. Across twelve directed paths among four decoder-only Transformer families and four controlled tasks, the resulting directions are closer to each receiver's relational reference than randomly permuted donors in every path-task cell. With identical donors, source weights produce larger held-out effects than a target-neutral control. A receiver score from disjoint audit items ranks held-out additive effects across the source construction and three controls, with Spearman correlations of and at two doses. In natural-context tests including an encoder-decoder model, source responses reduce receiver-response prediction error by beyond label, action, and clean-output baselines. These results establish two concrete uses of cross-model information: constructing receiver-native directions whose additive effects can be ranked by a local measurement, and predicting output changes from observed source responses.

Primary area:
interpretability and explainable AI
Keywords:
Activation steering, Cross-model transfer, Representation similarity, Causal interventions

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_gkxsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This research investigates whether information from one model can guide predictable interventions in another model without sharing hidden-state coordinates. It proposes transferring signed relation weights from a source model and applying them to the receiver model's own representations. Experimental results demonstrate that source-model information can be used to construct effective intervention directions and predict changes in the receiver model's outputs.

Critical strengths
  • This paper addresses an interesting research problem: transferring information across models without explicitly aligning their hidden representations.
  • The controlled experimental framework is well-designed and provides evidence supporting the applicability of the proposed approach.
  • The empirical results demonstrate cross-model transfer capabilities, with receiver-local audit scores showing strong correlations with held-out additive steering effects.
Critical weaknesses
  • The experiments are limited to models ranging from 7B to 9B parameters. It would be valuable to evaluate the proposed method across a wider range of model scales and include more recent model families to assess its generalizability.
  • The experiments primarily focus on controlled tasks, leaving the practical applicability of the proposed method unclear. Additional evaluations on real-world tasks would help demonstrate its effectiveness and generalizability beyond the controlled experimental settings.
Questions for authors

see the weakness

4 comments

Thank you for the positive assessment of the problem and the experimental framework; we address your two concerns, breadth and practical use, in turn.

What is new, in brief. (1) Hidden interventions at new scales and architectures: three encoder-decoder receivers (0.8B-3B) and Qwen3.6-27B. (2) Public-task interventions against strong baselines, including learned activation mappings. (3) Output prediction on three further public benchmarks. Each is summarized below.

Model scale and families. Hidden interventions now cover four decoder-only families (Qwen2.5-7B, Llama-3.1-8B, Gemma-2-9B, Mistral-7B; 48 path-task cells in the submission) and three encoder-decoder receivers (FLAN-T5-Large, mT0-Large, FLAN-T5-XL). On the encoder-decoder receivers, with a Qwen source, all four tasks and all 12 receiver-task cells, source weights beat neutral weights (the same receiver donors, weighted by baseline-prompt similarity) by .126/.251 and a same-access ridge mapping by .035/.069 margin units at doses .5/1; all simultaneous 95% CIs exclude zero. Source weights exceed neutral weights in 11/12 cells and ridge in 10/12; accuracy rises from 75.0% to 78.1%/77.1%. For scale and recency, we added hidden interventions on Qwen3.6-27B, a recent model whose layers are mostly linear attention, from Qwen2.5-7B and Mistral-7B sources (Sec. 5.2, App. I.4). The prespecified contrast holds: source weights beat neutral weights by .20/.24 (simultaneous CIs [.14,.26]/[.19,.29]), concentrated in two of four tasks, and ridge by .15/.16; accuracy rises from 95.0% to 96.6%/98.8% (Fig. 3). This adds to the submission's 27B representation result (App. E.3; gap .215 after label removal). Hidden interventions now span 0.8B-27B receivers. Fixed cuts at relative depths .625/.875 give effects correlating .69-.93 with the central cut (Table 18).

Real-world tasks. The submission already includes BoolQ and RTE (App. F). We added a full comparison on both, in both Qwen/Mistral directions, with 64 paired support and 64 held-out test items per task, against 12 alternative constructions using the same support. Relational steering exceeds neutral weighting by 3.72/3.92, label-preserving permutation by .66/.62, and two applications of the learned activation mapping of Oozeer et al. (ICML 2025), a Linear-ReLU-Linear network trained to map source to receiver activations, by 1.83/2.04 and 1.00/1.15 margin units, all with simultaneous CIs excluding zero and positive in all four path-task cells at both doses. Class-level variants perform comparably (see our reply to Reviewer P99D). Accuracy rises from 84.8% to 86.7%/85.5%; we claim larger causal effects, not higher accuracy than the strongest controls. Practically, an intervention can be moved into a new model without training a map between their hidden spaces.

A second use predicts how a receiver's output will change when its context is edited, from a source's observed response, without querying the receiver on the edit. The submitted natural-context study (Sec. 5.5) already includes FLAN-T5-XL: source responses cut this prediction error by 32.5% overall and by 36.6% on directions involving the encoder-decoder model. We added ARC-Challenge, OpenBookQA and HellaSwag, each with 192 calibration and 512 new held-out questions. Across the six directed paths among FLAN-T5-XL, Qwen2.5-7B and Mistral-7B, adding source responses reduces prediction error by 11.1% relative to a receiver-only predictor, with corrected support in every domain, and by 2.4% overall relative to the source's hard answers alone. A separate probability study uses Qwen2.5-0.5B, with 36% accuracy, as a cheap source: it improves the Brier score of the three larger receivers' probabilities in all 16 prespecified comparisons, though not their accuracy.

What a practitioner gets. Moving an intervention into a new model needs only 56-64 paired support items and no training; a 16-item audit on the receiver then predicts held-out effects (audit gain ρ\rho of .86-.88 at 7-9B; weaker at 27B); and a cheap source can forecast how a larger receiver's outputs change. For binary concepts, class-level weights suffice, which further simplifies the construction. Each component is evaluated on held-out items against matched controls.

Presentation. We agree the paper was dense, and we rewrote the main text around one line of argument: relations recur (Sec. 5.1), transported relations steer every receiver tested (Sec. 5.2, new Fig. 3), the source contributes a class judgment (new Sec. 5.3), and a receiver audit predicts held-out effects (Sec. 5.4). Each section opens with its answer, every coined term is defined at first use, and Table 1 states the question each control answers. Appendix numbers are unchanged. We will also redraw Figure 1 as a three-step pipeline for the final version.

We hope the 27B, architecture, public-task and learned-mapping evidence addresses your main concerns, and respectfully ask you to consider raising the score.

Area_Chair_fuDyOpenReview ↗

Please respond to the authors

Hello!

Just a friendly bump that you should engage with the authors response

Dear Reviewer gkxs, as the discussion closes soon, a short summary may save you time. Your two points are tagged in purple in the right margin of the current PDF.

On model scale and recent families, hidden interventions now cover receivers from 0.8B to 27B parameters. They include three encoder-decoder models and Qwen3.6-27B, a recent model in which most layers use linear attention. On the 27B receiver the contrast fixed before measurement holds at .201 [.141, .261] and .240 [.190, .291], and item accuracy rises from 95.0% to 98.8% (Section 5.2, Figure 3).

On practical use, we added an experiment on the use the paper names (Appendix I.9 and our new general comment). A refusal direction and a sentiment direction found in one model are installed in a model from another family, between Qwen2.5-7B and Mistral-7B, and scored on free generation. The source sends 64 numbers and the receiver needs no labels. With Qwen as receiver, the transported refusal direction raises refusals of harmless instructions from 2% to 98%, the same as the receiver's own direction. Over both behaviors and both directions it reaches 0.87 of the effect of the receiver's own direction.

On real tasks, the interventions on BoolQ and RTE now include strong controls and the learned activation mapping of Oozeer et al., which our construction exceeds there by 1.00 to 2.04 margin units. Output prediction also covers ARC-Challenge, OpenBookQA and HellaSwag, with 512 new held-out questions each, where source responses reduce prediction error by 11.1% relative to a receiver-only predictor.

On presentation, the main text was rewritten around one line of argument, and each section now opens with its answer.

We would be grateful to know whether this addresses your concerns.

I appreciate the authors’ additional evaluations across a broader range of model sizes, including a recent model family. My assessment remains borderline, but I would not object to acceptance if the other reviewers support it.

Reviewer_P99DOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper asks whether one LLM can help construct a predictable activation intervention in another without aligning their hidden coordinates.

Method. For each query, the source model computes signed cosine weights between the query’s “context innovation” and the innovations of K = 56 paired support items. The innovation is the mean-centered difference between the hidden state with the task context vs without it. The receiver then sums its own donor innovations using those weights (Eqs. 2–3). The resulting direction is applied as additive steering or as projection removal, at relative depth 0.75.

Experimental setup. 12 directed paths among Qwen2.5-7B, Llama-3.1-8B, Gemma-2-9B and Mistral-7B, on four synthetic binary tasks, giving 48 path-task cells. The source construction is compared with four matched controls:

  • target-neutral weights
  • a task ridge mapping
  • unrestricted donor permutation
  • label-preserving donor permutation

Receiver score. A “receiver-local score” computed on audit items is correlated with effects on held-out test items.

Other experiments. A small BoolQ/RTE stress test, and a separate experiment that predicts a receiver’s output change from a source’s output change under natural context edits (this one also includes FLAN-T5-XL).

Theory. An L1 bound on proposal error, a Taylor bound on finite-dose effects, and a kernel-inclusion identifiability condition.

Critical strengths
  • Inference is carefully done and reporting is honest.

    • Crossed bootstrap resamples shared models and tasks, which respects the reuse of each model across directed paths.
    • Holm corrections, leave-one-source/receiver/task/model checks (Table 12), and decision rules fixed in advance (App. D.2) are all mentioned and adequately pointed to.
    • The paper reports its own unresolved or failed tests openly: fine donor identity (Fig. 2b), functional matching (Fig. 2c), projection ranking (Table 12), and the response-kernel mechanism (Table 14).
    • Holding receiver access and edit norm fixed across all arms (Table 1) is the right design.
  • The finding that relational structure recurs across models is robust at the representation level.

    • Source-instantiated proposals beat unrestricted derangement in all 48 cells (Fig. 2b).
    • Task-matched recurrence survives removing the label-indicator and neutral-prompt components, and it replicates on disjoint query splits and three depths (Tables 10–11).
    • It also holds in a 27B scale check (L1188–1191).
Critical weaknesses
  • The functional value credited to the source reduces to label information.

    • Label-preserving permutation does as well. Against the label-preserving derangement, the source construction is no better/slightly worse for steering (Table 6: −0.257 and −0.239, with only 13/48 and 20/48 cells positive; Fig. 3). Net decision corrections tell the same story (Table 20: 187 vs 208). Same-access task ridge also ties (Table 6).
    • The paper’s own analysis point the same way:
      • Label-indicator component carries a median 0.9993 of response-kernel energy (L1323–1325).
      • Raw relation weights are almost as similar across tasks as within a given task (0.906 vs 0.939/0.905, L1125–1127).
    • With binary labels and signed weights, Eq. 3 approximately reduces to the receiver’s class-mean contrast, signed by the query’s label. A simple yet representative baseline to be considered, the receiver’s own class-mean direction selected by the source’s predicted answer, is missing from evaluation.
    • The control the source beats is weak. The target-neutral control never sees the query’s context (App. C.2, L829–831). So “source value” (Table 2) partly measures having access to context information at all, not anything specific to the source.
  • Both prediction results are weaker than they are framed.

    • Audit score mostly re-measures the test effect. Its numerator is the same class-balanced secant as the test endpoint, just at a smaller dose on other items (Eqs. 7–8 vs Eq. 27). Small-dose gain alone correlates more strongly than the composite score (Table 13: 0.860/0.875 vs 0.837/0.825). So the “prospective” ranking is close to split-half reliability.
    • Pooled correlation looks driven by one arm. In Fig. 4 it appears driven by the unrestricted-derangement units in the negative quadrant. Among the other three constructions the ordering is visibly weaker.
    • The 32.5% natural-response gain is narrow (Sec. 5.5).
      • It involves no intervention; it measures how much two models’ outputs agree under the same edits.
      • Gain comes mostly from the object-tracking task (Table 28: about 0.9 → 0.4).
      • On ordering, the source-response predictor is at or above 1.0 in three of six directions, pointing it is no better than predicting no change.
Questions for authors
  • How does a label-conditioned baseline compare with the source construction? Here the source supplies only its predicted answer, and the receiver uses its own class-mean direction.
  • What is Spearman ρ within the three non-deranged constructions, or within each cell? How does the composite score compare with simply measuring the β = 0.5 effect on the audit items?
  • What is the intended use case? If the receiver can be run on the query’s context, the target-relational reference needs no source. If it only sees the baseline prompt, what does the source contribute beyond the answer?
  • Two closely related lines of work are missing:
  • Mistral on subject agreement sits at 0.50 clean balanced accuracy and Llama on spatial relation at 0.59 (Table 17). What do margin effects mean for a receiver at chance, and do the conclusions hold without these cells?
4 comments

Thank you for the exceptionally careful reading and for proposing the key class-mean control.

Q1: label-conditioned baseline. We ran it as specified on both Qwen/Mistral directions and all four tasks, holding the 56 donors, receiver access, norms and doses fixed. Symmetric margins at doses .5/1:

  • relational (ours): 5.18/5.44
  • receiver class mean signed by the source's answer: 5.39/5.62
  • class-projected source weights: 5.49/5.74
  • receiver class mean signed by the receiver's own answer: 5.29/5.55
  • neutral weights: 1.63/1.67

You are right: the functional value is class-level, and for binary labels Eq. 3 acts like a source-signed class contrast. The revised PDF says so in the abstract and in a new Sec. 5.3 built on this decomposition (Fig. 3b); the same holds at 27B (App. I.4). Fig. 3b shows the same on public tasks and encoder-decoder receivers. Two points remain. Eqs. 2-3 never read donor labels, yet reach 96-97% of the labelled class-mean effect. And recurrence beyond labels is representational: held-out task-matching gaps of .240 after label projection and .097 after neutral conditioning (Table 10, Holm p=.005/.009p=.005/.009), replicated with a 27B model.

Learned mapping (Oozeer et al., ICML 2025). We adapted their MSE-trained Linear-ReLU-Linear map under matched 64-item support on BoolQ/RTE, both Qwen/Mistral directions. Relational steering exceeds it by 1.83/2.04 margin units (direct application) and 1.00/1.15 (finite-difference application), and ridge by .45 at dose 1, all with simultaneous 95% CIs excluding zero. This is a terminal-state adaptation, not their full all-token pipeline.

Neutral control and Q3 (use). Agreed: all arms edit the same contextual receiver pass, but neutral weights come from baseline-prompt states (App. C.2), so source-minus-neutral measures context-dependent weighting, not source-specific value; the controls above are the informative tests. With full receiver access, the target-relational reference indeed needs no source; we use it only as a reference. Our use case is moving an intervention into a receiver without a learned coordinate map, where our construction beats the learned map. On your second case: in binary tasks the source contributes essentially its class judgment, which helps when the receiver is weaker: on three encoder-decoder receivers, a calibrated source class direction beats the equally calibrated receiver one (+.051/+.101, CIs exclude zero); with equally capable Qwen/Mistral receivers the gap is .005. On the controlled tasks the source's calibrated answer is more accurate than every steered receiver; these are causal testbeds, not end tasks (Sec. 6, App. I.6). In a new multi-question test where the receiver never sees the context, one query-free 64-coefficient source message (a minimum-norm variant of Eq. 3) lets it rebuild its own prefix state and answer four later questions at 93.8%/93.0% (21.9%/24.2% without; 87.5%/85.2% with class-only coefficients). Rendering the decoded facts as text does slightly better (96.1%/97.7%), so what crosses is the context's facts.

Q2: audit ranking. Agreed that the audit re-measures the test endpoint on 16 disjoint items at a smaller dose; the revision presents it as a small-dose measurement on new items and reports gain beside the composite (Sec. 3.4, 5.4), since the composite adds nothing (Table 13). Without unrestricted permutation, the composite gives ρ=.695/.675\rho=.695/.675 (CIs [.11,.96]/[.05,.95]); the audit effect at β=.5\beta=.5 gives ρ=.854/.876\rho=.854/.876 with all four arms and .733/.778 without it. Within cells, the three remaining constructions have mean ρ=.615\rho=.615 (composite; positive in 41/48 and 42/48 cells) and .656/.677 (β=.5\beta=.5 audit).

Natural responses. This assay predicts receiver output changes, with no hidden intervention; Sec. 5.5 now says so. It is not raw agreement: matched wrong-scene or cross-task source outputs raise risk to .929/.940 versus .476. Objects carry 87% of the error reduction, but ordering also improves (8.99%, scene-bootstrap CI excluding zero). For ordering, two directions exceed 1 and one is near 1, while pooled ordering risk is .518; App. I.7 reports their range. Shared errors (Kim et al., ICML 2025) are part of this signal; Sec. 5.5 now cites it. We also extended the assay to ARC-Challenge, OpenBookQA and HellaSwag (512 new held-out questions each): source responses cut the error of predicting receiver output changes by 11.1% versus a receiver-only predictor, in every domain.

Chance-level receivers. There a margin shift measures score sensitivity, not task performance. Dropping the two named cells leaves 42 cells with source-minus-neutral 2.19 [.48,4.08] and 2.34 [.54,4.37] (Sec. 5.2).

Sec. 5.2-5.3 and new Fig. 3 now include the class controls and learned-mapping comparison, and App. I adds every control, direct answers and the 27B receiver. We will add the new code to the supplement. We would be grateful if you reassessed the score in light of these answers.

Area_Chair_fuDyOpenReview ↗

Please respond to the authors

Hello!

Just a friendly bump that you should engage with the authors response

Dear Reviewer P99D, your review shaped this revision more directly than any other, so we would value your reading of it before the discussion closes. Your points are tagged in green in the right margin of the current PDF.

The label-conditioned baseline you proposed is in the paper, run as you specified it (Table I1). It matches our construction, and the new Section 5.3 explains why with an exact split of the transported edit into a class part and a within-class part. We agree with your reading that the functional value is class-level, and the abstract now says so. What follows from it is that a source's class judgment, delivered as label-free weights over shared examples, is enough to steer a receiver whose coordinates it never sees.

Against the learned activation mapping of Oozeer et al., fit on the same 64 paired items, our construction is larger by 1.00 to 2.04 margin units on BoolQ and RTE, with simultaneous CIs above zero (Table I2). At dose 1 it also exceeds same-access ridge on the public, encoder-decoder and 27B receivers.

On the ranking, gain alone gives .745 and .775 without the unrestricted control, and within cells the three remaining constructions are ordered with mean rho between .59 and .66, depending on the audit measure. Section 5.4 now states that the composite adds nothing beyond gain. Without the two receivers at chance, the source-minus-neutral effect is 2.19 and 2.34 with CIs above zero (Section 5.2). Kim et al. is cited in Section 5.5.

On your Q3, Section 6 states the intended use and that the source's own answer is more accurate on these binary tasks. Appendix I.9 now tests that use on something a source cannot simply state as an answer, a steering direction for refusal and for sentiment, scored on free generation. The transported direction reaches 0.87 of the effect of the receiver's own labeled direction. We also report there that a ridge map on the same 64 texts is slightly higher, and that a class contrast with source-assigned classes matches ours, as your class-level reading predicts.

If any of these does not settle the point you raised, we would be glad to know which.

Thanks a lot for addressing my rather granular comments clearly and in a visibly well-presented way both here and on the manuscript. While adapting claim generality does indeed reduce novelty scope, I feel that the reformulated claims along with presented results + the general premise of relational transport and how it is investigated in the paper are a good compromise for acceptance in my opinion. I do raise my score in light of that to a 3.

Reviewer_UPTJOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper investigates whether relational information from a source language model can guide interventions in another model without explicit representation alignment. It proposes a signed relation-weighting mechanism for constructing receiver-native intervention directions and a receiver-local score for predicting intervention effectiveness. Empirical studies across four Transformer families report recurring relational structure, improved additive steering, and correlations between predicted and held-out intervention effects.

Critical strengths

The use of matched controls, held-out intervention outcomes, and multiple model pairs provides a systematic evaluation of cross-model information transfer.

Critical weaknesses

W1. The source-relational construction does not consistently outperform label-preserving donor permutations, suggesting that much of the observed benefit may arise from shared label-level structure rather than fine-grained source-specific relations.

W2. The receiver-local score's predictive performance is largely driven by local gain, making it unclear whether the proposed relational framework provides substantial additional predictive information beyond simpler receiver-only measurements.

W3. Most intervention experiments use closely related language model architectures and a limited set of controlled tasks, leaving cross-architecture and broader behavioral transfer insufficiently validated.

Questions for authors

Q1. Can the authors explain why label-preserving donor permutations achieve comparable intervention effects and clarify what source-specific information is actually transferred?

Q2. How much predictive improvement does the proposed qualification score provide over receiver-local gain alone on held-out interventions?

Q3. Does the proposed approach maintain its effectiveness across substantially different architectures, model scales, and intervention locations?

6 comments

Thank you for the careful evaluation and three precise questions; we answer each directly.

Q1/W1: why label-preserving permutation matches, and what transfers. Your reading is correct; we now test it directly. Label-preserving permutation keeps each class's total weight, and a binary decision margin depends mainly on the class-contrast part of an edit. We added class-level controls on both Qwen/Mistral directions and all four tasks, with identical donors, access, norms and doses. Symmetric margins at doses .5/1 are 5.18/5.44 for our construction, 5.39/5.62 for the receiver class-mean direction signed by the source's answer, 5.49/5.74 for class-projected source weights, and 1.63/1.67 for neutral weights. The steering effect is therefore class-level: within-class relations add no measurable steering value on these binary tasks. Across settings, within-class effects are small and change sign: label-preserving permutation is slightly stronger in the submitted cells (Table 6) and on encoder-decoder receivers (by 8-9%), and weaker on public BoolQ/RTE (source minus label-preserving .66/.62, simultaneous 95% CIs [.12,1.20]/[.11,1.12]).

What is source-specific is twofold. At the representation level, task-specific relations persist after removing query- and donor-label components (Table 10: held-out same-minus-cross-task gap .240, Holm p=.005p=.005; .097, p=.009p=.009, after also removing neutral-prompt relations), and again with a 27B model (.215). Functionally, the source supplies class information the receiver may lack: on encoder-decoder receivers, a calibrated source class direction beats the equally calibrated receiver class direction (+.051/+.101, CIs exclude zero), whereas with equally capable Qwen/Mistral receivers the gap is .005 (CI includes zero). The relational weights reach 96-97% of the labelled class-mean effect without reading any donor label. Fine relations can matter for richer endpoints: when one source message must serve four later questions, within-class coefficients raise accuracy from 87.5%/85.2% (class-only) to 93.8%/93.0% (significant against a norm-matched class control in one direction; text rendering does slightly better).

Q2/W2: predictive value beyond local gain. None: gain alone is at least as predictive as the composite (Table 13: .860/.875 vs .837/.825; paired differences .02/.05, CIs include zero). The revision reports gain beside the composite (Sec. 3.4, 5.4, Table 2) and states that gain carries the ordering, with stability and off-task response as diagnostics. This matches the paper's central claim: Eq. 5 shows that relational similarity alone cannot determine a receiver's response, and representation similarity to the receiver's own relational reference correlates only .41-.49 with steering effects (App. E.7), versus .86-.88 for audit gain. The relational step proposes; the receiver measurement predicts. Nor is the ordering driven by one weak arm: without unrestricted permutation, ρ\rho stays at .70/.68 (composite) and .75/.77 (gain), and within each source-receiver-task cell the remaining three constructions are ordered with mean ρ=.61\rho=.61, positive in 41/48 and 42/48 cells. It is weaker at 27B (App. I.5).

Q3/W3: architectures, scale, locations. (i) Architecture, new: hidden interventions on three encoder-decoder receivers (FLAN-T5-Large, mT0-Large, FLAN-T5-XL; 0.8-3B) from a Qwen source, on all four tasks (12 cells). Source weights beat neutral weights by .126/.251 and same-access ridge by .035/.069 margin units at doses .5/1, all simultaneous CIs excluding zero; accuracy rises from 75.0% to 78.1%/77.1% (Fig. 3). (ii) Learned mappings, new: on BoolQ/RTE in both Qwen/Mistral directions, relational steering exceeds an adaptation of the learned activation mapping of Oozeer et al. (ICML 2025) by 1.83/2.04 (direct) and 1.00/1.15 (finite-difference application), CIs excluding zero. (iii) Scale, new: hidden interventions on Qwen3.6-27B, a recent, mostly linear-attention model. Our prespecified source-minus-neutral contrast holds, .20/.24 (simultaneous CIs [.14,.26]/[.19,.29]), concentrated in two of four tasks; source beats ridge by .15/.16, and class controls again match ours (Sec. 5.2, App. I.4). (iv) Locations: fixed cuts at relative depths .625/.875 (Table 18) give additive effects correlating .69-.93 with the central cut. (v) Broader behavior, new: on ARC-Challenge, OpenBookQA and HellaSwag (512 new questions each), source responses cut the error of predicting receiver output changes by 11.1% versus a receiver-only predictor, in every domain.

Revisions (uploaded PDF). We rewrote the main text around these answers: new Sec. 5.3 explains the class-level result (Q1), Sec. 5.4 reports gain beside the composite (Q2), and Sec. 5.2 with new Fig. 3 covers every receiver and control (Q3); App. I gives all new tables. We will add the new code and analyses to the supplement.

We hope these direct answers to Q1-Q3 and the added evidence lead you to reconsider your score.

Reviewer_UPTJOpenReview ↗

Make the edits visible

Thanks for your edits. Can you please make your edits in any color (my portion)? It will help me identify the edits quickly.

Thank you so much for your prompt reply. We've added color and annotations to the PDF, and we hope that will be helpful. Please feel free to let me know if you still have any further questions or concerns. Thank you again for your willingness to reassess.

In the PDF, blue text marks where each point is addressed, and the right margin names the reviewer and point (UPTJ, P99D, gkxs).

Comments on the paper

2 comments

Revised PDF: main text reorganized around the reviewers' questions, and a new 27B result

We thank all reviewers. We have uploaded a revised PDF. Reviewer gkxs found the paper hard to follow, and the new results changed what the paper can say, so we rewrote the main text around one line of argument instead of adding patches. Appendix A-H, Tables 3-37 and Eqs. 1-8 keep their submitted numbers; new studies are in Appendix I (Tables I1-I9).

How the main text now reads.

  • Sec. 5.1: relations recur across model families. Fig. 2c now shows task-matched recurrence after label and neutral-prompt removal (submitted Table 10).
  • Sec. 5.2: transported relations steer every receiver tested: four decoder-only families, three encoder-decoder receivers, and Qwen3.6-27B. New Fig. 3 places every control from five studies on one scale, including same-access ridge and an adaptation of the learned activation mapping of Oozeer et al. (ICML 2025) (UPTJ Q3, P99D, gkxs).
  • Sec. 5.3 (new): the source contributes a class judgment. A decomposition of the transported edit explains why label-preserving permutation matches, and class controls confirm it (UPTJ W1/Q1, P99D Q1).
  • Sec. 5.4: the receiver's response predicts held-out effects. Audit gain is reported beside the prespecified composite (Table 2), with the ordering without the unrestricted control, within cells, and at 27B (UPTJ W2/Q2, P99D Q2).
  • Sec. 5.5 and Sec. 6: natural-response endpoint, public extension, Kim et al. (ICML 2025), intended use, and direct source answers (P99D).
  • Presentation (gkxs): each section opens with its answer; coined terms are defined at first use ("innovation" is now "context shift"); Table 1 states the question each control answers. The submitted Fig. 3 is now Fig. 6 in App. E.4.

New result: hidden interventions on a 27B receiver. Qwen3.6-27B is a recent model in which 48 of 64 layers use linear attention. Sources were Qwen2.5-7B and Mistral-7B on the four controlled tasks, with the submitted prompts, scorer, hooks, localization, norm matching and doses. Before any measurement we fixed one primary contrast, source relational minus target-neutral at both test doses, supported only if both simultaneous 95% lower bounds exceed zero (55 method pairs x 2 doses; 10,000 paired query resamples). The receiver first passed zero-hook identity, reference-stability and left-padding checks.

  • The primary contrast holds: .201 [.141,.261] and .240 [.190,.291], positive for each source. The source adds most where the receiver's neutral organization is weakest (spatial relation and subject agreement, +.35 to +.64); on entity binding and two-hop precedence, neutral weights are as effective.
  • Source minus ridge is .149 [.105,.193] and .155 [.111,.200]. Item accuracy rises from 95.0% to 98.8% at dose 1, versus 93.8% with neutral weights (difference 5.0 points [0.9,9.1]).
  • Class controls again match our construction, consistent with Sec. 5.3; the calibrated receiver class contrast is slightly stronger at dose 1 (.063 [.002,.123]).
  • Audit gain orders the four original constructions (ρ\rho = .51-.58, CIs exclude zero) but does not separate the three non-deranged ones, whose effects lie within .16 margin units (ρ\rho = .09-.22). Sec. 5.4 reports this.

In the PDF, blue text marks where each point is addressed, and the right margin names the reviewer and point (UPTJ, P99D, gkxs).

New experiment: refusal and sentiment steering moved between Qwen and Mistral without a map

All three reviews asked, in different words, what the method is for. gkxs asked about practical use, UPTJ (W3) about broader behavioral transfer, and P99D (Q3) about the intended use. We ran that use directly. The result is in the revised PDF as Appendix I.9 (Table I10, Figure 7, example generations in Table I11) and is summarized in Section 6. The run script is in the updated supplement; its header fixes the endpoint, the dose rule and the comparisons, and it was written before any generation was scored.

The task is to take a steering direction found in one model and install it in a model from another family. We did this for two behaviors between Qwen2.5-7B and Mistral-7B, in both directions, and scored free generation on 64 held-out prompts. One behavior is refusal: the refusal direction of Arditi et al. is added while the receiver answers harmless instructions. The other is sentiment. The source sends 64 numbers, its relation weights over 64 unlabeled texts that both models read, and the receiver combines its own states of those texts. The receiver uses no labels and receives no source activation. Every direction is rescaled to one norm, that of the receiver's own direction, so directions differ only in where they point.

First, the transfer works. Pooled over the four cells, the transported direction reaches 0.87 of the effect of the receiver's own direction, which needs labeled data in the receiver (95% interval 0.72 to 1.04), and the two do not differ significantly. A random direction of the same norm has no effect. With Qwen as receiver, the transported refusal direction raises the refusal rate on harmless instructions from 2% to 98% at twice the reference norm. The receiver's own direction also gives 98%, and a random direction leaves it at 2%. This dose lies outside the pre-set primary endpoint for that cell, for a reason Appendix I.9 explains, and Figure 7 shows every dose. Asked for a new way to mark a place in a book, the steered Qwen answers "I'm sorry, but I can't assist you with that request."

Second, we report the comparison with fitted maps as it came out. With the same 64 texts, ridge is slightly higher than ours (pooled effect .117 against .094; the interval of the difference, .002 to .044, excludes zero), and the learned activation mapping of Oozeer et al. is slightly lower (.073; the difference does not exclude zero). Appendix I.9 shows why ridge is close: our construction is the strong-penalty limit of ridge, with nothing to fit or tune. With twelve times more paired texts the maps improve to .131 and .139, while ours gives .105. A fitted map is therefore the better tool when many paired activations can be shared. Relational transport serves the case where only a few numbers can cross between models.

Third, the weakest of the four cells is sentiment from Qwen to Mistral, where ours reaches .034 against .098 for the receiver's own direction. In the other three cells the ratio lies between 0.90 and 1.38. A class contrast with classes assigned by the source does not differ from ours, which agrees with the class-level account of Section 5.3.

Finally, the receiver audit carries over to an endpoint it does not re-measure, a point P99D raised. A 16-prompt audit that needs forward passes only ranks the generation effects of the twelve directions with a mean Spearman correlation of 0.58 (interval 0.30 to 0.74). The cosine with the receiver's own labeled direction gives 0.71, but it needs the labeled direction that the transfer is meant to replace.

The study covers two behaviors and one pair of model families.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.