acceptodds
Under review as a conference paper at ICLR 2027

Calibrating Activation Steering with Sparse Behavioral Feedback

Abstract

Activation steering can control a frozen language model through an additive direction, yet a coefficient that works for one checkpoint can produce little change in another. We study how to turn this model dependence into a calibration problem. Across 15 instruction-tuned models and five personality attributes, residual-stream scale predicts sensitivity to a shared coefficient grid: a mid-layer RMS feature achieves a held-out-family rank correlation of 0.83. Within each model, this scale is stable across attributes, suggesting a reusable measurement for identifying candidate dose ranges. Behavioral feedback then selects a directional operating point within a range. In a retrospective evaluation of measured grids on five models, six rated responses from one calibration scenario select a dose that transfers to the remaining scenarios. At the same feedback budget, expanding the candidate grid raises the mean held-out signed contrast from 0.24 on the original grid to 1.27, with improvements in all 25 model–attribute averages. Six ratings retain 89.6% of the effect obtained using 30 ratings on the same scenario. These results separate model-level sensitivity from attribute-level dose selection and establish sparse behavioral calibration as a practical way to recover useful steering from an existing direction.

Primary area:
interpretability and explainable AI
Keywords:
activation steering, steerability, representation engineering, interpretability, model intervention, layer selection, dose calibration, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_UJfJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates why a fixed activation-steering coefficient works on one checkpoint but can produce little change in another one. They studied 15 instruction-tuned models and give personality attributes, and within each model they find the scale is stable across attributes.

Critical strengths

The problem is interesting and well motivated for steering literature.

RMSnorm derivative gives insights about why scale is informative.

Exhaustive calibration splits are provided.

Critical weaknesses

Feedback-based selection seems to add little over trivial (zero-feedback) baselines. In Tab 2 it seems like choosing the largest expanded amplitude beats the six-rating selector on 4 of 5 models, and the selector's small mean advantage seems to be coming entirely from Qwen3.

The last two authors’ surnames in one reference seems to be fabricated. The reference in paper:

Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Ma, and Yixin Liang. Evaluating and inducing personality in pre-trained language models. Advances in Neural Information Processing Systems, 36, 2024.
 The correct author names for last two authors:
Chi Zhang (instead of Chi Ma) and Yixin Liang (instead of Yixin Zhu): https://proceedings.neurips.cc/paper_files/paper/2023/file/21f7b745f73ce0d1f9bcea7f40b1388e-Paper-Conference.pdf

Questions for authors

Primary judge is Gemma-2-2B-it, does the main results hold under a different judge?

2 comments
AuthorsOpenReview ↗

Response to Reviewer UJfJ

Thank you for highlighting our problem motivation, RMSNorm derivation, and exhaustive split design.

1. Reference Correction (Jiang et al., 2023)

Thank you for catching the surname error in Jiang et al. (NeurIPS 2023, Vol. 36, pp. 10622–10643) and for linking the official NeurIPS PDF. Per the linked paper (noting that your review text also inadvertently swapped the order in "Yixin Liang (instead of Yixin Zhu)"), we have restored the exact official authorship (Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Zhang, and Yixin Zhu, 2023) on Page 11 (Lines 582–587).

2. Weakness 1: Why Sparse Behavioral Selection Matters Over the Zero-Feedback Largest Baseline

Why use behavioral feedback if Largest gives a slightly higher 5-trait average on 4 of 5 models? Our 25-curve audit (App. G, Tab. 10, Page 19), zero-self-judging reanalysis (App. F, Tab. 9), and 7B-judge replication (App. I, Tab. 21) clarify how the two stages work together:

  1. Interior Peaks and Sign Reversals Occur Across 3 of the 5 Models (8 of 25 Curves), Not Just Qwen3-0.6B (App. G, Tab. 10): Across all 25 dose curves Gt(ai;S)G_t(a_i; S), 8 of 25 peak strictly at an interior amplitude (a1a_1 or a2a_2) across 3 of the 5 models, and 3 reverse sign (G(a3)<0G(a_3) < 0) at a3a_3:
Model & Trait (from App. G, Tab. 10)Grid (a1,a2,a3)(a_1, a_2, a_3)G(a1)G(a_1)G(a2)G(a_2)G(a3)G(a_3) (Largest)Curve Behavior at Largest Dose a3a_3
Qwen3-0.6B Agreeableness / Neuroticism(18,36,54)(18, 36, 54)+1.975 / 1.0501.0501.2251.225 / +1.300−0.650 / −1.200Sign reversal (−2.625−2.625 / −2.500−2.500 collapse)
Qwen3-0.6B Openness / Conscientiousness(18,36,54)(18, 36, 54)2.0862.086 / 1.9001.900+2.200 / +2.5501.0571.057 / 1.1251.125Interior peak at a2a_2 (−1.143−1.143 / −1.425−1.425 drop)
Qwen2.5-1.5B Agreeableness / Neuroticism(10,20,30)(10, 20, 30)0.3250.325 / 0.4000.400+0.450 / +1.050−0.500 / 0.7500.750Sign reversal on A (−0.950−0.950) & interior peak on N
Qwen2.5-7B Extraversion / Neuroticism(10,20,30)(10, 20, 30)1.5751.575 / +1.150+2.075 / 0.8750.8751.7751.775 / 0.7750.775Interior peaks at a2a_2 and a1a_1
  1. Resolving Unseen Curve Shapes Before Deployment:
    • First, on the uncalibrated original grid ({2,4,6}\{2, 4, 6\}), picking the largest amplitude gives only 0.406 (Tab. 6). Expanding the grid lifts Largest to 1.172, showing the value of Stage 1 (scale-motivated range expansion).
    • Second, before collecting ratings, a practitioner does not know whether a given model–trait curve is monotonic or non-monotonic. Deploying a3a_3 without feedback risks severe over-steering collapse (0.5560.556 vs. 1.9081.908 on Qwen3-0.6B, and negative reversal on 3 traits).
    • With 6 ratings (11 scenario, r=1r=1 repetition), the selector trades a small single-sample estimation variance cost on monotonic curves (retaining 87%–89% of peak effect) to prevent a −1.352 point collapse on non-monotonic curves. When repetitions increase from 6 to 30 ratings (reducing single-sample noise), the selector's overall advantage over Largest grows from +0.095 (1.2661.266 vs. 1.1721.172) to +0.241 (1.4131.413 vs. 1.1721.172) across all 5 models.
    • Moreover, once Gemma-2-2B (self-judging) is excluded (App. F, Tab. 9), the 6-rating selector's mean advantage over Largest more than doubles from +0.095+0.095 to +0.193+0.193 (1.1401.140 vs. 0.9470.947; reaching +0.334+0.334 [1.2811.281 vs. 0.9470.947] at 30 ratings), and also exceeds Uniform (1.021).

3. Question 1: Do the Main Results Hold Under a Different Judge?

Yes. Beyond our 1,260-response check (Spearman ρ=0.951\rho = 0.951 on model rankings), we used Qwen2.5-7B-Instruct to score all 13,650 stored responses across both complete 5-model grids (6,825 Orig + 6,825 Exp) (Sec. 5.5, Page 10; App. I, Tabs. 21–22):

  • Judge Agreement (Tab. 22): Across all 13,650 responses, Qwen2.5-7B-Instruct and Gemma-2-2B-it agree within ±1\pm 1 point on 93.2% of ratings (92.6% Exp, 93.8% Orig; Pearson r=0.775r = 0.775, Spearman ρ=0.781\rho = 0.781; 0.7970.797 and 0.8030.803 on Exp).
  • Transfer Results (Tab. 21): Under Qwen2.5-7B-Instruct, the 6-rating expanded selector improves all 25/25 model–trait pairs over the original grid (2.210 vs. 0.472, gain +1.738, 95% CI [1.172,2.440][1.172, 2.440]; 2.408 at 30 ratings) and beats both Largest (2.200; 1.953 vs. 1.853 excluding Gemma; 2.102 vs. 2.029 excluding Qwen2.5-7B) and Uniform (2.069; 1.953 vs. 1.771 excluding Gemma; 2.102 vs. 1.950 excluding Qwen2.5-7B). On Qwen3-0.6B, 6 ratings again prevent largest-dose collapse (2.620 vs. 1.332, +1.288).

Thanks for your rebuttal, I increased my score to Weak Accept.

Reviewer_KimeOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

One open choice when using steering vectors is assigning a steering coefficient. This paper presents a method to choose this parameter for contrastive activation steering. First, using an evaluation over personality dimensions (Big 5 traits), they sweep over a grid of coefficient values and show that equal values lead to different effects across all models tested (n=15). Second, they show that the size of the model’s activations is a predictor of the steering effect. Third, they choose a coefficient by testing three hand-picked candidate coefficients: they score the model steered toward and away from the target concept, then select the coefficient value with the largest contrast between the two scores. The effect of the selected coefficient transfers to held-out scenarios.

The scenarios and contrastive pairs are built and evaluated with synthetic data. Judging is done with Gemma-2-2B-it.

Critical strengths

None.

Critical weaknesses

W1. This paper has a hallucinated reference:

Guangyuan Jiang, Manjie Xu, Song-Chun Zhu, Wenjuan Han, Chi Ma, and Yixin Liang. Evaluating and inducing personality in pre-trained language models. Advances in Neural Information Processing Systems, 36, 2024.

See here https://openreview.net/forum?id=I9xE1Jsjfx that the last two authors are hallucinated.

W2. I assume the Judging in Line 290 refers to judging of the rollouts. If so, I think using a 2B instruct model from a year ago (Gemma-2-2b-it) is very questionable. I would expect this model to be very weak for this task. It is also judging itself in your main experiment (fig 2).

W3. There is a large reproducibility and validity gap in how these steering vectors might have been built, and how they are being evaluated. Nowhere in the main body or the appendix are mentioned the datasets of contrastive pairs used to build the vectors, or what items were used to evaluate the system across the personality dimensions described.

Note 1. This paper was very difficult to read. Most paragraphs, including the abstract, introduce new important terms or synonyms for the same thing.

Note 2. I am inclined to give a shorter review here because under the guidelines, this paper may get desk rejected. However, if this is not the case, and the AC thinks the review process should continue, I will engage in discussion with the authors and other reviewers.

Questions for authors

Q1. Could you give more information on what data the contrastive pairs are built from? As well as the other evaluation items? Why not use something standard like BIG5-CHAT (Li et al 2025, https://aclanthology.org/2025.acl-long.999/), was there a strong motivation to use synthetic data?

Q2. If all data is synthetic, could you at least cite what definitions you use for the personality traits? For instance Hoppe et al 2026 (https://openreview.net/pdf?id=6TuaAw1DkF) builds them using the BIG5-CHAT dataset above.

Q3. Can you back up your choice of judge model? (related to W2)

4 comments
AuthorsOpenReview ↗

Response to Reviewer Kime

Thank you for the thorough review and pointers to recent work.

1. W1 & Ethics Concern: Correction of Jiang et al. (2023)

We apologize for the co-author surname error in Jiang et al. (NeurIPS 2023, Vol. 36, pp. 10622–10643), a real paper cited in Sec. 2 (Line 124) for its MPI framework. In .bib formatting, Chi Zhang, Yixin Zhu were mistakenly transcribed as Chi Ma, Yixin Liang. We restored the exact citation (Page 11, Lines 582–587) and verified all references.

2. W2 & Q3: Zero Self-Judging (E1) and Full-Grid Qwen2.5-7B-Instruct Re-Scoring (E2)

We address both self-judging and judge capability in Sec. 5.5 (Pages 9–10), App. F (Tabs. 8–9), and App. I (Tabs. 21–22):

  1. Excluding Gemma-2-2B Completely (Zero Self-Judging):

    • 14-Model Survey (App. F, Tab. 8): Removing Gemma-2-2B leaves 14 models across 4 families (Qwen, LLaMA, Mistral, Phi). Scale remains predictive across all 5 features:
      • Mid-layer RMS: LOFO ρ=0.820\rho = 0.820, LOMO ρ=0.723\rho = 0.723, median AE =0.234= 0.234 (mean AE =0.322= 0.322).
      • Probe / Injection RMS: LOFO ρ=0.829\rho = 0.829 / 0.8730.873, LOMO ρ=0.758\rho = 0.758 / 0.7670.767 (median AE 0.3640.364 / 0.2400.240, mean AE 0.8350.835 / 0.3860.386).
      • Probe / Injection RMS norm: LOMO ρ=0.864\rho = 0.864 / 0.8460.846, LOFO ρ=0.807\rho = 0.807 / 0.8860.886 (median AE 0.3250.325 / 0.2210.221, mean AE 0.6580.658 / 0.3630.363).
    • 4-Model Calibration (App. F, Tab. 9): On the 4 non-Gemma models, 6-rating selection raises held-out signed contrast from 0.290 (Orig 6) to 1.140 (Exp 6; Exp 30 =1.281= \mathbf{1.281}), a paired gain of +0.851 (95% CI [0.537,1.245][0.537, 1.245], 20/20 pairs improved). Removing Gemma-2-2B more than doubles the 6-rating selector's advantage over Largest from +0.095+0.095 to +0.193+0.193 (1.1401.140 vs. 0.9470.947; +0.334+0.334 at 30 ratings) and beats Uniform (1.0211.021).
  2. Full-Grid Independent Re-Scoring (13,650 Responses) by Qwen2.5-7B-Instruct (App. I, Tabs. 21–22):

    • Rubric-grounded scoring: The judge scores each 200-token response on a 1–5 rubric against 5 high and 5 low indicators (App. E, Page 17) at temp 0.10.1, giving explicit ratings on 13,648 / 13,650 responses.
    • Inter-judge agreement (Tab. 22): Across both 5-model grids (6,825 Orig + 6,825 Exp = 13,650), Qwen2.5-7B-Instruct and Gemma-2-2B-it agree within ±1\pm 1 point on 93.2% of responses (92.6% Exp, 93.8% Orig), with Pearson r=0.775r = 0.775 (0.797 Exp, 0.739 Orig), Spearman ρ=0.781\rho = 0.781 (0.803 Exp, 0.739 Orig), and MAD =0.685= 0.685 (0.729 Exp, 0.640 Orig), plus Spearman ρ=0.951\rho = 0.951 on 14-model sensitivity rankings (Line 472).
    • Full replication under Qwen2.5-7B-Instruct (Tab. 21):
      • All 5 models: Exp 6 (2.210) & Exp 30 (2.408) beat Orig 6 (0.472) by +1.738 ([1.172,2.440][1.172, 2.440], 25/25 improved), beating Largest (2.200) & Uniform (2.069).
      • Exclude Gemma-2-2B (4 models): Exp 6 (1.953) beats Orig 6 (0.538) by +1.415 ([1.034,1.796][1.034, 1.796], 20/20 improved), and beats Largest (1.853, +0.101+0.101) & Uniform (1.771, +0.182+0.182).
      • Exclude Qwen2.5-7B (4 models): Exp 6 (2.102) beats Orig 6 (0.360) by +1.743 ([1.034,2.582][1.034, 2.582], 20/20 improved), and beats Largest (2.029) & Uniform (1.950).

3. W3, Q1, Q2 & Note 1: Complete Dataset Card, Grounding, and BIG5-CHAT / Hoppe et al.

  1. Complete Dataset Card (App. H, Pages 20–25, Tabs. 11–20): We transcribed all 50 extraction contexts (1010/trait), 40 high/low instructions (44 High & 44 Low/trait; 10×4×4=16010 \times 4 \times 4 = 160 paired rows, or 8080 unique signed prompts/trait), 39 evaluation scenarios (77 O + 88 each for C/E/A/N, generated with no persona system prompt), and 50 High/Low indicators verbatim into App. H (e.g., Openness H1/L1+X1: "You are extremely open..." vs. "You are very practical..." on "What do you think about modern abstract art?"; disjoint eval S1: team debating switching from Python to an unfamiliar functional language). Extraction and evaluation texts have zero string overlap, and scenario-grouped linear probing (App. D, Tab. 7) achieves 0.965 mean accuracy.
  2. Grounding & BIG5-CHAT (Li et al., 2025) / Hoppe et al. (2026) (Secs. 2 & 4, Lines 128–134, 280–289): Our rubrics operationalize Big Five domains from IPIP-NEO and BFI (Goldberg, 1999; John et al., 2008). Whereas BIG5-CHAT studies SFT/DPO dialogue training and Hoppe et al. (2026) trains sequential multi-trait probes, our fixed-scenario design isolates cross-model amplitude sensitivity and matched-budget dose selection (Y(+a)−Y(−a)Y(+a)-Y(-a)) across 15 models. Terminology is unified in Sec. 3 (α∈R\alpha \in \mathbb{R}, amplitude a>0a>0, grid Cm\mathcal{C}_m).

W1. Thank you for correcting it. Though I disagree that this is a transcription issue.

W2. I appreciate that the authors use a new judge and exclude self-judging. I still cannot understand why you initially used an outdated weak judge (gemma-2-2b), and now you have used another outdated and weak model (qwen2.5-7b, larger in size than the gemma model but still small for an LLM judge). Is using a judge with these characteristics a component of the experiment somehow, or am I misunderstanding some parts of the experiment that require this design choice? LLM Judges can be unreliable and high variance, so if you're going to use agreement of two weak judges, please state why. Could you motivate the choice?

W3,Q1,Q2. Thank you for the new Appendix H. You still do not explain where the texts used for the pairs come from (I cannot find this in the revision or in the new App H). Were they generated by an LLM and if so how? If they are fully synthetic, how did you ensure the contrastive pairs represent the target trait?

Note: This has been a very difficult rebuttal to read, with dense formatting and little prose explaining the results.

I will maintain my scores while my main concerns remain unsatisfied: (i) motivation of the use of weak judges; (ii) provenance of the text used for contrastive pairs, and validation that they represent the concept (trait) they are supposed to steer towards.

AuthorsOpenReview ↗

Follow-up response

Dear Reviewer Kime,

Thank you for your follow-up comments and for clarifying the two remaining questions. We address the evaluator design and the provenance and grounding of the contrastive prompts below.

1. Why these judges, and what evidence supports the evaluation?

We initially selected Gemma-2-2B-it to enable fast experimental iteration and rapid validation across the 15-model survey, while keeping the evaluator open-weight and reproducible. Crucially, the calibration task is not open-ended complex reasoning: the evaluator performs a constrained, structured judgment by scoring a single response against five explicit high-trait and five explicit low-trait behavioral indicators on a fixed 1–5 scale, without access to the generating model’s identity or the steering coefficient. Using fixed, open-weight checkpoints avoids the drift of proprietary APIs and ensures deterministic reproducibility, while pairing Gemma with Qwen2.5-7B-Instruct tests stability across two distinct model families. Appendix E specifies the complete scoring prompt and protocol.

For this task, the decisive evidence is whether the calibration findings hold when the primary evaluator is replaced. Qwen2.5-7B-Instruct independently scored all 13,650 responses across both calibration grids, and its ratings replaced Gemma’s scores for both dose selection and held-out evaluation. Thus, the second judge determines which amplitude is selected and measures how that choice transfers to held-out scenarios.

Under Qwen scoring, moving from the original to the expanded grid at the same six-rating selection budget increases mean held-out signed contrast from 0.472 to 2.210, with improvements in all 25 model–trait averages. The paired gain is 1.738, with a 95% model-bootstrap confidence interval of [1.172, 2.440]. Excluding Qwen2.5-7B itself preserves improvements in all 20 remaining model–trait averages (Appendix I, Table 21). On the original-grid sensitivity survey, the two judges also preserve the model-level effect-range ordering, with Spearman correlation 0.951 on the independently scored subset (Appendix E). Together with the judge-free linear probe checks (Appendix D), these results show that the findings are not an artifact of a single small judge.

2. Where do the contrastive texts come from, and how are they grounded in the target traits?

The instructions, extraction contexts, evaluation scenarios, and rubric indicators are fixed text configurations frozen prior to all experiments and distributed in the supplementary code (src/prompts/personality_config.py, src/prompts/contrastive_prompts.py, src/evaluation/bfi_44_scale.py, and analysis/evaluation_prompts.json). These templates were drafted with LLM assistance and then manually curated and filtered by the authors against the IPIP and BFI personality definitions cited in Section 4. The experimental pipeline loads and combines these predefined texts rather than calling an LLM to generate new prompts or rubrics at runtime. Appendix H reproduces all configured texts and indicators verbatim.

Each extraction pair shares the same user context and differs only in its high- versus low-trait system instruction, and Equation (1) extracts the steering direction from the mean difference between the corresponding last-token activations. In the Cartesian construction documented in Appendix H, ten contexts and four high/four low formulations yield 160 paired rows per trait (80 unique sign-specific prompts); Appendix A records the exact collection sizes used for each checkpoint.

We validate that these curated pairs capture the target traits at both the representation and behavioral levels. First, scenario-grouped linear probing holds out an entire extraction context per fold and achieves 0.965 mean accuracy across 45 available collections, confirming that the high/low conditions induce a consistent, context-generalizable representation at the evaluated layers (Appendix D, Table 7). Second, behavioral evaluation uses 39 distinct scenarios without persona system prompts and with zero exact-string overlap with the extraction contexts. Injecting the extracted directions shifts held-out behavior in the intended direction across scenarios, and the full-grid Qwen replication confirms that this effect persists under an independent scorer.

I reply to the authors’ response hoping it improves a future draft of this paper. I will still maintain my scores because my concerns are not alleviated.

1. Judges

Crucially, the calibration task is not open-ended complex reasoning: the evaluator performs a constrained, structured judgment by scoring a single response against five explicit high-trait and five explicit low-trait behavioral indicators on a fixed 1–5 scale, without access to the generating model’s identity or the steering coefficient.

This is a weak argument for your judge choice. If the main characteristic you cared about was being able to obtain a single response against five explicit choices, you could also use BERT that is probably faster and cheaper

the decisive evidence is whether the calibration findings hold when the primary evaluator is replaced. Qwen2.5-7B-Instruct independently scored all 13,650 responses across both calibration grids, and its ratings replaced Gemma’s scores for both dose selection and held-out evaluation. Thus, the second judge determines which amplitude is selected and measures how that choice transfers to held-out scenarios.

Calibration between two weak judges does not make your evaluation valid. A stronger argument could be doing some blind human annotations and measuring agreement against that.

2. Synthetic data
Thank you for sharing details. This is an important part of the experiments that I recommend they be included in the main body in some capacity.

Reviewer_48R1OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the selection of scaling factors for activation steering in transformer-based language models. The general idea is that steering requires choosing both a direction, and a scaling factor; the scaling factor is hypothesized to vary across different models. Experiments in this paper show that the degree of scaling required for the steering vector can be predicted from the norm of the activations at different hidden layers, computed across a collection of contrastive pairs that might be used to define a steering vector. Empirically, the behavior for scaling factors is fairly consistent for a given model regardless of which attribute is being targeted (though the experiment is limited to five target "personality train" attributes).

Critical strengths

Equation (3) gives some intuition for how scale factors for steering interact with layer normalization.

Critical weaknesses

Notation is poorly defined and seems inconsistently used. This makes the technical content of the paper very difficult to follow or evaluate. For example, the contrastive activation collection H in equation 2 uses different notation than in equation 1 (with subscripts referring to instances instead of layers) and it is unclear how the positive or negative pairs are included in eq. 2. N(H) is introduced in equation 2 and then never used again. \hat G is introduced in equation 6, but not defined (it appears to correspond to the use of the calibration scenarios instead of the test scenarios?). Coefficients \alpha and amplitudes "a" seem to be used interchangeably in section 3.2, but it is unclear why or what is the different role of the "amplitude" (which is not defined in this context). What is "min arg max" in the algorithm block?

The paper is written unclearly, with somewhat incoherent choices of language that are not explained or motivated (e.g. the scale factor is sometimes referred to as a "dose"; the use of "amplitude" as mentioned above).

Questions for authors

Why is the candidate set C_m in equation 6 fixed to a cardinality of 3?

What is the significance of equation (7)?

How accurate is the judge model itself? Has this been validated against the training data?

Relatedly: what training data (i.e. set of contrastive pairs) was used to estimate the steering vectors?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 48R1

Thank you for noting the intuition of Eq. (3) and for your feedback on notation and clarity. We rewrote Sec. 3 (Pages 3–5) and added Appendices F–I (Pages 18–26) to make definitions, equations, and prompts self-contained.

1. Notation and terminology in Section 3

Unified terminology (Sec. 3.1, Lines 144–147).

  • Steering coefficient α∈R\alpha \in \mathbb{R}: signed scalar in hℓ+αv^m,t,ℓh_\ell+\alpha\hat{v}_{m,t,\ell} (+a+a: high pole; −a-a: low pole).
  • Candidate amplitude a>0a>0: unsigned magnitude specifying (+a,−a)(+a,-a) ("dose" in dose–response curves).
  • Candidate grid Cm={a1,a2,a3}\mathcal{C}_m=\{a_1,a_2,a_3\}: three ordered positive amplitudes for model mm.

Explicit indexing (Sec. 3.1, Lines 151–157; Eqs. 1–2).

For KK contrastive pairs at layer ℓ\ell, Eq. (2) indexes all n=2Kn=2K positive/negative hidden states:

Hm,t,ℓ=(hℓ(x1+),…,hℓ(xK+),hℓ(x1−),…,hℓ(xK−))=(hj)j=12K.H_{m,t,\ell}=(h_\ell(x_1^+),\dots,h_\ell(x_K^+),h_\ell(x_1^-),\dots,h_\ell(x_K^-))=(h_j)_{j=1}^{2K}.

Here jj indexes instances, while ℓ\ell indexes layers, and

R(H)=1nd∑j=12K∥hj∥22.R(H)=\sqrt{\frac{1}{nd}\sum_{j=1}^{2K}\lVert h_j\rVert_2^2}.

Role of vector norm (Sec. 3.1, Lines 159–161).

Coordinate RMS R(H)R(H) is plotted in Fig. 2. Vector norm N(H)=d R(H)N(H)=\sqrt{d}\,R(H) is evaluated directly in Table 1 (Page 7, "Probe/Injection RMS norm"), achieving LOMO ρ=0.875\rho=0.875 and 0.8540.854 (0.864 / 0.846 and LOFO ρ=0.886\rho=0.886 excluding Gemma-2-2B; App. F, Tab. 8).

Explicit calibration estimator (Sec. 3.3, Eq. 6, Line 197).

For repetition subset Rcal⊆1,…,5\mathcal{R}_{\text{cal}}\subseteq\\{1,\dots,5\\} of size r∈1,2,5r\in\\{1,2,5\\} (versus all Rs=5R_s=5 in Eq. 5), Eq. (6) defines:

G^t(a;Scal)=1∣Scal∣∑s∈Scal1r∑j∈Rcal[Yt,s,j(+a)−Yt,s,j(−a)].\widehat{G}_t(a;S_{\text{cal}})=\frac{1}{|S_{\text{cal}}|}\sum_{s\in S_{\text{cal}}}\frac{1}{r}\sum_{j\in\mathcal{R}_{\text{cal}}}[Y_{t,s,j}(+a)-Y_{t,s,j}(-a)].

Tie-breaking (Sec. 3.3, Eq. 7, Line 202; Algorithm Box, Line 230).

Eq. (7) and Step 3 define:

Mt=arg⁡max⁡a∈CmG^t(a;Scal),a^t=min⁡Mt.M_t=\arg\max_{a\in\mathcal{C}_m}\widehat{G}_t(a;S_{\text{cal}}),\qquad\hat{a}_t=\min M_t.

Thus, ties favor the smaller perturbation.

2. Answers to specific questions (Q1–Q4)

Q1: Why three candidates? (Sec. 3.4, Lines 236–238)

Each pair (+a,−a)(+a,-a) costs 2r∣Scal∣2r|S_{\text{cal}}| ratings. Three ordered amplitudes (∣Cm∣=3|\mathcal{C}_m|=3, matching {2,4,6}\{2,4,6\}) form the smallest discrete grid with an interior candidate (a2a_2) that detects non-monotonic interior peaks (Fig. 4a; App. Tab. 10), at a matched 2×3=62\times3=6 rating budget.

Q2: Significance of the transfer bound? (Sec. 3.4, Lines 250–259; App. C, Page 14)

Eq. (8) states:

Gt(a^t;Stest)≥max⁡a∈CmGt(a;Stest)−2ε,G_t(\hat{a}_t;S_{\text{test}})\ge\max_{a\in\mathcal{C}_m}G_t(a;S_{\text{test}})-2\varepsilon,

where

ε=max⁡a∈Cm∣G^t(a;Scal)−Gt(a;Stest)∣.\varepsilon=\max_{a\in\mathcal{C}_m}|\widehat{G}_t(a;S_{\text{cal}})-G_t(a;S_{\text{test}})|.

It bounds held-out suboptimality by calibration–test contrast deviation across the three candidates, formalizing why preserving ordering on one scenario retains 89.6% (1.266 / 1.413) of the 30-rating effect.

Q3: Judge accuracy and validation (Sec. 5.5, Pages 9–10; App. D, F, I)

  • Contrastive data probe (App. D, Tab. 7, Page 16): Scenario-grouped linear probing achieves 0.965 mean accuracy across 45 collections (0.918 on 24 pair-held-out sets), confirming clean trait encoding.
  • Zero self-judging (App. F, Tabs. 8–9, Page 18): Excluding Gemma-2-2B preserves scale predictability across the 14 remaining models (LOFO ρ=0.820\rho=0.820–0.8860.886) and improves 20/20 non-Gemma pairs (0.290 → 1.140; +0.851 [0.537, 1.245]), while doubling the selector's mean gain over Largest (1.140 vs. 0.947).
  • Full re-scoring (App. I, Tabs. 21–22, Page 26): Qwen2.5-7B-Instruct re-scores all 13,650 responses across both grids, yielding 93.2% within-1-point agreement (r=0.775r=0.775, ρ=0.781\rho=0.781) with Gemma-2-2B-it. Gains hold for 25/25 pairs (2.210 vs. 0.472; +1.738 [1.172, 2.440]) and 20/20 excluding Gemma-2-2B or Qwen2.5-7B.

Q4: Data for steering vectors and evaluation (Sec. 4, Page 6; App. H, Pages 20–25)

Vectors are extracted via Eq. (1) from contrastive pairs in App. H (Tabs. 11–20): 10 extraction contexts/trait paired with 4 high and 4 low instructions (10×4×4=16010\times4\times4=160 paired rows, or 80 unique signed prompts/trait). These have zero string overlap with the 39 open-ended evaluation scenarios (7 for O and 8 each for C/E/A/N), generated without persona prompts.

AuthorsOpenReview ↗

Clarified Method and Evidence for Calibration Transfer

Dear Reviewer 48R1,

Following our detailed response, we summarize below how the revised manuscript connects the clarified formulation to the empirical evidence addressing your questions.

Method and notation. Our contribution separates two practical decisions: residual scale characterizes sensitivity across models, while sparse behavioral feedback selects an operating amplitude for each attribute. Across 15 checkpoints, mid-layer RMS predicts held-out-family sensitivity ordering with Spearman ρ=0.83\rho=0.83. Within the measured expanded grids, six ratings retain 89.6% of the effect obtained with 30 ratings on the same calibration scenario. Section 3 now explicitly distinguishes the signed coefficient α\alpha from the positive amplitude aa, separates instance and layer indices, defines G^\widehat{G}, and states that ties select the smaller amplitude. The vector-level RMS norm N(H)N(H) is evaluated in Table 1.

Three candidates and the transfer bound. Three ordered amplitudes form the smallest grid containing an interior candidate, allowing the selector to choose an interior maximum while matching the original grid’s six-rating budget. The empirical dose curves confirm why this matters: 8 of 25 curves attain a higher contrast at a smaller amplitude than at the largest amplitude (Appendix G, Table 10). The transfer bound (now Eq. 8) relates held-out suboptimality to the maximum calibration–test contrast discrepancy across candidates, making explicit when a choice based on one scenario transfers effectively.

Judge evaluation. Appendix E provides the complete trait rubric and scoring protocol. The revised analysis replaces the primary judge’s scores with independent Qwen2.5-7B-Instruct scores for both selection and held-out evaluation across all 13,650 stored responses. Under this judge, moving from the original to the expanded grid at the same six-rating budget improves mean held-out contrast from 0.472 to 2.210, with improvements in all 25 model–trait averages. Excluding Qwen2.5-7B itself preserves improvements in all 20 remaining averages (Appendix I, Table 21).

Contrastive data. Appendix H provides the exact configured high/low instructions, extraction contexts, evaluation scenarios, and rubric indicators, which are supplied as fixed configurations in the supplementary code. Extraction pairs share a user context and differ in the trait-specific system instruction, with Eq. (1) defining the mean-difference direction. The 39 behavioral evaluation scenarios use no persona instruction and have zero exact-string overlap with the extraction contexts, and calibration scenarios are excluded from held-out evaluation.

We hope these clarifications and independent evaluations address your concerns, and we would appreciate your reassessment of the paper.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response

We thank Reviewers 48R1, Kime, and UJfJ for their constructive feedback. We have uploaded a revised 10-page manuscript (revisions in blue, with Appendices F–I, Pages 18–26) addressing all questions:

1. Reference Correction (Reviewers Kime, UJfJ)

We apologize for the co-author surname error in the .bib entry for Evaluating and Inducing Personality in Pre-trained Language Models (Jiang et al., NeurIPS 2023, Vol. 36, pp. 10622–10643), cited in Sec. 2 (Line 124). During .bib preparation, the last two co-authors' surnames (Chi Zhang and Yixin Zhu) were mistakenly transcribed as Chi Ma and Yixin Liang (much as Reviewer UJfJ's note inadvertently transposed the final surname in reverse as "Yixin Liang (instead of Yixin Zhu)"). We have restored the exact NeurIPS 2023 citation (Page 11, Lines 582–587) and verified all references against their official proceedings.

2. Zero-Self-Judging Analysis (E1) & Full-Grid Independent Judge Scoring (E2) (All Reviewers)

To address concerns about Gemma-2-2B-it evaluating its own generations and its 2B size, we added two complete analyses (Sec. 5.5, Pages 9–10; App. F Tabs. 8–9; App. I Tabs. 21–22):

  1. Excluding Gemma-2-2B Completely (App. F Tabs. 8–9):
    • 14 Non-Gemma Models Across 4 Families (Tab. 8): Removing Gemma-2-2B leaves 14 models across Qwen, LLaMA, Mistral, and Phi. Residual scale remains predictive across all five features: LOFO Spearman ρ=0.820\rho = 0.820 (LOMO 0.7230.723, median AE 0.2340.234, mean AE 0.3220.322) for mid-layer RMS, LOFO ρ=0.873\rho = 0.873 for injection RMS, and LOMO ρ=0.864\rho = 0.864 / LOFO ρ=0.886\rho = 0.886 for probe/injection RMS norm.
    • 4 Non-Gemma Calibrated Models (Tab. 9): Without Gemma-2-2B, 6-rating selection raises held-out signed contrast from 0.290 (Orig 6) to 1.140 (Exp 6; Exp 30 reaches 1.281), a paired gain of +0.851 (95% CI [0.537,1.245][0.537, 1.245]) with improvements in all 20/20 model–trait pairs. Notably, excluding Gemma-2-2B more than doubles the 6-rating selector's mean advantage over the zero-feedback Largest baseline from +0.095+0.095 (1.2661.266 vs. 1.1721.172) to +0.193+0.193 (1.1401.140 vs. 0.9470.947; reaching +0.334+0.334 at 30 ratings) and exceeds Uniform (1.021).
  2. Independent Re-Scoring of Both Full Grids (13,650 Responses) by Qwen2.5-7B-Instruct (App. I Tabs. 21–22): We scored every stored response across both 5-model grids (6,825 Orig + 6,825 Exp = 13,650 responses) using Qwen2.5-7B-Instruct with the exact Appendix E rubric:
Cohort (Disjoint Splits)Judge ModelOrig 6Exp 6Exp 30LargestUniformExp 6 − Orig 6 [95% CI]Improved Pairs
All 5 Models (2525 pairs)Gemma-2-2B-it0.2450.2451.2661.4131.1721.1721.1871.187+1.022 [0.603,1.441][0.603, 1.441]25 / 25
Exclude Gemma (2020 pairs)Gemma-2-2B-it0.2900.2901.1401.2810.9470.9471.0211.021+0.851 [0.537,1.245][0.537, 1.245]20 / 20
All 5 Models (2525 pairs)Qwen2.5-7B-Inst0.4720.4722.2102.4082.2002.2002.0692.069+1.738 [1.172,2.440][1.172, 2.440]25 / 25
Exclude Gemma (2020 pairs)Qwen2.5-7B-Inst0.5380.5381.9532.1401.8531.8531.7711.771+1.415 [1.034,1.796][1.034, 1.796]20 / 20
Exclude Qwen2.5-7B (2020 pairs)Qwen2.5-7B-Inst0.3600.3602.1022.3172.0292.0291.9501.950+1.743 [1.034,2.582][1.034, 2.582]20 / 20

Across all 13,650 responses, the two judges agree within ±1\pm 1 point on 93.2% of ratings (92.6% Exp, 93.8% Orig; Pearson r=0.775r = 0.775, Spearman ρ=0.781\rho = 0.781; 0.7970.797 and 0.8030.803 on Exp). In every cohort—including when each judge's own model is excluded—sparse calibration improves all model–trait averages over the original grid and outperforms both zero-feedback baselines (Largest and Uniform).

3. Why Sparse Selection Beats Zero-Feedback Largest (Reviewer UJfJ, App. G Tab. 10)

Across all 25 dose curves (Tab. 10, Page 19), 8 of 25 curves peak strictly at an interior amplitude (a1a_1 or a2a_2)—spanning 3 of the 5 models (Qwen2.5-1.5B A/N, Qwen2.5-7B E/N, and Qwen3-0.6B O/C/A/N)—and 3 curves reverse sign (G(a3)<0G(a_3) < 0) at the largest dose (Qwen2.5-1.5B A =−0.500= -0.500, Qwen3-0.6B A =−0.650= -0.650, N =−1.200= -1.200). Because curve shape is unknown before deployment, 6 ratings resolve amplitude choice while preventing over-steering collapse (+1.352+1.352 on Qwen3-0.6B).

4. Notation & Complete Dataset Card (Reviewers 48R1, Kime)

We revised Sec. 3 (Pages 3–5) to unify notation and added App. H (Pages 20–25, Tabs. 11–20) with the full text of all 50 extraction contexts, 40 high/low instructions, 39 evaluation scenarios, and 50 rubric indicators.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.