- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies whether coding practices can be represented and controlled through residual activation directions. Using factorial prompts and compliance-conditioned mean differences across six Qwen2.5-Coder models, the authors analyze comments, logging, defensive constructs, and programming languages, then evaluate held-out steering, vector composition, prompt alignment, and token costs. The main contribution is an empirical characterization of when these directions correlate with and alter coding behavior, including their limited specificity and substantial correctness costs, rather than a new steering algorithm.
Critical strengths
- The factorial multilingual design matches tasks and non-target behavior settings, providing a clearer extraction protocol than loosely contrasted prompts. Separate training, validation, and test tasks reduce direct tuning on the reported test outcomes.
- The representation analysis combines variance decomposition, cross-language alignment, analogy retrieval, vocabulary projection, and random-direction lexical diagnostics. These complementary views offer useful descriptive evidence while acknowledging the limitations of each diagnostic.
Critical weaknesses
- My main concerns lie in the gap between prompt-conditioned representation differences and behavior-specific causal mechanisms. The vectors are extracted from teacher-forced post-token states after selecting outputs that already satisfy behavior detectors, so they may encode the consequences of generated code and selection effects rather than a distinct variable that initiates the behavior. Intervening with such a direction establishes an effect on generation, but not the causal interpretation assigned to the direction itself.
- The primary steering experiments lack norm-matched random and shuffled-vector intervention controls. For example, in the 14B model, logging steering increases comment detection by 0.36 but logging detection by only 0.08, leaving target specificity unresolved.
- Validation selects interventions by detector induction without a correctness constraint, and some selected comment settings reduce Python test passing to zero. This establishes perturbation capacity but leaves the practical operating region much less convincing than the headline behavior changes suggest.
Questions for authors
- Why do the logging directions reverse their effective steering sign at 1.5B and 3B?
- Does a direction increase the probability of initiating a coding behavior, or mainly amplify that behavior after its first occurrence?
3 comments
Thank you for distinguishing a representation contrast from a selective causal control. We ran three checks to address that distinction directly. All rates below are over all requested tasks; intervals are paired task-bootstrap 95% intervals.
Does extraction require the original instruction to be present? We replayed exactly the same accepted 7B training programs under a common neutral prompt for each task and language. We retained the original matched pairs and matched the injected residual norm. Original/neutral-replay cosines are 0.81 for comments, 0.77 for logging, and 0.90 for defensive constructs. That geometric agreement did not imply equivalent interventions. At the same moderate injection norm, the original comment vector raised detection by 44 percentage points (95% CI [34, 54]), while the neutral-replay vector raised it by 5 points ([-2, 13]); the direct difference was -39 points ([-49, -29]). Both passed 45% of the same 100 Python tasks. In contrast, the neutral-replay defensive direction retained a 28-point detector gain ([20, 37]). Thus a common replay prompt preserves much of the geometry but changes some intervention effects. Both extracts remain post-token and outcome-conditioned; neither identifies an endogenous causal variable.
First occurrence versus continuation. At a moderate 7B setting, we held the assistant prefix fixed and compared decoding with and without the same comment direction. With a comment-free code fence as the prefix, first detected comment/docstring occurrence rose from 40% to 85% (+45 points [35, 55]). When the prefix already supplied one comment, further occurrences rose from 69% to 94% (+25 points [17, 34]); mean additional count rose from 2.65 to 4.48. In a separate ordinary generation ablation, the rates were 38% without steering, 83% for all positions, 83% for continuation only, 80% for only the first 32 continuation calls, and 48% when steering began after those calls. This demonstrates an effect on later first occurrence as well as on additional occurrences, under these prompts and this operating point. It does not tell us whether a model had already planned the behavior internally.
Logging sign and specificity. Holding layer and absolute coefficient fixed, the extraction-negative vs. extraction-positive directions produced logging on 26% vs. 20% of 1.5B tasks and 51% vs. 6% of 3B tasks. Across 100 random disjoint halves of the training tasks, median split-half cosine for extraction was 0.978 and 0.980, respectively. Thus the more effective negative injection is not explained simply by an unstable extraction sign; these experiments do not identify the underlying cause. Three Gaussian and three within-pair sign-shuffled directions matched the injected norm at each frozen setting. At 1.5B, the learned logging setting did not outperform the mean of either null family, which limits the selective-control interpretation. In a separate positive comment calibration at 7B, no tested learned coefficient met the five-point validation correctness screen against the stronger of prompting and the neutral reference. We will distinguish the reproducible extraction direction, the effect of an intervention, and the subset of settings supported as usable controls.
Thank you for considering our follow-up. We added neutral-context replay and fixed-prefix experiments that distinguish the first occurrence of a comment from further commenting. We’d welcome your updated assessment and would be grateful if you could reflect any improvement in your assessment in your rating.
Thank you for identifying the empirical characterization as the main contribution. Representation differences, intervention effects, and practical operating points require separate interpretations.
1. What the extracted direction represents
Section 3.2 estimates the average within-pair difference in teacher-forced post-token residuals, matching task, language, and the two non-target instruction settings. These states have already observed the generated token and still attend to the original instruction prompt. Examples are selected by syntax and target-detector compliance. Thus the contrast can contain prompt effects, output-content differences, and selection effects.
Matching reduces the specified nuisance differences; it does not identify an endogenous variable that initiates the behavior. Adding the direction to held-out neutral-prompt generations establishes an effect of that intervention, while its interpretation as a distinct behavior-specific mechanism remains unresolved. We will define "concept vector" operationally at first use and avoid assigning antecedent causal meaning to the extraction.
Removing comment/docstring spans before pooling yields cosine 0.45-0.66 with the full direction (Table 4). The pooled positions exclude literal comment spans, but remaining states can still encode preceding comments and the prompt. Neither conditioning nor selection is removed; neutral-context and pre-onset analyses are untested.
2. Specificity controls
We agree with the 14B counterexample: logging steering increases commenting by 36 percentage points and logging by 8 (Table 8). At 7B, comment and logging directions each increase commenting by 57 points. These observed off-target responses prevent interpreting the three directions as independently controllable axes.
Random lexical and alignment diagnostics do not substitute for matched-norm decoding interventions, which are absent. Equal coefficients do not match perturbation norms because Equation 4 includes the direction's extraction norm. Controls should match the injected delta norm, layer, coverage, and decoding, using multiple random and within-pair sign-shuffled directions with comparable validation-selection budgets. Held-out evaluation should include every detector and correctness. Specificity remains unresolved.
3. Why does logging reverse sign at 1.5B and 3B?
The extraction orientation is fixed as positive minus negative. Validation selects coefficient -4 at zero-based layer 14 for 1.5B and layer 18 for 3B (Table 7). Held-out logging increases by 19 and 23 percentage points, with 95% CIs [8, 29] and [12, 34], respectively (Table 2). We mark both as directional failures, rather than evidence that the extracted positive direction induces logging.
The experiment does not identify why the sign reverses. A mean difference among post-token states need not have the same signed effect when injected during generation. Prompt conditioning, selected output features, correlated behaviors, and nonlinear response are possible explanations, not established mechanisms. We will state the observation and this distinction directly, without attributing the reversal to model scale or sampling noise.
4. Initiation versus amplification
Our endpoint detectors cannot distinguish these mechanisms. "After the first token" means after the first output token, not after the first comment or logging occurrence. Subsequent steering is applied whether or not the target behavior has appeared, so it can affect a later first occurrence as well as continuation after an occurrence; the first output-token distribution itself is unchanged.
Common behavior-free prefixes could test first-onset probability; separate prefixes containing an occurrence could test continuation intensity. These analyses and coverage ablations are unperformed, so we make no initiation-specific claim.
5. Correctness-aware operating points
The primary sweep intentionally maximizes detector induction, then reports held-out behavior and correctness together. Its selected 7B/32B comment settings have 0% Python pass, and 14B logging has 4% (Section 6.1). These are destructive settings, not deployment recommendations.
Appendix G uses a separate validation screen: parsing/extraction at least 95%, and pass no more than five percentage points below the higher prompt/neutral pass rate. The held-out 7B concise-Python profile reaches 100% detected compliance, 99% parsing, and 43% pass (prompting: 100%, 100%, and 44%) and 9.0% fewer total tokens. The paired pass difference is -1 percentage point, 95% CI [-8, +6]; neither equivalence nor noninferiority is established. No tested positive logging-plus-defensive strength qualifies. The single calibrated suppression result does not establish a broad practical operating region.