- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper argues that activation-steering approaches to LLM personalization implicitly "tie" three choices that should be treated separately: (i) the layer where a preference direction is extracted versus the layer where it is applied; (ii) a scalar (rank-1) preference axis versus a low-rank subspace; and (iii) the direction that best decodes a preference ("read") versus the direction that most changes a downstream decision ("write"). The authors call the separation read–write factorization and test each assumption with its own diagnostic. Most experiments use Qwen-2.5-7B-Instruct and Mistral-7B-Instruct-v0.3 on a synthetic binary team-decision task (escalate vs. review) and on PrefEval.
Critical strengths
1.Conceptual novelty and clarity: The paper clearly distinguishes between read and write directions and provides strong evidence that they are geometrically and causally distinct. This challenges a widely held assumption in activation steering and opens new research directions. The formalization of extraction vs. application depth, preference rank, and read-write separation is well-motivated and rigorously tested. 2.A simple, informative crossed-depth experiment and unusually candid scoping. Table 1 shows that late-extracted directions retain their effect at mid depth (2.91 vs. 2.87), while mid-extracted directions lose much of their effect when applied late (0.37). The practical insight and transparent reporting of limitations strengthen the paper.
Critical weaknesses
-
The paper's presentation could be improved to enhance readability and clarity. Some aspects of the notation, mathematical exposition, and figure presentation make the methodology difficult to follow, and greater consistency would help readers understand the proposed framework and reproduce the experiments (see Question 1).
-
The experimental validation is not yet sufficiently comprehensive to fully support some of the paper's central claims. More detailed ablations are needed to distinguish the contributions of individual components. Furthermore, some reported results appear inconsistent with the paper's broader conclusions, raising concerns about whether the evidence is interpreted comprehensively. Clarification of key implementation details is also needed to strengthen the validity and reproducibility of the experimental findings (see Questions 2–5).
Questions for authors
1.The paper's readability is compromised. For example, equation numbering is inconsistent (only Eqs. 1–5 are numbered; key definitions like Δodd are not) with no stated convention—please clarify and unify, and the formulas in Figure 3 are too small to read clearly—please enlarge the font.
2.In the CL-Bench experiment of TABLE 3, the random direction with inherited coupling achieves 82.5% BDA, while the learned direction reaches 90.5%. Can you provide a more detailed ablation that isolates the contribution of the learned direction versus the coupling interface and context representation? 3. In §4.3, the 32B replication shows a non-significant crossed-depth asymmetry. Given that 14B and 72B are significant, what might explain this? Could it be related to model scale, training data, or architecture-specific properties? Additional analysis would help clarify the robustness of the depth asymmetry claim.
4.In Eq. 4, how exactly is q(x) added to the decision-token logits under randomized A/B labels? Does any inference-time computation use information about which option is correct?
5.The paper's claim that read and write directions are "essentially none" overlapping and "consistent with random" across scales is contradicted by its own Table 11, which reports read-span coverage of 0.44–9.75% on other models—far above the random baseline (~0.11%)—making the conclusion a selective summary of Qwen-7B alone.
Dear Reviewers,
Thank you for the review. The authors have responded to the points you raised, and the response period closes on 18 November (AoE). I would be grateful if you could post a brief reply on whether your concerns are resolved, partly resolved, or still open, so that the authors have a fair chance to address them. If anything still needs clarifying, please ask now while they can still respond.
Regards,
AC