- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes UPSHIFT, a training-free method for LLM personalization that extracts user-specific steering vectors from historical human–AI interaction trajectories. The method computes differences between the hidden representations of initial assistant responses and subsequent feedback-conditioned responses, then averages these differences across sessions to obtain a user-specific preference direction. The resulting vector is injected into the model's residual stream during inference, enabling personalization without updating model parameters or including historical conversations in the input prompt.
The authors evaluate UPSHIFT on PrefEval and through experiments involving 40 simulated users with Qwen3-8B and Qwen3-32B. The results demonstrate improvements over unsteered baselines on held-out topics. Additional experiments involving mismatched user vectors and positive-versus-negative steering directions suggest that the extracted vectors contain user-specific preference signals.
Critical strengths
1. An interesting source of implicit personalization signals. The paper explores the possibility of extracting user preference information from changes in assistant responses following user feedback, without requiring explicit preference labels or user-authored target responses. Although the underlying activation steering framework is closely related to prior work, using feedback-conditioned response trajectories as a source of personalization signals is an interesting idea.
2. A simple formulation with useful control experiments. The proposed method is straightforward to implement and does not require model parameter updates. The authors also conduct user-mismatch and positive-versus-negative steering experiments, which provide useful evidence that the extracted vectors contain user-specific information rather than merely inducing arbitrary changes in model behavior.
Critical weaknesses
1. Limited methodological novelty and lack of comparisons with closely related prior work.
The methodological novelty of UPSHIFT appears limited in the context of existing activation-based personalization methods, such as StyleVector (Zhang et al., 2025), SteerX (Zhao et al., 2025), and GLASS (Chen et al., 2026). In particular, StyleVector already constructs user-specific steering vectors by averaging contrastive differences between hidden representations and applies them at inference time without updating model parameters. UPSHIFT follows essentially the same overall framework, with its primary distinction being the use of initial and feedback-conditioned assistant responses as contrastive pairs. While this is a different source of supervision, it is unclear whether it represents a sufficiently substantial methodological contribution beyond existing contrastive activation steering approaches.
More importantly, despite explicitly discussing these closely related methods in the Related Work section, the paper does not include them as experimental baselines. The main results are based on comparisons against an unsteered LLM, supplemented by user-mismatch and steering-direction controls. Although these experiments demonstrate that the extracted vectors can influence responses in a user-specific manner, they do not establish any advantage over existing personalization techniques.
This is a fundamental limitation rather than merely a missing experimental detail. Given the substantial methodological overlap with prior work, direct comparisons are essential to establish the significance of the proposed contribution. Without such comparisons, it is difficult to determine whether the proposed feedback-trajectory-based extraction offers any meaningful improvement over existing approaches, or whether similar results could already be achieved using established contrastive activation steering methods. Consequently, the paper fails to convincingly establish either its methodological novelty or its relative effectiveness.
2. Insufficient empirical validation of the central implicit-preference hypothesis.
The paper's central hypothesis is that aggregating hidden-state changes across feedback trajectories can recover stable, implicit user preferences that generalize to new tasks. However, the experimental validation does not sufficiently establish this claim.
The main personalization experiments rely entirely on 40 simulated users generated from publicly available LinkedIn profiles, with evaluations performed by a profile-conditioned LLM judge. Since the underlying profiles primarily contain biographical and professional information rather than explicit subjective response preferences, it is unclear whether the observed improvements reflect genuine preference inference or consistency between the simulated users and the evaluation judge, both of which are conditioned on the same profile information.
Furthermore, the PrefEval experiments primarily demonstrate that activation steering can represent and control preference-related information in a benchmark setting. They do not directly validate whether stable, latent preferences can be reliably recovered from natural human feedback trajectories.
Therefore, the experiments do not convincingly establish that the proposed method can discover and generalize actual human preferences. Given that this is the central motivation and claimed contribution of UPSHIFT, the reliance on simulated interactions and LLM-based evaluation substantially limits the strength of the empirical evidence.
Taken together, these concerns make it difficult to establish the paper's significance beyond existing activation-based personalization research and substantially weaken the support for its central claims.
References
[1] Zhang, J., Liu, Y., Wang, W., Liu, Q., Wu, S., Wang, L., and Chua, T.-S. (2025). Personalized Text Generation with Contrastive Activation Steering. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL 2025), pp. 7128–7141.
https://aclanthology.org/2025.acl-long.353/
[2] Zhao, X., Yan, M., Qiu, Y., Ni, H., Zhang, Y., Feng, F., Cheng, H., and Chua, T.-S. (2025). SteerX: Disentangled Steering for LLM Personalization. arXiv preprint arXiv:2510.22256.
https://arxiv.org/abs/2510.22256
[3] Chen, L., Zhang, Z., Zhang, X., Nie, S., Liu, Q., Wu, S., and Wang, L. (2026). From Profiles to Steering Vectors: Global Sparse Priors and Local Semantic Calibration for Personalized Text Generation. arXiv preprint arXiv:2607.21620.
https://arxiv.org/abs/2607.21620
Questions for authors
Q1. What is the substantial methodological contribution of UPSHIFT beyond existing contrastive activation steering approaches?
In particular, StyleVector already constructs user-specific steering vectors from averaged hidden-state differences and uses them for inference-time personalization. Apart from replacing the contrastive pairs with initial and feedback-conditioned assistant responses, what fundamentally distinguishes UPSHIFT from existing approaches? What evidence demonstrates that this particular choice of contrastive pairs offers additional benefits?
Q2. Why are closely related personalization methods not included as experimental baselines?
The paper discusses StyleVector, SteerX, and GLASS but does not compare against them. Can the authors provide comparisons against these methods, or appropriately adapted versions of their vector extraction procedures, under matched experimental conditions? In particular, can the authors isolate whether the proposed temporal feedback contrast provides improvements beyond those achievable through existing activation-based personalization techniques?
Q3. How can the authors establish that the extracted vectors represent genuine implicit user preferences rather than artifacts of the simulation and evaluation setup?
Since both the simulated users and the LLM judge are conditioned on LinkedIn-derived profiles that do not explicitly specify subjective response preferences, how is the validity of the inferred preferences established? Is there any independent human evaluation or evidence from real user feedback trajectories supporting the central implicit-preference hypothesis?
3 comments
Official Comment by Authors
Q3. How can the authors establish that the extracted vectors represent genuine implicit user preferences rather than artifacts of the simulation and evaluation setup?
We appreciate this fundamental concern and agree that relying exclusively on simulated users and profile-conditioned LLM judgments leaves room for potential bias. In particular, conditioning both simulated users and evaluators on the same LinkedIn-derived profiles may introduce correlations that do not necessarily reflect genuine subjective preferences.
To address this concern, we conducted an additional small-scale human study involving 10 volunteers. Each participant selected six topics of personal interest and completed at least three rounds of interaction with the assistant on each topic. We extracted individual preference vectors using UPSHIFT with the ALL-RM configuration. For evaluation, each participant completed ten A/B comparisons on newly sampled topics, comparing unsteered responses against responses steered by their own extracted preference vectors. We used Qwen3-32B with a steering coefficient of α = 0.05.
The results show that participants preferred UPSHIFT-steered responses in approximately 63% of the pairwise comparisons. Importantly, both the interaction histories used for vector extraction and the subsequent preference judgments came from real human participants, without relying on LinkedIn-derived profiles or profile-conditioned LLM judges. The evaluation also uses previously unseen topics, providing initial evidence that feedback-derived preference directions can generalize beyond the interactions from which they were extracted.
We acknowledge that this study is relatively small-scale and does not establish the robustness of the implicit-preference hypothesis across a broad population. Nevertheless, it provides an independent human-based validation that complements our larger simulated-user experiments and directly addresses the potential evaluation bias raised by the reviewer.
We will incorporate this additional human evaluation into the revised manuscript and clarify the limitations of both simulated and human evaluations.
We sincerely thank the reviewer again for these constructive comments, which have helped us clarify the methodological contribution and strengthen the empirical validation of UPSHIFT.
I thank the authors for their detailed response and, in particular, for conducting an additional human evaluation. The new study with real participants is a meaningful improvement over the original evaluation, as it partially addresses my concern about the reliance on simulated users and profile-conditioned LLM judgments.
However, my primary concerns remain unresolved.
First, while I appreciate the clarification that the contribution lies in feedback-trajectory-based supervision rather than a new activation steering mechanism, the rebuttal does not provide empirical evidence that this supervision formulation offers meaningful advantages over existing contrastive activation steering approaches.
Second, the lack of comparisons with closely related personalization methods remains a fundamental limitation. I understand that these methods were originally evaluated under different settings, but my concern was precisely the need for comparisons under a common evaluation protocol, rather than comparisons of previously reported results.
Third, although the additional human study is encouraging, its small sample size and the absence of user-mismatched steering controls in the human evaluation make it difficult to determine whether the observed preference improvements reflect genuinely user-specific preference inference rather than more general improvements in response quality.
Overall, I acknowledge that the rebuttal strengthens the empirical evidence, particularly through the new human study. Nevertheless, the central concerns regarding the paper's distinct contribution and its effectiveness relative to existing methods remain. Therefore, I maintain my original rating.
We sincerely thank the reviewer for the thoughtful and detailed assessment of our work. We appreciate the recognition of UPSHIFT's implicit feedback-based personalization approach and its controlled experiments. We address the concerns regarding methodological novelty, baseline comparisons, and the empirical validity of the implicit-preference hypothesis below.
Q1. What is the substantial methodological contribution of UPSHIFT beyond existing contrastive activation steering approaches?
Thank you for raising this important question. We agree that the activation addition operation and cross-sample averaging themselves are established techniques. Our contribution is not to introduce a new activation-steering operator, but to investigate a different source of personalization supervision and formulate a training-free approach to extracting reusable preference directions from natural human–AI interaction trajectories.
Specifically, existing approaches such as StyleVector derive user-specific directions primarily from user-authored texts and style-neutral counterparts, targeting the reproduction of individual writing styles. SteerX further investigates the disentanglement of preference-driven information from user histories, while GLASS extracts global and local style representations from historical responses. In contrast, UPSHIFT uses the temporal contrast between an assistant's initial response and its feedback-conditioned response within the same interaction session. This provides assistant-side supervision about how users want responses to be adjusted, rather than relying on how users themselves write or on explicitly constructed preference descriptions.
We believe this distinction is important because users' preferred assistant responses do not necessarily resemble their own writing styles. For example, a user may communicate concisely while preferring detailed and rigorous explanations from an assistant. UPSHIFT aims to capture this distinction by learning from users' interventions on assistant-generated responses.
Thus, the methodological contribution lies in the feedback-trajectory-based supervision formulation and its cross-session aggregation, rather than in the underlying vector addition operation itself. We will clarify this distinction and better articulate the scope of our methodological contribution in the revised manuscript.
Q2. Why are closely related personalization methods not included as experimental baselines?
We acknowledge that direct comparisons with existing methods would provide additional evidence regarding their relative effectiveness. However, the original evaluation objectives and supervision settings of these methods differ from those of UPSHIFT.
For instance, StyleVector and GLASS primarily evaluate personalized text generation by measuring the correspondence between generated outputs and user-specific historical text. SteerX also investigates preference-driven information, but its empirical evaluation focuses on personalized review generation and topic writing, using reference-based text-generation metrics such as ROUGE, METEOR, BLEU, and BERTScore. These evaluations do not directly measure whether users prefer an assistant's responses on previously unseen topics based on implicit preferences inferred from feedback trajectories.
Consequently, directly comparing the reported results across these methods would not establish a meaningful performance ranking. A rigorous comparison would require adapting their extraction procedures to the same interaction-feedback setting and evaluating all methods under a common user-preference protocol.
Our current experiments establish its effectiveness relative to unsteered generation and provide user-specificity and directionality controls. We will make this distinction explicit and better position UPSHIFT as a complementary approach that explores a different source of personalization supervision.