acceptodds
Under review as a conference paper at ICLR 2027

When Personalization Compromises Truth: Joint Control of User Alignment and Factuality in Large Language Models

Abstract

Personalization makes large language models more useful by adapting responses to user specific context, but it can also degrade factual reliability when personal signals override world knowledge. We study this failure mode, termed personalization-induced hallucination (PIH), as a dual-objective control problem involving personalization and factuality. We posit that personalized information contains both helpful components that improve user alignment and harmful components that interfere with factual reasoning. To address this challenge, we propose PerFact, an inference-time steering framework that jointly controls the two objectives within their shared activation subspace. PerFact identifies representations relevant to both personalization and factuality, removes PIH-related components from the personalization direction, and combines the resulting safe personalization direction with a factuality-enhancing direction for model intervention. Experiments across multiple backbones and benchmarks show that PerFact achieves consistent gains in both personalization and factuality, mitigating the trade-off between the two objectives observed in existing baselines. Our results identify a characteristic representation shift associated with PIH when incorporating personalized context, and show that targeted representation steering can reduce such shifts while improving factual reliability.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Personalization, LLM Hallucination, Representation Steering, Activation Steering, Inference-Time Intervention

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DKz9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates personalization-induced hallucination (PIH), defined as a degradation in a model's performance on objective questions when user-specific context is provided. The authors find that stronger emphasis on user context is associated with higher PIH rates, that PIH cases exhibit larger hidden-state magnitudes, and that two contrastive activation directions have higher cosine similarity. Based on these observations, the paper proposes PerFact, which identifies shared attention heads and applies singular value decomposition (SVD) to identify the harmful direction. Experiments on two LLMs show that PerFact achieves the best overall performance across various settings, demonstrating its effectiveness in mitigating PIH.

Critical strengths

The paper addresses an important problem: user-specific context can undermine factual accuracy, an increasingly relevant concern as personalization becomes more prevalent in LLMs. Moreover, the formulation in Equation 7 and the use of SVD to identify harmful activation directions offer a novel methodological perspective. In particular, Equation 7 preserves the interaction between user preferences and answer correctness, providing a meaningful basis for identifying and mitigating personalization-induced hallucinations.

Critical weaknesses

The empirical gains appear marginal. Given the relatively small number of questions in each dataset, the improvements amount to only one additional correct answer on CommonsenseQA and MMLU when using Llama-3.1-8B, raising concerns about the practical significance and robustness of the reported gains.

Moreover, although the authors cite [1], a more comprehensive comparison with this work is needed. Since [1] also investigates personalization-induced hallucination (PIH) and proposes a steering mechanism to mitigate it, the authors should more clearly distinguish PerFact from this prior approach and provide further empirical comparisons to establish its advantages.

[1] When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs, Findings ACL 2026

Questions for authors
  1. In the paragraph spanning lines 245–250, could the authors clarify how the probing embeddings are mapped to prediction probabilities? Specifically, is an MLP projection layer trained for this purpose?

  2. Could the authors clarify the key differences between the proposed method and [1] and provide empirical comparisons to better establish the advantages of their approach?

  3. How is the residual magnitude reported in Figure 4 computed?

  4. Are the selected shared heads relevant to each other? I'm curious whether this head configuration changes when you select those heads using only partially sampled questions.

[1] When Personalization Misleads: Understanding and Mitigating Hallucinations in Personalized LLMs, Findings ACL 2026

Reviewer_nYnSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Personalized LLMs condition on user context such as interaction history and preferences. The paper argues this can lower factual accuracy on objective questions whose answers should not depend on the user, a failure it calls personalization-induced hallucination (PIH). It defines PIH as the personalized response being less factual than the non-personalized one, and sets the goal as improving both factuality and personalization. Its analyses suggest that stronger reliance on personal context increases PIH, that PIH samples show larger hidden-state shifts, and that the factuality and personalization directions overlap in activation space.

Based on this, the authors propose PerFact, an inference-time steering method. It uses probes to find attention heads informative for both objectives, removes a PIH-related subspace from the personalization direction, combines the result with a factuality direction, and steers those shared heads. On PERG and PFQABench with two open models, they report that PerFact improves factuality and personalization over the original model and over steering baselines (CAA, StyliTruth, SPACE). They conclude that joint steering in the shared subspace reduces the trade-off between the two objectives.

Critical strengths

The problem is practically relevant and set up with care. PIH is defined formally (Eq. 1–3), and Section 3.1 and Appendix E.1 measure it against the non-personalized response. The prompting experiment (Figure 1) shows that stronger reliance on personal information increases PIH, and the length-matched relevant-versus-irrelevant context comparison (Table 6) helps rule out context length as the cause.

The method is lightweight and training-free. The authors include a random-direction control (Table 2) and sensitivity analyses over the intervention strength α and the number of shared heads. Reporting that a random direction on the shared heads already helps is to their credit, and it makes it possible to judge what the method itself contributes.

Critical weaknesses

The evidence does not support the claims about why PerFact works or which of its components matter. In Table 2 (Qwen2.5-7B), steering the shared heads with a random direction recovers about 75% of PerFact’s factual gain on TruthfulQA (Fact Acc 59.29→64.60, against 66.37 for PerFact). On CommonsenseQA it gives the same Pers. Rate as PerFact (93.81). Removing the harmful-component step, the paper’s central idea, changes results by a net two test questions on each dataset (TruthfulQA Fact Acc 65.49 vs 66.37, out of 226 test questions; CommonsenseQA 84.66 vs 85.25, out of 339). Removing the factual direction lowers Fact Acc by 1.3 points (TruthfulQA) and 0.6 (CommonsenseQA). Removing the personalization direction changes Pers. Rate by 0.4 on TruthfulQA and raises it on CommonsenseQA (95.28 vs 93.81). The explanation that the gain comes from “a sensitive shared region” is not tested, for example by steering other heads.

The entanglement claim also lacks a null reference. The similarities in Figure 2 (medians 0.34–0.75) have no random or shuffled baseline, and the paper does not explain how positive alignment between the two directions supports a trade-off. The PIH-versus-normal comparison conditions on the outcome (base correct and personalized wrong vs both correct), so a larger shift may reflect the answer change itself. Figure 2 reports no sample counts or tests. Eq. 11 centers the PIH residuals before the SVD, so the mean PIH shift is not part of the retained subspace by construction. Appendix A.1 promises to characterize alignment with the factuality direction, but Appendix A contains only Proposition 1, the standard minimal-modification property of an orthogonal projection, which says nothing about whether the removal improves factuality. Table 1 reports no non-personalized accuracy and no PIH rate after PerFact, although PIH (Eq. 2) is the paper’s own definition of the failure.

The gains are small, rest on single runs, and are framed favorably. Results are single greedy runs, with no variance or repeated runs, on test sets of 226 (TruthfulQA), 339 (CommonsenseQA) and 231 (MMLU-Law) questions (Table 3), and the reported accuracies are whole counts of these. Some gains are a net change of one question, such as Llama MMLU-Law Fact Acc (+0.43, 1/231) and Llama CommonsenseQA Fact Acc (+0.30, 1/339). Table 1 reports changes relative to “Original” only. Against the “Personalized” row, PerFact has a lower Pers. Rate in 5 of 6 settings (for example Qwen TruthfulQA, 87.17 vs 93.81), and its Overall gain is under 1 point in 4 of 6. On PFQABench (Table 7), P-Score is nearly constant across methods (37.2–38.8), and the Qwen Overall scores of CAA, StyliTruth, SPACE and PerFact span only 0.6 points (47.0–47.6). The Table 1 setting for Llama TruthfulQA (k=100, α=2) is the best cell of the Table 4 grid, and 5 of 12 cells have a Pers. Rate below the original model’s 74.34. Section 3.1 reports that the relevance-gating prompt (P4) substantially mitigates PIH, but Table 7 has no prompting baseline.

The most direct prior work, Sun et al. 2026, introduced PFQABench and, per its abstract, an inference-time method (FPPS) and the “entanglement” explanation. This paper uses the benchmark but does not compare against the method. It also does not compare against ITI (Li et al. 2023, cited) or PerFit (Liu et al. 2026, cited). StyliTruth and SPACE, designed for other problems, were adapted. As the paper itself describes, SPACE already localizes a shared subspace with multiple probes, so the novelty is narrower than the introduction suggests.

Questions for authors

In Table 2, a random direction on the shared heads gives most of the factual gain, and removing the harmful-component step changes results by about two questions. What evidence shows that harmful-component removal and shared-head selection, and not perturbing these heads in general, produce the improvement? Is the removal effect distinguishable from run-to-run variation? Results are single greedy runs on test sets of 226, 339 and 231 questions, and several gains are one question (0.43 = 1/231 for example)? Authors may want to justify why the improvements in Tables 1, 2 and 7 are reliable.

Results are single greedy runs on test sets of 226, 339 and 231 questions (Table 3), where one question is 0.44, 0.29 and 0.43 points. Several gains are a net change of one question (Llama MMLU-Law Fact Acc +0.43, Llama CommonsenseQA +0.30). Which improvements in Tables 1, 2 and 7 are reliable?

Table 1 reports changes relative to “Original”, but the “Personalized” row has a higher Pers. Rate than PerFact in 5 of 6 settings. What do these two rows correspond to, and how should the claim of gains on both objectives be read against the “Personalized” row?

The paper defines the failure by the PIH rate (Eq. 2), but Table 1 reports neither the non-personalized accuracy nor the PIH rate after PerFact. What are they for each method? Does PerFact’s factual direction also raise accuracy without personalized context?

The entanglement claim rests on cosine similarity between the two CAA directions (Figure 2). What similarity would random or shuffled directions give, and how does a positive alignment show that personalization harms factuality? How many PIH and normal samples are in the shift analysis, and does the larger shift hold for samples where the answer is unchanged?

Table 4 contains the configuration reported in Table 1 (61.06 / 80.53), and 5 of 12 cells have a Pers. Rate below the original model. Which split was used for Table 4, and how were k and α selected for Table 1?

Section 3.1 reports that relevance-gating prompting (P4) substantially mitigates PIH. How does PerFact compare with it on PFQABench, and why is FPPS from Sun et al. 2026 not a baseline? How were StyliTruth and SPACE adapted and tuned?

Table 5 reports accuracies of 0.428 (Qwen) and 0.144 (Llama) with personalized context on PFQABench-RAG, while Table 7 reports F-Scores of 53.6 and 38.8 for RAG. Do the two tables use different protocols or subsets?

Reviewer_FcdsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies personalization-induced hallucination, where adding user context reduces factual accuracy. It proposes PerFact, an inference-time steering method that identifies attention heads relevant to both factuality and personalization, removes estimated harmful components from personalization directions, and refines a joint steering direction.

Critical strengths
  1. The paper investigated the compromise between fatuality and personalization/faithfulness, which is an important failure mode of personalized assistants and need more attention from the community. Its behavioral comparisons, including length-matched irrelevant-context controls, provide useful evidence that factual degradation involves more than simply adding context.

  2. The proposed approach PerFact improves both reported metrics over the original personalized models across the main evaluation settings. The TruthfulQA results are particularly encouraging.

Critical weaknesses
  1. The contribution of the central purification mechanism is insufficiently established. In Table 2, this step adds only 0.88 points in factual accuracy on TruthfulQA and 0.59 points on CSQA, corresponding to roughly two additional correct answers in each case. Random-direction intervention also captures much of the TruthfulQA gain. Without uncertainty estimates or stronger controls, it is difficult to tell how much the PIH-specific construction contributes. The paper also does not show whether the later fusion and refinement steps preserve the reduction in the estimated harmful components.

  2. The factuality evaluation does not fully support the broader claims about reducing hallucinations. On PERG, it checks which answer option the response selects, so an explanation can contain errors and still receive full credit. Table 15 illustrates this: PerFact selects the correct option but incorrectly says that the wife contracted the disease from her husband, reversing the scenario. The main intervention results also do not report the paired PIH rate. Higher overall accuracy alone does not tell us whether fewer previously correct answers become wrong after personalization.

Questions for authors
  1. How are jointly factual and personalized positive samples identified from the separate HDBSCAN results? Does the final refined direction retain reduced projection onto the estimated PIH subspace?

  2. Do the gains from harmful-component removal persist across repeated fits and paired confidence intervals? How does it compare with removing a random subspace of the same rank and with refinement without purification?

  3. What are the post-intervention PIH rates relative to the same non-personalized responses? Does an audit of complete explanations, rather than selected options alone, support the claimed factuality improvements?

2 comments

Dear Reviewer,

Thank you for your thoughtful review and constructive feedback on our work. We sincerely appreciate the time and care you have devoted to evaluating our paper.

As the rebuttal period closes in approximately four hours, we wanted to follow up on our response. If your schedule permits, we would be grateful to hear whether it addresses your concerns and whether any points would benefit from further clarification. We would be happy to elaborate within the remaining time.

Thank you again for your time and valuable feedback.

Best regards,

The Authors

The rebuttal addresses many of my concerns by adding experiments with new controls, although the gains from purification remain modest, and the added evaluation and human audit are still missing some details. Overall, I consider increase my assessment, but I still view the paper as a borderline case rather than a clear accept.

Comments on the paper

1 comment

Thank you for the detailed responses and additional experiments. I appreciate the authors' efforts to address my questions, and I believe that the responses have satisfactorily resolved my other concerns. Overall, I think the work is moving in a positive direction. However, I still have some concerns about the baseline FPPS comparison, so I will maintain my original rating. I believe that including FPPS results under the experimental settings of Tables 2 and 9 would provide a more direct comparison with the proposed method. Without this comparison, I still find it somewhat difficult to fully assess the relative effectiveness of the proposed approach against this baseline.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.