acceptodds
Under review as a conference paper at ICLR 2027

Personalized but Unchecked: Safe Personalization Requires Conditional Control in Medical Advice

Abstract

Safe personalization requires following user preferences when doing so is safe and overriding them otherwise. In medical advice, however, multiple language models can recognize clinical constraints and follow safe preferences separately, yet fail to combine these abilities reliably. We propose a two-stage explanation: preferences can impair patient-specific judgment formation and prevent correct judgments from constraining treatment recommendations. Repair should therefore support both reliable judgment and constraint-guided action. Controlled dialogues behaviorally localize these failures: preferences introduced before judgment can distort its formation, while those introduced after a correct self-generated judgment can redirect recommendations. Internal analyses and causal interventions link these failures to a shared component carrying clinical information. Switching between clinical-decision and preference-fulfillment framing shifts both judgments and recommendations, and the same clinically relevant component participates in this framing effect at recommendation. Intervening in the component at judgment and recommendation jointly recovers safe advice missed by either intervention alone, supporting complementary repair. Following this direction, training models to follow preferences subject to clinical constraints improves safety while preserving safe preference following, whereas suppressing preferred answers at matched safety incurs greater personalization costs. Benefits extend to single-turn and free-text advice, while further evaluations examine reasoning gains and non-medical capability retention. Together, these findings provide a unified account of safe-personalization failures and a repair principle grounded in conditional control across judgment and recommendation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Safe Personalization, Large Language Models, Clinical Decision-Making, Mechanistic Interpretability, Preference Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_VjeYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The authors aim to better understand model behaviour in clinical scenarios where patients ask for health-related assessments and recommendations. Specifically, they investigate how the model's default behaviour is influenced by patient preferences. They first approach this from a mechanistic perspective, using causal interventions to identify which layers are responsible for these behavioural changes. They then fine-tune the model to improve its performance on this task, aiming to reduce discrepancies in its behaviour. Finally, they investigate whether the changes in latent representations induced by fine-tuning can be transferred back to the base model.

Critical strengths
  • The problem addressed is an interesting and relevant issue with LLMs, and it is well motivated through a clear, real-world example.

  • The paper follows a nice structure, starting with experiments aimed at identifying and better understanding the issue, before moving on to explore a potential solution.

  • The experimental setup is ok and investigates the problem from different perspectives. The authors first examine the model's internal mechanisms through several analyses using causal interventions and then fine-tune the model to improve its performance on the task. I also appreciated the analysis of how transferable the latent representations produced by the fine-tuned model are to the base model.

Critical weaknesses
  • Although the overall structure is reasonable, the paper is not always easy to follow. The writing is dense and sometimes fragmented. Moreover, since the paper moves directly into experiments and results, it would benefit from a higher-level overview of the hypothesis and rationale behind each experiment. This is especially noticeable in Section 3, where the authors immediately describe what they did and the results they obtained, without spending enough time explaining the experimental setup (e.g., at which tokens the activations are injected?), stating the hypotheses, and clarifying the expected outcomes. As a result, some sections feel more like a report of experimental results than a research paper.

  • The experimental setup is generally reasonable, but I would like to see whether the findings generalise to at least one additional model for Section 3 results (causal interventions), preferably from a different model family.

  • Is there a drawback to choosing the same layer for both tasks? Looking at Figure 2a, later layers also seem to have a stronger impact.

  • I think my main concern is how the experimental setup and results are presented. The presentation feels somewhat disorganised, making it difficult to identify the main findings of each section on a first reading. More effort should be put into highlighting the key outcomes and guiding the reader through the hypothesis behind each experiment, its main findings, and why these findings are (or are not) aligned with the initial expectations.

  • Figure 1 must be redrawn, fixing the visible AI-generated artifacts.

Questions for authors

See above

Reviewer_nzDrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies whether patient preferences interfere with recognizing contraindications and making safe treatment recommendations. It separates these two stages through controlled prompts, investigates an attention module using internal interventions, and compares different fine-tuning objectives.

Critical strengths
  • Changing the clinical condition while keeping the preference fixed, and testing both preferences when both treatments are acceptable, provides a meaningful control. This makes the behavioral finding more informative than reporting unsafe responses alone.
Critical weaknesses
  • The task definition needs to be clearer. After reading the abstract and introduction, I was still unsure what patient information the model receives and what it is expected to produce. Please explicitly explain what are included in the constructed clinical vignettes and describe the inputs, candidate treatments, and expected outputs before introducing the mechanistic analysis.

  • A complete worked example in Figure 1 would substantially improve readability. Please add the actual patient vignette, preference, treatment options, and expected judgment and recommendation. The paired version should highlight the single clinical fact that changes treatment eligibility. This would help with clarify how detailed the cases are and how patient-specific information determines the contraindication. Without a concrete example, it is difficult to interpret what the experiments are testing.

  • The dataset has limited clinical coverage, and the validation process needs more detail. The cases cover relatively few treatment-contraindication relationships. Although a physician and a pharmacist reviewed the evidence and treatment options, it is unclear whether every complete vignette was independently reviewed, how disagreements were resolved, and whether the cases contain sufficient information to establish both the contraindication and the suitability of the alternative. Please describe this process and clarify how the limited case diversity affects the generalizability of the findings.

  • The two-stage findings may partly depend on the forced-choice dialogue format. The first turn asks which treatment is contraindicated, while the second asks which treatment is recommended. Both use option letters, and the option order can change between turns. This introduces additional demands involving answer-label tracking and switching from identifying the unsafe option to selecting the safe one. Although option order is counterbalanced, I would like to see results separated by whether the orders agree, and a control that states the judgment explicitly using treatment names. This would help establish that the observed failure is about using clinical knowledge rather than the particular response format.

  • The free-text results do not consistently support the strongest repair claims. Table C8 shows that the benefits depend substantially on the model and training objective. In Qwen, joint supervision reduces paired correctness relative to the base model; in Llama, recommendation-only supervision increases unsafe recommendations. Direct recommendation supervision has the highest reported free-text paired correctness in both models. These results are important because they make it unclear what the two-stage supervision adds over simpler direct fine-tuning for ordinary advice. Please bring this comparison into the main results and compare methods directly, rather than mainly comparing each method with its base model.

Questions for authors
  • Important qualifications are buried in the appendix. The free-text method comparison and the uncertainty around the mechanistic controls should be summarized in the main paper. These directly affect how the central claims should be interpreted.
Reviewer_FC9DOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies safe personalization in a medical context. They study how LLMs respond to user preferences when it’s safe vs. unsafe to follow them; they create a conflict and observe the LLM’s behavior in response. They then claim this failure can happen at two steps, judgement (not seeing the failure) and recommendation, which is basically the already-studied know vs. act conflict. They then run a mechanistic investigation and compare four fine-tuning strategies as a fix.

Critical strengths

I think the underlying phenomenon in this paper is not new. It has been widely known that when there is a conflict between personalization and safety, models can collapse, but they study it in the medical domain. I think the best part of this paper is the behavioral study alone. The benchmark is small, but it can be a useful resource if scaled up while keeping its rigor and validation. The behavioral part of the paper is sound and carefully designed.

Critical weaknesses

I believe this paper has a problem in presentation. This paper could be a strong case for conferences focused on the medical domain if it kept the behavioral and fix parts and removed the mechanistic part. The benchmark, as I mentioned in the strengths, is a valuable contribution: it addresses an important problem, it has a clean design, and if scaled it can be really useful. But right now it is too small in scale to make broad judgements upon.

The benchmark itself is very small: effectively 24 distinct medical facts, and the six-model comparison uses about 20 cases. I strongly suggest the authors scale the benchmark.

Although the authors report that it was medically validated, there is missing information on who reviewed it. The authors should state the reviewers’ specialties and agreement statistics. Also, how the vignettes were created isn’t described or shown.

In my opinion, the mechanistic part was added to provide an explanation but is more of a post-hoc addition to the paper. The mechanistic part only weakens the whole paper and does not contribute meaningfully other than showing that we also have some internal insights, which could be given for any behavior in an LLM. The localization is not a unique finding; with the method used, it is almost guaranteed to find a responsible layer. It also doesn’t play any role in the repair: the four training objectives don’t touch the interpretability side, and it is only done on a single model.

The paper also has a thin evaluation. There are no frontier closed models (Claude or GPT especially; since the dataset is very small, this would be feasible), and for this important medical study, they don’t do any medical-capability retention check.

If the authors still want to include the mechanistic side, they should use it for a purpose, such as monitoring, repair, or making predictions. Right now it reads awkwardly in the paper, as it doesn’t contribute much but is stated as a main contribution and takes up space that could have gone to more experiments on the behavioral side.

Figure 1 is really not informative. It doesn’t really give an overview or add any clarification to the design; by itself it is just a pipeline and doesn’t really contribute to the paper.

Questions for authors

Does the mechanistic analysis inform the repair in any way? If not, why is it presented as a main contribution?

I highly suggest scaling up the benchmark while maintaining the same level of rigor and validation. This benchmark has the potential but it needs revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.