- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies over-personalization: LLMs applying stored preferences selectively in the current context (each attribute in a set of preference attributes could either be applied or suppressed in the generation). The authors first identify three potential sources of over-personalization errors: lack of knowledge, wrong decision, and wrong generation, and then use linear probes to show that the knowledge is retained, and use an explicit Decide+Answer format to show that most errors are caused by wrong decisions, yet the responses could follow decisions correctly. The authors then identified that coupling the answer with the decision shifts the decision logits toward Apply for all preference attributes (compared to only asking the LLM to make the decision), but the AUC among attributes is well preserved, which the authors call a "bias" in the decision. The authors then proposed two fixes: subtracting a scalar for all attributes (which is estimated on the validation set), or making the decision separately before answering.
Critical strengths
- The diagnosis is cleanly organized. It narrows the cause step by step: first among the knowledge, decision, and generation stages, then, within the decision stage, between response bias and loss of sensitivity.
- The controls are careful: a neutral two-step prompt, gold-prefix scoring, prompt-wording variants, a shuffled-label probe control, and noise margins fixed in advance.
- The two fixes are simple and follow directly from the diagnosis. Subtracting a single bias scalar and separating the decision from generation are both cheap and easy to interpret.
Critical weaknesses
-
Although the diagnosis is cleanly organized and well controlled, the behavioral pattern it uncovers is rather immediate and contributes limited understanding to the community.
-
The performance of the "Debiasing" intervention is limited and slightly contradicts the phenomenon of "sensitivity preserved." A pure shift could be undone by one constant, yet debiased Decide+Answer is worse than Direct Decision on both AR and SR: Ministral-8B on BenchPreS goes from to , and Gemma from to . One possible reason is that the AUC evidence uses gold prefixes, while real decoding conditions on the model's own earlier labels, where errors can cascade. In that case, maybe reporting whether "sensitivity preserved" still holds under pred-prefix, and reporting which prefix is used when estimating lambda, would be helpful.
-
The experiments are limited: there are two small benchmarks; the fixes aren't compared with simple alternatives such as an instruction to suppress, or the mitigations proposed in the benchmark papers. This raises concerns about whether the phenomenon uncovered and the intervention proposed are generalizable
-
Presentation. The signal-detection-theory framing adds terminology without adding analysis. The actual measurements are a mean logit shift and an AUC compared between prompt conditions, and they can be stated directly. Section 4 lists all conditions, prefix modes, and metrics before saying what each hypothesis predicts. The verdict appears only in Section 4.3, and it is slightly overstated ("proving," "purely").
Questions for authors
Please see weaknesses above
W1.
While the pattern may seem intuitive, to our knowledge no prior work has systematically analyzed why it arises. Our contribution is identifying its source. We localize the failure to the decision stage and show that merely stating a generation objective shifts the decision criterion toward Apply while discriminability is largely preserved. This was not obvious in advance: leakage could also have stemmed from applicability no longer being encoded under the generation prompt, from a loss of discriminability, or from insufficient reasoning, and our experiments rule out each of these.
W2.
λ (ABS_GD^held-out) is estimated with pred-prefix on the held-out split and frozen before decoding, matching ordinary decoding.
Discriminability with pred-prefix. We apply the same decision rule (ΔAUC_GD ≥ −ε_AUC; Appendix E), re-estimating ε_AUC with pred-prefix on the held-out split. Discriminability (sensitivity in SDT) is preserved in 9 of 12 cells:
The three exceptions (†) include both cells you cite.
Why debiased D+A falls below Direct Decision. The subtraction is an analytical intervention to test whether the Apply bias drives leakage. It generally raises decision accuracy and lowers PLR (Table 3), linking the bias to leakage. Because λ is the mean shift and we do not assume that every preference shifts by this amount, subtracting it moves the decision threshold rather than restoring Direct Decision exactly.
Role of earlier decisions. In the three exceptions, ΔAUC_GD drops further with pred-prefix than with gold-prefix are where ΔABS (pred − gold) is the largest across datasets for each model (Table 2). Across all cells, however, ΔABS (pred − gold) is positive in six and negative in six, so the effect is not consistent, and with gold-prefix ABS_GD stays positive in all 12 cells. Errors in earlier decisions thus coincide with the exceptions but do not cause the shift.
§5.1 now states that λ is estimated with pred-prefix, and we will add the pred-prefix AUC table to the appendix.
W3.
Benchmarks. To our knowledge, BenchPreS and RPEval are the only benchmarks that label whether each stored preference should be applied or suppressed in the given context, so we use both, covering explicit and implicit preferences (3,557 labeled).
Suppress instruction. Our prompts already include one. Decide+Answer tells the model to ignore the preferences it labeled Suppress (Table 11), yet leakage stays high.
RP-Reasoner. We compare against RP-Reasoner, the RPEval's mitigation (Feng et al., 2026). Like Des2Gen, it decides before generating, but in two reasoning calls (query likelihood and intent prior) rather than a single Direct Decision call. We only changed its output format and mapped Support/Dominate to Apply. Its two-call procedure yields no higher SS than a single Direct Decision call on any dataset, and its responses are no better than Des2Gen’s.
W4.
SDT framing. In Section 4, we removed the SDT background (the analogy to human decision-makers and the 𝑑'/𝑐 discussion) and now state directly that ABIDE measures discriminability and response bias from the decision score. We replaced "sensitivity" with "discriminability," cut the metric definitions from five equations to two, replaced "cascade amplification" with ΔABS (pred − gold), and moved the hit/false-alarm paragraph to Appendix E.1.
Order of Section 4. Both hypotheses are now stated as predictions about the decision score: a loss of discriminability predicts more overlap between Apply and Suppress scores, and a response bias predicts a shift toward Apply. The captions of Figure 4 and Table 2 now state the result directly.
Overstated verdict. We toned down overstated claims throughout the paper. §4.3 now says the results "indicate" the shift is driven "primarily" by the Apply bias, rather than "proving" it is driven "purely" by it. The introduction and conclusion now say discriminability is "largely preserved" rather than "preserved," and §5 says offsetting the bias "reduces leakage" rather than "repairs the behavior."