- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work proposes PULSE, a framework for long-term LLM personalization that combines evidence-grounded profile updating with selective preference utilization. It introduces tri-state profile updates and criterion-guided residual rewards to improve personalized response ranking.
Critical strengths
-
This work addresses two relevant challenges in long-term LLM personalization: determining when user profiles should be updated and when stored preferences should influence response evaluation.
-
PULSE combines evidence-grounded profile updating with request-specific criterion construction and residual reward modeling. This structured design improves the traceability of profile revisions.
-
The experiments cover both static and evolving preferences, seen and unseen users, and two backbone models. The results show consistent improvements in overall response-ranking performance.
Critical weaknesses
-
The central claim of selective personalization is not sufficiently isolated from ordinary instruction following. In the case study, the user explicitly requests a "choose-your-own-adventure" format, yet PULSE derives essentially the same criterion from the user profile. The resulting ranking improvement therefore does not clearly demonstrate the additional value of historical preferences. A query-only criterion baseline is needed to distinguish genuine personalization gains from improved evaluation of explicit request requirements.
-
The experimental evaluation is limited to offline candidate ranking on a single benchmark, without demonstrating improvements in open-ended response generation or long-term interactive personalization. Moreover, comparisons with closely related criterion-based reward modeling approaches are missing.
-
The provenance-aware design does not fully guarantee that profile updates are semantically supported by historical evidence. The training objective primarily checks the structural consistency of evidence references rather than whether the cited interactions actually entail the inferred preferences. Unsupported preference interpretations may still be incorporated into the profile despite having valid citations.
-
The tri-state update mechanism assigns a single update state to the entire profile, potentially overlooking situations where different preferences require different update decisions. Furthermore, the reported update-state Macro-F1 is only 54.3%, and the evaluation includes just six reference cases of long-term revision.
-
The profile updater is optimized using downstream utility computed by the learned personalized reward model, creating a risk of reinforcing existing reward-model biases rather than correcting inaccurate user profiles. Although the authors introduce evidence-consistency constraints, they do not sufficiently investigate this potential self-reinforcement or the stability of repeated profile updates.
Questions for authors
Pls see the weakness
3 comments
Q3: The provenance-aware design does not fully guarantee that profile updates are semantically supported by historical evidence. The training objective primarily checks the structural consistency of evidence references rather than whether the cited interactions actually entail the inferred preferences. Unsupported preference interpretations may still be incorporated into the profile despite having valid citations.
A3: We thank the reviewer for this important concern. We would like to clarify that PULSE distinguishes structural provenance constraints from semantic evidence support.
(1) The update objective goes beyond structural citation consistency. The attribution term () enforces traceable evidence references, but it is only one component of the complete update objective. As defined in Eq. (4), the updater additionally optimizes current and future preference-ranking gains, preference-drift consistency, and profile-growth regularization. Therefore, candidate update traces are evaluated not merely by the validity of their citations, but also by their downstream utility on labeled preference comparisons.
(2) Semantic evidence support is explicitly evaluated. RQ3 introduces Evidence Support Rate (ESR), which directly assesses whether newly introduced or revised profile statements are semantically supported by their cited historical interactions, rather than merely containing valid evidence IDs. As shown in Table 2, PULSE achieves 82.2% ESR, compared with 72.4% for Prompted Tri-State, representing a 9.8-percentage-point improvement.
These results show that PULSE does not rely solely on structural citation validity: its profile updates are additionally guided by downstream preference utility, while their semantic evidence support is independently assessed through ESR. The observed improvement provides empirical evidence of more reliable evidence-grounded profile updates.
Q4: The tri-state update mechanism assigns a single update state to the entire profile, potentially overlooking situations where different preferences require different update decisions. Furthermore, the reported update-state Macro-F1 is only 54.3%, and the evaluation includes just six reference cases of long-term revision.
A4: We thank the reviewer for raising these concerns. We clarify the distinct roles of update-state prediction and evidence-grounded profile generation, and provide additional human-validation results.
(1) Why a single update state is meaningful. The tri-state label specifies the scope of revision justified by the recent evidence, rather than assigning the same edit operation to every profile statement. Specifically, No Update preserves the existing profile; Local Refinement incorporates context-bound preferences without replacing established long-term preferences; and Long-term Revision permits changes to persistent preferences when supported by evidence.
Importantly, the update state determines what kind of revision is warranted, while evidence-grounded generation determines which statements to change. Under Long-term Revision, for example, the generator may revise an outdated persistent preference while retaining unrelated preferences and their provenance. Thus, a single update state does not imply uniform changes across the profile. The distinction between Local Refinement and Long-term Revision remains essential because it explicitly controls whether established preferences can be revised.
(2) Update-state performance and human validation. On the original LLM-adjudicated references, PULSE achieves 54.3% Macro-F1, outperforming Prompted Tri-State (40.8%) by 13.5 percentage points. To further examine the limited number of long-term-revision references, we conducted additional human validation on the same 175 instances:
| Reference | No Update | Local Refinement | Long-term Revision | PULSE Macro-F1 |
|---|---|---|---|---|
| LLM-adjudicated | 52 | 117 | 6 | 54.30% |
| Human-validated | 66 | 73 | 36 | 57.61% |
Human validation identifies 36 long-term-revision cases, compared with only six in the original LLM-adjudicated references. Under these human-validated labels, PULSE achieves 57.61% Macro-F1.
Together, these results clarify the functional role of tri-state attribution and provide additional human-validated evidence for its effectiveness.
Q5: The profile updater is optimized using downstream utility computed by the learned personalized reward model, creating a risk of reinforcing existing reward-model biases rather than correcting inaccurate user profiles. Although the authors introduce evidence-consistency constraints, they do not sufficiently investigate this potential self-reinforcement or the stability of repeated profile updates.
A5: We thank the reviewer for raising these concerns. We clarify two aspects of PULSE's training and evaluation.
(1) Reward-model self-reinforcement. PULSE's updater is not optimized by maximizing reward scores on its own generated responses. Instead, candidate update traces are evaluated against dataset-provided preference labels. Specifically, Eqs. (4–8) measure whether an updated profile improves the ranking margin between labeled preferred and rejected responses, relative to the frozen profile-free anchor, on both the current request and subsequent requests. For preference-drift cases, the objective additionally measures whether the updated profile favors the current preference over the outdated-preference distractor. Furthermore, the criterion inducer and reward model remain fixed during updater optimization. Thus, update supervision is tied to externally provided preference comparisons, rather than treating the reward model's own predictions as ground-truth preference labels.
(2) Stability under repeated updates. Our evaluation does not assume oracle profiles at each interaction. As described in Section 4.1, PULSE updates profiles autoregressively using its own previous predictions, without reference-profile teacher forcing. Consequently, the results in Table 1 evaluate downstream personalization using profiles generated through successive updates, rather than reference-profile teacher forcing. The results in Table 1, covering both static and evolving preferences, therefore evaluate personalization under successive model-generated profile revisions. Additionally, the explicit No Update candidate and profile-growth regularization discourage unnecessary changes during training.
Q1: The central claim of selective personalization is not sufficiently isolated from ordinary instruction following. In the case study, the user explicitly requests a "choose-your-own-adventure" format, yet PULSE derives essentially the same criterion from the user profile. The resulting ranking improvement therefore does not clearly demonstrate the additional value of historical preferences. A query-only criterion baseline is needed to distinguish genuine personalization gains from improved evaluation of explicit request requirements.
A1: We thank the reviewer for highlighting the importance of distinguishing historical personalization from explicit instruction following. We clarify the case study and provide the requested Query-only baseline.
(1) Clarification of the case study. Figure 4 primarily illustrates how PULSE updates an outdated preference for numbered options to reflect the user's newly expressed preference for branching choices, while preserving the supporting evidence. To directly assess the additional contribution of historical preferences beyond explicit request requirements, we conduct the following controlled experiment.
(2) Additional Query-only criterion baseline. We introduce a Query-only baseline that constructs evaluation criteria exclusively from the current request, without accessing historical interactions or user profiles. It uses the same frozen anchor, candidate responses, and evaluation protocol as PULSE, thereby isolating the contribution of history-grounded personalized criteria.
PULSE consistently outperforms Query-only across all four settings, with Top-1 improvements ranging from 0.37 to 1.95 percentage points. Since Query-only already captures requirements explicitly stated in the current request, these results provide direct evidence that incorporating historical preferences yields additional personalization benefits beyond ordinary instruction following.
Q2: The experimental evaluation is limited to offline candidate ranking on a single benchmark, without demonstrating improvements in open-ended response generation or long-term interactive personalization. Moreover, comparisons with closely related criterion-based reward modeling approaches are missing.
A2: We thank the reviewer for these valuable suggestions. We clarify the scope of our long-horizon evaluation and the differences from closely related criterion-based methods.
(1) Long-horizon personalization and open-ended generation. PULSE is designed to improve evidence-grounded profile updating and personalized response evaluation, rather than directly modifying the response-generation policy. As formulated in Section 2, its objective is to select the preferred response from a candidate set using the updated user profile. Therefore, Top-1 accuracy and MRR directly measure the effectiveness of the proposed personalization mechanism. Evaluating methods on identical candidate responses also isolates personalization gains from differences in response generation.
Regarding long-horizon personalization, HorizonBench provides chronological, multi-session histories covering both static and evolving preferences. Crucially, PULSE updates user profiles autoregressively using its own previous predictions, without reference-profile teacher forcing. Consequently, subsequent response evaluations depend on earlier profile updates, allowing the experiments to assess personalized response selection throughout evolving interaction histories.
(2) Additional comparisons for criterion-based personalized evaluation.
To further examine the effectiveness of criterion-based personalization, we complement the existing w/o Criterion-Guided Reward ablation in RQ2 with two additional controlled baselines: Anchor-only, which removes the personalized residual, and Query-only, which constructs evaluation criteria solely from the current request without accessing historical preferences. Both use the same frozen anchor and candidate responses as PULSE.
PULSE consistently outperforms both baselines across all four settings. Together with the RQ2 ablation, these results demonstrate the benefits of criterion-guided personalized evaluation, historical preference information beyond the current request, and residual correction beyond the frozen anchor.