- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
HYPREFLECT maintains explicit preference hypotheses inferred from interaction chunks and refines them through a bounded reflection window. The refined hypotheses condition both the teacher and student during self-distillation, while the teacher additionally receives privileged information from subsequent user interactions. Experiments cover explicit feedback, multi-session personalization, and implicit behavioral preferences across three backbone models, with additional analyses of generalization, hypothesis reuse, and component contributions.
Critical strengths
-
HYPREFLECT provides a well-motivated way to connect user modeling with response learning. Its bounded reflection window jointly revisits independently generated local hypotheses and an earlier consolidated state, allowing accumulated evidence to inform subsequent revisions. Conditioning both teacher and student on the refined hypotheses gives the method a clear interface between explicit user memory and self-distillation.
-
The empirical evaluation covers complementary personalization signals across three datasets and three backbone models. Comparisons with incremental summaries and hypotheses, reflective prompting, and the reflection ablation help assess the contribution of the proposed components. The unseen-user, cross-domain, and hypothesis-reuse experiments further support the practical value of the learned user representations.
Critical weaknesses
1.The main limitation is the strength of the demonstrated advance over existing self-distillation and incremental hypothesis maintenance. While the proposed integration is useful, the added benefit of reflective refinement over the closest alternatives is uneven. For example, on Flight with Qwen3.5-4B, SD-IncHyp achieves 74.8% accuracy, compared with 74.0% for HYPREFLECT and 74.2% for HYPREFLECT+Sum (Table 1). The reflection ablation on HiCupid yields gains of 1.4 and 0.7 percentage points (Table 5), without reported uncertainty estimates. Although larger improvements elsewhere support the approach, the evidence for a substantial and robust contribution from the proposed refinement mechanism remains less convincing than the effectiveness of the complete system.
2.The uncertainty-aware aspect is not sufficiently validated. Each hypothesis receives an LLM-generated score from 1 to 5, with prompt-defined levels of evidential support (Section 3.1 and Figure 11). While this is a reasonable representation, the experiments do not establish whether these scores improve refinement or response quality beyond the hypothesis statements themselves. A comparison with unweighted hypotheses or uniform scores is missing, leaving unclear whether the reported benefits depend on uncertainty information. Thus, the framework explicitly represents confidence, but the practical contribution of this component remains unestablished.
Questions for authors
-
Could the authors explain the explict situation when reflection offers a meaningful advantage over simpler hypothesis maintenance?
-
How much do the confidence scores contribute independently of the hypothesis text? A comparison using the same hypotheses with omitted or uniform scores would help establish their role, particularly when the evidence is ambiguous or conflicting.
2 comments
Thank you for the response. My overall assessment of the paper remains positive, and I will keep my original score.
We thank the reviewer for the thoughtful feedback and for highlighting how HypReflect connects explicit user modeling with response learning through self-distillation. We also appreciate the reviewer’s recognition of our evaluation across different personalization signals, including experiments on unseen users, cross-domain transfer, and hypothesis reuse. We address the weaknesses raised by the reviewer below and highlight how we intend to incorporate the changes in the revised paper.
As shown in the comparison between SD-IncHyp and HypReflect in Table 1, the reflection ablation in Table 5, and the qualitative examples in Figures 6 and 7, we find that reflection can help when preference evidence is spread across several chunks and the model needs to reconcile conflicting inferences or preserve specific preferences during consolidation. More specifically, on HiCuPID, reflection preserves specific preferences that direct consolidation obscures (Figure 6), and on Flight, it resolves conflicting inferences about price and duration (Figure 7). Empirically, adding reflection improves scores by 0.7–1.4 points on HiCuPID and 1.8–3.9 points on Flight (Table 5). HypReflect also outperforms SD-IncHyp, our baseline for simpler hypothesis maintenance in the continual update setting, in six of nine settings (Table 1).
While Table 1 shows that HypReflect outperforms SD in eight of nine settings across datasets and model backbones, we agree with the reviewer on the value of a multi-seed comparison and report results from three random seeds below.
HelpSteer. We note that Table 1 already reports scores averaged over eight user profiles on evaluation set. For the three-seed experiment, we trained the Qwen3.5-4B assistant with a user simulator configured with the concise, casual beginner profile. The simulator provided feedback specific to each interaction, and we used the same evaluation set for all runs. The mean scores were 84.9 for SD and 87.6 for HypReflect, a gain of 2.7 percentage points. At each checkpoint (steps 0, 50, …, 250), we computed the sample standard deviation across the three seeds. Averaged across checkpoints, these standard deviations were 1.8 points for SD and 2.7 points for HypReflect.
HiCuPID and Flight. The three-seed results are broadly consistent with Table 1. HypReflect scores higher than SD on average for both datasets and both backbones, with modest gains on Flight. We also acknowledge that Flight is a tough setting for hypothesis extraction because preferences must be inferred from implicit choices alone, and the available context in the dataset often gives little insight into some synthetic choices (e.g., why a user would pick a 30-hour flight over an otherwise identical 8-hour flight). Here HypReflect is marginally better than SD. We will discuss this limitation in the appendix and include the table below, which reports the original scores alongside the mean ± sample standard deviation across seeds.
We thank the reviewer for suggesting this ablation. To further examine whether the confidence scores reflect meaningful differences in hypothesis quality, we evaluated each hypothesis individually by asking the model to predict the user's observed choice from that hypothesis alone. On Flight, where HypReflect generates up to three hypotheses per user, prediction accuracy increased with the assigned confidence score (
w) across all three models, as shown in the table below.To assess the contribution of explicit confidence scores, we trained Qwen3.5-4B separately on Flight with and without them. The model trained with confidence scores achieved 73.1% accuracy, compared with 71.8% without confidence scores, a gain of 1.3 percentage points.
These results demonstrate that explicit confidence scores provide useful information beyond the hypothesis text, while making the model's assessment of each hypothesis more transparent and interpretable. We will include both analyses in the revised paper.