- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes PraMem, which builds experiential memory consisting of user behavioral patterns and bias alerts for the model through prediction practice, reflection, and periodic updates over users’ historical behavior sequences. The method further introduces groundedness review based on historical sequence perturbation and generalizability review based on scene identification to filter proposals for memory updates. Experiments on three datasets show that the method outperforms the compared baselines in most scenarios.
Critical strengths
- Leveraging ground-truth feedback from historical behaviors to construct experiential memory is well motivated. It directly connects memory updates to the prediction objective without updating the underlying model parameters.
- The experiments cover three datasets and two prediction backbones, with notable accuracy improvements in scenarios such as advertising and live streaming.
- The paper provides ablations of the core modules and an analysis of memory evolution, and demonstrates that the constructed memory can be reused by another prediction model, offering preliminary evidence of its practical utility.
Critical weaknesses
- The groundedness review does not adequately distinguish perturbation sensitivity from groundedness in historical evidence. A bias-alert experience supported by repeated prediction errors may remain valid even when the user's behavioral pattern changes substantially. Its invariance to such perturbations does not necessarily imply a lack of historical grounding. Conversely, an unsupported proposal may be judged invalid after perturbation and thus pass the groundedness review.
- The generalizability review may conflate a lack of specificity with generalizability. Equation (7) assesses generalizability based on whether a proposal uniquely identifies the original scene. However, an overly general statement that cannot identify the original scene may have no predictive value, whereas a valid brand-preference experience may be rejected simply because the original scene is the only candidate involving that brand.
- The analyses in Table 3 and Figure 5 do not fully validate the proposed self-review criteria. The audit sample size, annotation criteria, and annotator agreement are not adequately described, and the precision and recall of self-review are not broken down by review criterion or experience type. Figure 5 supports the usefulness of the modules, but does not directly establish that their acceptance decisions reliably reflect groundedness and generalizability.
- The paper should clarify its novelty over existing feedback-driven methods. Reflexion [1] already uses task feedback to construct reflective memory, while DGDPO [2] diagnoses and revises user profiles based on discrepancies between simulated and actual user behaviors. Although Section 5 highlights bias-alert experience and self-review as distinctions, these differences do not establish a paradigm shift from context compression to practice-derived memory. A more focused comparison with feedback-driven approaches would help clarify the contribution.
[1] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao. Reflexion: Language Agents with Verbal Reinforcement Learning. NeurIPS 2023. [2] Hongyang Liu, Zhu Sun, Tianjun Wei, Yan Wang, Jiajie Zhu, Xinghua Qu. Diagnostic-Guided Dynamic Profile Optimization for LLM-based User Simulators in Sequential Recommendation. AAAI 2026.
Questions for authors
- How does the self-review procedure described in the main text correspond to the actual implementation? Equation (6) describes judgments based on a perturbed historical sequence, whereas Appendix F states that only a perturbation plan is provided without actually applying the perturbation.
- Does consensual experience adjustment require support from distinct historical events? Each practice sample generates ten proposals, while Appendix E allows a viewpoint supported by three proposals to be incorporated into memory. Must these three proposals come from different practice samples? How are repeated proposals from the same case prevented from being counted as multiple pieces of supporting evidence?
- Please clarify the evaluation protocol. How many users and prediction targets are used in each dataset? How are practice samples selected across the 80 rounds?