- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper examines generative recommendation using a partition based on recurring cross-user item transitions. An instance belongs to the Memory subset when the transition from its last history item to its target occurs in another user's training history; the remaining instances form the Generalization subset. The reference index uses training data only and excludes the current user. MemOPSD trains a same-architecture teacher on Memory data and then post-trains a student with the original backbone objective on all instances plus teacher distillation on Memory instances. Distillation is evaluated at prefixes sampled from the current student, with teacher and student conditioned on the same prefix. The deployed student requires neither the teacher nor target-dependent subset labels. Experiments on three Amazon Reviews 2023 datasets and four generative backbones report consistent Overall and Memory improvements, with mixed effects on the Generalization subset. Additional experiments examine teacher and distillation configurations, model size, language model backbones, runtime, and gradient conflicts.
Critical strengths
1. The supervision scope and distillation mechanism form a coherent, testable design. Sections 3.1–3.4 clearly separate the observed-instance label, which determines whether to distill, from the student-generated SID prefix, which determines where teacher guidance is evaluated. The training-only reference construction and exclusion of the current user are explicit, and inference uses a single student. Table 2 tests continued backbone training and ground-truth-prefix KD, while Table 9 separately examines teacher data, distillation scope, normalization, and additional Memory CE. In particular, Group F improves Overall and Memory performance over Group E across the three datasets and both reported metrics. These controls support a contribution beyond merely adding more supervised updates on Memory samples.
2. The empirical gains are consistent across several backbones, and the paper reports important boundaries. Table 1 shows improvements for TIGER, LETTER, CARE, and LatentR3 across all three datasets. Tables 8 and 10 extend the analysis to larger configurations and several language model families. Table 7 explicitly reports the cases where Generalization performance declines, and Section 3.1 states that the partition does not establish out-of-distribution or compositional generalization. Appendix F.5 also distinguishes reduced gradient conflict from a causal explanation of the gains. This combination of positive evidence and explicit qualifications makes the empirical contribution more credible.
Critical weaknesses
1. The experiments expose a remaining supervision trade-off that the overall improvements do not resolve. Table 9 allows a more precise reading than the aggregate results alone. On Video Games, moving from Group D to full MemOPSD, Group F, changes the denominator from the full batch to the selected Memory samples while retaining the same teacher and Memory-only scope. Memory NDCG@10 increases from 0.1988 to 0.2085, but Generalization decreases from 0.0226 to 0.0217. Using the 14.35% Memory proportion in Table 6, the rounded values imply an Overall contribution of about +0.00139 from Memory and -0.00077 from Generalization. Thus, the better Overall score includes a substantial offsetting loss on the complementary subset. Table 7 also reports Generalization declines for CARE and LatentR3 on Office Products. The authors acknowledge trade-offs, and I do not interpret this as a hidden contradiction. The remaining weakness is that the evidence does not establish an improved Memory–Generalization trade-off at comparable supervision strength. Appendix C itself shows that the local relative KD strength depends on the Memory fraction as well as lambda. A matched-strength comparison or trade-off curve is needed to separate the value of selective transfer from a stronger emphasis on Memory under the chosen normalization.
2. The principal comparative claim lacks the most directly relevant external control. Section 2 describes adaptive MemGen-GR, which addresses the same memorization–generalization problem, and the experiments use its implementation framework. Yet Table 1 reports the constituent ID-based and generative models without the adaptive combination. The paper also cites collaborative memory augmentation, but does not compare with OMEGA. These are specific alternatives to the proposed way of exploiting cross-user knowledge. Table 9's internal controls, including the useful extra-Memory-CE control, do not determine whether MemOPSD improves the accuracy/cost trade-off over those approaches. Please prioritize a matched-data comparison and account for their different inference costs. The four named backbones provide useful within-backbone evidence, but Appendix E.3 places them in a shared lightweight T5 setup; those gains alone do not establish superiority over the broader set of published systems. This limits the demonstrated comparative contribution, even though the plug-in gains are consistently positive.
Questions for authors
-
Separate selection from effective weighting: For Table 9, were lambda and the extra-CE coefficient tuned with comparable validation budgets for each group? Please compare selected- versus batch-normalized distillation at comparable effective strengths and report the Memory–Generalization trade-off with paired uncertainty. How much of the Overall gain remains when Generalization performance is constrained to match the relevant baseline?
-
Closest-method comparison: Can adaptive MemGen-GR or OMEGA be evaluated on the same splits with an explicit accounting of serving and training costs? This directly tests the proposed memory-transfer contribution. Newer general competitors such as ACERec or LLaDA-Rec are additional context, with decoder compatibility assessed separately from benchmark comparability.
-
Statistical precision: Section 4.2 reports five-run averages, while Appendix E.3 lists seed 2026 and Table 1 provides significance marks without a test specification or variability. Please identify all seeds, the test and pairing unit, and confidence intervals for Overall and both subsets, especially the reported Generalization losses. Three Amazon categories with similar short histories also leave distributional scope limited, but that fact alone does not establish that the gains are accidental.
-
Check the cross-user exclusion in the released evaluator: In the code archive retrieved on October 18, 2026,
genrec/trainer.pybuildsFineGrainedEvaluatorfrom pooled training sequences and callsget_case_labels(item_seq)without the current user.mem_gen_categorizer.pythen tests transition membership without subtracting that user's contribution. In contrast, the training-label export explicitly performs leave-one-sequence-out exclusion. A transition recurring only in the evaluated user's own training history is therefore labeled Memory by this evaluation path, contrary to Equation (2). I verified that conditional behavior with a minimal source-level example, but have not measured its incidence in the paper's datasets. Does preprocessing guarantee that such repeated transitions cannot occur? Please identify the code path used for Tables 6–7 and provide either that guarantee/count or labels with consistent cross-user exclusion. This is an implementation question with a stated condition, not evidence that the reported Overall gains are invalid. -
Complete compute accounting: Appendix D openly excludes teacher training from Table 3, and the measured runs use R=1 although Table 5 selects R=2 or R=4 in several settings. Please report teacher-plus-post-training time for those selected configurations and identify the exact experiment commands. The anonymous repository is accessible through its source/download API and contains the core training implementation; this request concerns matching the reported experiments and costs, not a claim that code is absent.
2 comments
We apologize for following up again and fully understand that this is a busy period. As the discussion period closes tomorrow, we would be sincerely grateful for any brief feedback on whether our new comparison have addressed your concerns. Even a short comment would help us greatly.
Dear Reviewer,
Thank you again for your valuable comments and feedback. As the rebuttal deadline is approaching, we would like to kindly follow up on our responses, in which we have addressed your questions and concerns.
We would greatly appreciate it if you could revisit our responses and consider whether they warrant an upward revision of your score. If you have any remaining questions or concerns, please do not hesitate to let us know. We would be more than happy to provide further clarification.
Thank you for your time and consideration.
Best regards,
The Authors