- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper highlights the limitation of uniformly optimizing all training instances in sequential recommendation and proposes a selective learning framework called SL4SR for resolving this problem. SL4SR first pre-trains a reference model on the full training set. It then estimates the relative learnability of each training instance using the loss difference between the current backbone and the reference model, and selectively updates the backbone using only high-scoring instances within each mini-batch. Experiments on several public datasets report consistent performance improvements over the reproduced baselines.
Critical strengths
- The paper is clearly written and well organized. The motivation and proposed method are easy to understand.
- The experiments cover multiple real-world datasets and different backbone architectures. The paper also provides a relatively comprehensive set of ablation studies and additional analyses to investigate the behavior of the proposed method.
Critical weaknesses
- SL4SR is highly similar to RHO-LOSS and seems to be a straightforward adaptation in sequential recommendation with limited methodological novelty. Compared to RHO-LOSS, SL4SR replaces the out-of-sample reference loss with the in-sample reference loss, but the motivation of this change are not sufficiently discussed or experimentally validated. Besides, the experimental configuration of the RHO-LOSS baseline is not described in sufficient detail. Important settings (e.g., the held-out subset ratio, reference model configuration, and sample selection ratio) are missing, making it difficult to assess the fairness and reproducibility of the comparison.
- The reported performance of several backbone models is substantially lower than previously published results. For example, compared with the full-ranking results of the original FMLPRec paper, the HR@10 results of SASRec and FMLPRec in SL4SR are consistently lower and even SL-enhanced models do not reach the previously reported performance.
Questions for authors
- Is the SASRec baseline reported in Table 2 exactly the same trained model checkpoint used as the Estimator for SASRec+SL?
- For the RHO-LOSS baseline, is the backbone trained on the full training set or only on the remaining subset after excluding the held-out data used to train the estimator? What is the held-out ratio? Is the backbone trained with the same masking ratio and training budget as SL4SR?
- What loss functions are used to train the estimator and the backbone? What negative sampling strategy is employed?
- The total training cost (including estimator pre-training and backbone training) should be reported. Besides, a comparison with SL4SR and RHO-LOSS under the same computational budgets would help clarify the practical efficiency of SL4SR.
2 comments
Dear eR5i,
We have provided responses addressing your concerns regarding (1) methodological novelty, (2) baseline performance, and (3) implementation details and computational efficiency. If you have any remaining concerns or questions, please let us know. We would be happy to provide further clarification.
Best regards,
Authors
We sincerely appreciate the time and effort. We hope our responses address your concerns.
x
The key distinction between SL4SR and RHO-LOSS lies in the role of the reference model and the motivation for its training strategy. RHO-LOSS employs a holdout-trained reference model to approximate irreducible loss, whereas SL4SR uses a pre-trained estimator to assess the current backbone's relative fitting behavior. In sequential recommendation, where user interaction data is often sparse, training the estimator on the full dataset allows it to leverage all available supervision and identify instances that the backbone has not yet sufficiently learned. Importantly, the estimator guides instance selection without directly transferring its predictions, while the backbone is optimized using ground-truth targets.
Regarding the RHO-LOSS implementation, we reserved 20% of the training instances as a disjoint holdout for reference model pre-training and used the remaining 80% for backbone optimization. Both methods employed the same backbone and estimator architectures, model configurations, optimizer, learning rate, batch size, maximum number of epochs, and early-stopping protocol. In Table 3, we applied the same masking ratio (ρ) to both methods within each dataset–backbone combination, retaining the top (1-ρ) fraction of candidate instances, corresponding to the RHO-LOSS selection fraction (nb/nB).
The difference in training data usage reflects the respective methodological designs of the two approaches. RHO-LOSS relies on a disjoint holdout for reference loss estimation, whereas SL4SR intentionally uses the full training set to establish an estimator-relative fitting reference. Accordingly, our comparison preserves each method's training protocol while matching the model configurations and sample selection ratios. We clarify these settings to ensure transparency and reproducibility.
x
Although the original FMLPRec paper also reports full-ranking results, absolute performance may vary across execution environments and experimental runs, even under similar experimental settings. We implemented the backbone models based on their publicly released code, preserving their original architectures and configurations.
Importantly, SL4SR is a model-agnostic training framework, and our primary objective is to evaluate the effectiveness of selective learning when applied to the same backbone under identical experimental conditions. To ensure fair comparisons, we use consistent model architectures, datasets, and evaluation protocols for each backbone and its SL-enhanced variant. Therefore, the performance improvements in Table 2 reflect the effect of the proposed selective learning strategy under controlled settings. The implementation details and experimental configurations are also available in our publicly released code for reproducibility.
x
Yes. The SASRec estimator used for SASRec+SL is initialized from exactly the same trained checkpoint as the SASRec baseline reported in Table 2. The estimator remains frozen throughout selective learning, while the SL-enhanced backbone is trained separately using estimator-guided sample selection.
x
Both the estimator and backbone are trained using the cross-entropy loss for next-item prediction. We do not employ negative sampling and evaluate recommendation performance using full-item ranking over the entire item set.
x
Appendix B.5 (Figure 10) reports the per-epoch training runtime of the estimator and selective backbone. We further report the estimated total training time, including reference model pre-training and backbone optimization, alongside recommendation performance in Table C below. Both methods require relatively short training times with early stopping, making their computational costs practically manageable. Although SL4SR requires more training time in some settings, it consistently achieves higher HR@10 and NDCG@10 than RHO-LOSS and even requires less time on LastFM, demonstrating a favorable trade-off between training cost and recommendation accuracy.
Table C: Total Training Cost and Recommendation Performance. Ref./Back. denotes the number of training epochs for reference model pre-training and backbone optimization, respectively.