- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper frames recommendation as the task of "next semantic direction selection", where given a user action sequence, the goal is to identify a continuous location within the content space such that the previously unobserved items in its admissible neighborhood contain high-value relaizations. The author propose NORTH, a framework that learns cross-content behavior representation, employs a history-conditioned scorer for personalization of the cluster-based oppurtunities, and utilizes a policy that navigates via "stay or move" decisions and multimodal displacements. Directions are evaluated using the Best Realized Value (BRV), where the optimal response observed within a fixed "relaization budget". Experiments on KuaiRec-Full and KuaiRec-FO datasets demonstrates that the NORHT variant outperforms baselines that rely on fixed candidate pools, and that the full NORTH framework surpasses independent open-ended generation models such as DreamRec or PreferDiff. Ablation studies validate the effecrtiveness of history-conditioned proposals against various displacement strategies.
Critical strengths
- A principled reformulation of recommendation paradigm. This paper shifts the objective of recommendation from next item prediction to semantic direction prediction, formalizing this as the selection of the continous direction anchored to a fixed-size subset of real items. The reformulation is significant and establishes an intermediate objective that can be utilized by item retrieval or item generation paradigms.
- Clear, principled derivation accompanied by a corresponding evaluation protocol. NORTH separates the learning of collective behavior, the scoring personalized directions and training and evaluation of history conditioned search. The BRV objective correctly targets the optimal outcome at the core level, which is critical: by using a fixed candidate pool alongside an independent open-ended generation or retrieval components, while employing a shared, frozen scorer, the approach decouples scoring accuracy from quality of the direction .
Critical weaknesses
- A gap in presentation and reproducibility. Several key quantities in the methodology are left undefined or under specified. For example, the local semantic scale c_i in equation 14, the support region R used for grouding the direction, and the reference cited after "The representation used for short-history adaptation". The canddiate generation process where the predefined core is not fully discussed and motivated. This is significant because thse are essential components for reproducing NORTH and adapting it toward broader recommendation tasks. The core mechanism of the prosed method cannot be verified or replicated based on the main body of text.
- The evaluation is concentrated in terms of both scenario and metrics. All evidence is derived from single source, KuaiRec, and reported exclusively using the new proposed BRV metric, with no comparisons against established metrics based on recall or ranking. This is critical since BRV is a custom, core-level maximum value metrc, which makes it difficult to determine whether the observed performance gain reflect geneuine improves in recommendation quality.
Questions for authors
How is the support region R defined and sized, and how sensitive are the result to its choice?
Could the author presents experimental evidence that the direction search outperforms item based recommendation?
2 comments
Thank you also for asking how the support-region choice affects the results and whether the direction benefits actual item recommendation. We added experiments for both questions.
4. Support-radius sensitivity
Appendix G.6 and Table 14 evaluate radius multipliers 0.8, 1.0, and 1.2 across five additional search seeds. Expanding the default support from 1.0 to 1.2 preserves the original bank and anchors and produces the following paired changes:
| Setting | BRV change, 1.2 − 1.0 [95% CI] |
|---|---|
| Full, | +0.0011 [0, 0.0029] |
| Full, | +0.0017 [−0.0013, 0.0060] |
| FO, | +0.0003 [0, 0.0010] |
| FO, | −0.0001 [−0.0003, 0] |
The expansion adds 5,574 legal proposal occurrences but changes only 26 of 5,640 final selections (0.46%). Thus the default results are stable to this support expansion even when it creates additional geometric opportunities.
The tighter 0.8 setting retains 1,187 of the 6,434 original bank representatives and therefore also changes the available starting points. The paired changes on Full are −0.0174 and −0.0358 at and . Reporting both tests makes the distinction explicit: expanding an already supported neighborhood leaves the decisions largely unchanged, whereas substantially restricting support removes useful starting opportunities.
5. Direct evidence with item-based recommenders
We added a single-item experiment using BPR, SASRec, and LightGCN as frozen downstream rankers. Each ranker selects one item either directly from the catalogue or from NORTH's four-item core. We then measure the observed response of that chosen item. This evaluates an item actually chosen by a ranker, not an oracle maximum over four observed responses.
Table 15 reports the paired gains on common measurable users:
| Setting | BPR | SASRec | LightGCN |
|---|---|---|---|
| Full, | +0.0995 | +0.0644 | +0.0675 |
| Full, | +0.1621 | +0.0650 | +0.0373 |
| FO, | +0.0404 | +0.0775 | +0.0192 |
| FO, | +0.0444 | +0.0186 | +0.0345 |
All twelve paired differences are positive. This directly supports the proposed intermediate role: selecting the direction first supplies useful opportunities to several established item-ranking architectures. The experiment is summarized in Section 5.4 and detailed in Appendix G.7.
Regarding your request for established recommendation metrics, the added evaluation targets the underlying question directly through observed selected-item response, alongside BRV. Recall would require a specified relevant-item set; our watch-based feedback is graded, and the selected-item outcome retains that information.
6. Why the dataset is particularly suited to this task
KuaiRec combines near-complete target feedback with population logs and content information. Its 99.6%-dense small matrix (Gao et al., 2022) makes it possible to measure the complete four-item realization set using observed outcomes. Sparse logs generally leave some selected members unobserved, requiring an additional missing-outcome assumption or a change to the evaluated set. KuaiRec lets us preserve the actual selected core and the fixed budget.
The two settings also test different population-learning conditions: Full trains on 5,765 users outside the FO subset, whereas FO uses 903 training users from that subset. Both evaluate on held-out FO users, and the Full-trained model transfers without further population training (Section 5.1). Coverage and common-user comparisons are now reported explicitly in Appendices G.2–G.4.
We sincerely appreciate your suggestions. They have helped us make both the implementation and the connection to realized item recommendation much clearer, and we would be grateful for your reassessment of the revised evidence.
References
Gao, C., Li, S., Lei, W., Chen, J., Li, B., Jiang, P., He, X., Mao, J., and Chua, T.-S. (2022). KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems. Proceedings of the 31st ACM International Conference on Information & Knowledge Management (CIKM 2022).
Thank you for your careful reading and for recognizing both the upstream direction-selection problem and the separation between behavioral learning, personalized scoring, and search. Your questions identified implementation details that needed a precise account. We have added the local-scale explanation immediately after Eq. (14) and collected the geometric construction in Appendix G.1.
1. The local scale and support region are now specified
In Eq. (14), cj is the anchor coordinate and ℓ(cj) is its local semantic scale. All geometric operations use frozen 16-dimensional PCA content coordinates. For item i, let dk+(i) be the distance to its k-th nearest distinct content location, excluding coincident locations and distances within the numerical tolerance. We use
s_i = \max\left\\{\varepsilon, \min\left[d_8^+(i), 3d_3^+(i)\right]\right\\}
\ell(c) = \max\left\\{\varepsilon, \operatorname{median}_{i\in N_3(c)}s_i\right\\}
where N3(c) contains the three nearest catalogue items. Consequently, the same normalized displacement produces a shorter move in a dense region and a longer move in a sparse region; the 3d3+(i) term limits the influence of a distant eighth neighbor.
The support is
ρmax=Quantile0.95(si)
R(c) = \left\\{i : \lVert x_i-c \rVert_2 \leq \min\left(\ell(c), \rho_{\max}\right)\right\\}
Here ρmax=0.85487997. Grounding returns the four nearest distinct items in this support, resolving equal distances by ascending original item ID. Proposals with insufficient support are rejected; grounded cores must also be external to the target history and satisfy the training-donor support requirement. The formulas and operation order are in Eqs. (42)--(45).
2. The bank construction and its role are explicit
Each item contributes its own coordinate, two midpoints with its nearest two other items, and the mean coordinate of itself and its nearest three other items. These centers cover both item locations and intervening local positions while keeping every realization budget at four items.
The construction produces 12,852 center proposals. Deduplication by the generating unordered item pairs and quadruples gives 11,114 candidate records; common grounding retains 10,495 legal records; deduplication by the resulting four-item identities gives 6,434 bank cores. Search uses this bank to obtain the personalized incumbent and up to eight population-supported anchors, then evaluates 72 continuous proposal attempts. This makes the bank both a reproducible starting set and an identity-based reference for testing discovery beyond it (Appendix G.1).
3. The missing reference has been repaired
Section 4.1 now cites Bertinetto et al., "Meta-learning with differentiable closed-form solvers" for differentiation through a closed-form ridge solver to learn a representation that supports rapid adaptation. The cited arXiv version dates to 2018; the work was subsequently published at ICLR 2019. This reference supports the solver mechanism used in Eq. (4), while the population-centered core objective and direction-search formulation are specified in our own method.
Thank you for pointing out these presentation gaps. The additions now connect the main-paper equations to the concrete geometry and bank-building procedure.