acceptodds
Under review as a conference paper at ICLR 2027

Compositional Inductive Recommenders

Abstract

Modern recommender systems must serve ever-growing catalogs, including items unseen during training. This challenges models based on ID embeddings, which work well for observed items but do not generalize to new, “cold” items. We advocate for modeling item cold start as a generalization problem and formalize the induction loss allowing recommenders to generalize on cold items. We show that, around a local optimum of the standard recommendation loss, the induction loss is controlled by the reconstruction error between each item’s embedding and its reconstruction from its content-similars. Motivated by this analysis, we prove that the Softmax-Hull Embeddings (SHE), which represent each item as a content-weighted combination of neighboring item embeddings, provide the non-parametric construction that is optimal for generalization on cold items. We further show that the generalization capabilities of SHE are related to the alignment between the content representations and the item embeddings. We give empirical support to our work through recommendation experiments, demonstrating that SHE improves the overall accuracy of several ID-embedding models, simultaneously yielding the expected strong gains on cold items, and outperforming existing approaches for cold-start recommendation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Recommender systems, Cold-start recommendation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_whiaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies item cold-start recommendation through a leave-one-ID-out induction objective. The authors propose Softmax-Hull Embeddings (SHE), which construct an item representation as a content-similarity-weighted combination of neighboring item embeddings. The theoretical analysis bounds the empirical induction objective near a warm-model optimum using reconstruction error and a Lipschitz assumption relating content and behavioral representations. For efficient training, the method evaluates the recommendation loss on a single mixed embedding table that interpolates between each item’s own embedding and its neighborhood reconstruction. Experiments with MF and SASRec on Amazon and Onion report improved warm-target performance and competitive cold-target results.

Critical strengths

S1. The embedding construction is simple, interpretable, and readily integrated into existing recommenders. S2. The leave-one-ID-out formulation provides a useful framework for examining representation substitution.

Critical weaknesses

W1. The theoretical claims exceed what the analysis establishes. Proposition 2 proves that SHE minimizes a kernel-weighted least-squares objective over neighboring embeddings, but this does not establish optimality for leave-one-ID-out reconstruction or cold-item generalization. Moreover, Proposition 5 justifies the single-pass surrogate only locally around a warm optimum, whereas the actual training reinitializes the backbone and jointly optimizes its parameters. If the mixing matrix is invertible and the embeddings are unconstrained, the surrogate can even be viewed as a reparameterization of the original recommendation objective. The authors should clarify the scope of the optimality claim and empirically validate the surrogate against the original induction objective.

W2. The experimental evidence does not sufficiently establish the incremental contribution. SHE is closely related to kernel regression, proxy-based item representations (Seol et al., WSDM 2024), and training-time neighbor substitution (SwapRec, 2026). Comparisons with these approaches and a pretrained backbone using inference-only neighbor interpolation would better isolate the proposed training mechanism. Furthermore, Section 3.1 states that cold items use q=0q=0, whereas Section 4.3 attributes declining cold-target performance at larger qq to their untrained self-embeddings. This inconsistency requires clarification. Finally, differences in test-history handling between SHE and ID-based baselines may partly explain the warm-target gains, motivating a controlled comparison with identical input histories.

Questions for authors

See weakness.

1 comment

Reviewer whia

Thank you for the comments on the paper. The revised paper provides pointers in purple, with a topic name and the weakness, e.g., "(scope: whia W1)".

Summary of results. In the revised paper, SASRec + SHE is the most accurate method over the full test set on both datasets (Table 2, "All"), and with the new content fine-tuning (Remark 1) it is also the best cold-target method on Amazon.

W1, Prop. 2. (tag: "scope") We narrowed the claims (abstract; text after Prop. 2 in Sec. 2.2): Prop. 2 characterizes SHE as the kernel-regression estimate for a given neighborhood and kernel. We no longer claim optimality for leave-one-out reconstruction or for cold-item generalization, and we assess the latter empirically.

W1, Prop. 5. (tag: "scope") We agree that Prop. 5 (Sec. 2.4) holds around a local minimum. Outside this neighborhood the evidence is empirical: with both EE and ω\omega randomly initialized (Sec. 4.2, "Initialization of E"; Table 3), SASRec + SHE still outperforms its backbones.

W1, reparameterization. (tag: "reparameterization") We commented on this in Remark 2 (Sec. 2.4; App. A). The surrogate is the warm loss reparameterized as F=WqEF = W_q E, but cold items are represented through EE, not FF. Their embeddings differ from post-hoc interpolation over a warm optimum by a term linear in the hull residual (I−Wˉ)F∗(I - \bar{W})F^*, which vanishes only if Q=0Q = 0.

While the two parameterizations are equivalent on warm items (and hence for the training loss), they are not equivalent on cold items, as shown by the new inference-only ablation (Sec. 4.2; Table 4).

HR@10Amazon warmAmazon coldOnion warmOnion cold
SASRec, inference-only.0487.0292.0491.0058
SASRec + SHE (trained).0779.0401.3448.0980
MF, inference-only.0401.0217.1215.0083
MF + SHE (trained).0417.0361.1249.0441

As requested, we also validated the single-pass objective against the leave-one-out condition of Eq. 1 on the trained models (new App. C.10; Table 11). When an item is withheld and rebuilt from its content neighbors, the pre-trained SASRec cannot recommend it (HR@10 .0010 on Amazon), whereas SASRec + SHE recommends it almost as accurately as from its own embedding (.0717 vs. .0745). The additional training loss caused by rebuilding an item is more than ten times lower than for the backbone (.0067 vs. .0900).

W2, baselines. (tags: "inference-only", "swap baselines") The table above also addresses W2 on inference-only interpolation. We further added a comparison with SwapRec, its inference-only counterpart (nearest-neighbor swap), and the proxy-based item representations of Seol et al. (2024), all using SASRec as backbone (new App. C.11; Table 12). Over all test sequences, SASRec + SHE is more accurate than all three on both datasets (on Onion, within the confidence intervals of the two swap methods).

W2, q=0q = 0. (tag: "q=0q = 0 at inference") Thank you for spotting this; the statement was wrong. Cold items always use q=0q = 0 while the qq in Fig. 1 is the training self-share. We hence attribute the accuracy decline to a train–inference mismatch in qq-values, and reframed the discussion accordingly.

W2, histories. (tag: "identical histories") Previous work (SwapRec, 2026) showed that sequential models underperform when cold-item inputs are represented by the randomly initialized embeddings as compared to "dropping" them (i.e., the setup in which we evaluate them). Nevertheless, we added in App. C.9 (Table 10) an evaluation on the test sequences whose history contains no cold item, under which all models receive identical inputs. On Amazon (19,370 such sequences), SHE improves over its backbone by 46% (SASRec) and 136% (MF) in HR@10 over all targets. On Onion, only 203 sequences do not contain a cold item as input, and although our approach does not reach the best accuracy, on this small sample the differences are not statistically significant.

Reviewer_iP6LOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work aims to handle cold start item issue in recommender systems. The authors formulated item cold start as a generalization problem and proposed an induction loss to address it. This work proposes a non-parametric constructor (SHE) for behavioral embeddings of cold items. Experiments conducted on two datasets are used to examine the performance of the proposed SHE.

Critical strengths
  1. The research topic of addressing the cold-start item problem in recommender systems is relevant and has received considerable attention from the research community.
  2. The experiments are relatively extensive, examining the proposed method from various aspects.
  3. Overall, this work is well-written.
Critical weaknesses
  1. The novelty of this work appears limited and remains unclear. The cold-start issue in recommender systems is popular and has received wide attention in the community. Moreover, employing warm item embedding or content embeddings to construct embeddings of cold items is a common approach in this area. This work appears to offer limited novel insights into the problem, and its technical contributions also seem limited. The authors should clarify the fundamental differences between their approach and existing methods and highlight its unique contributions.
  2. The motivation should be enhanced. The core arguments lack strong supports. For example, the gap between item behavioral representations and content representations is mentioned as a key motivation, but the authors do not provide sufficient evidence to substantiate this claim. In addition, the rationale for introducing the induction loss to address the cold-start item problem is unclear. The authors should provide stronger empirical or theoretical evidence to support their main arguments and justify the proposed design choices.
  3. The Introduction contains excessive background information, while the unique insights of this work receive insufficient attention. In particular, Lines 36–78 devote substantial space to describing existing methods. Such a detailed discussion would be more appropriate for the Related Work section. The authors should streamline the Introduction to make room for a clearer presentation of the research gap and the key contributions of this work.
  4. The rationales behind some core technical designs and assumptions are insufficiently explained. For example, the formulation of the inductive recommender in Equation 1 lacks adequate justification and is somewhat difficult to follow. The authors should clarify the underlying assumptions, explain the motivation for this formulation, and provide sufficient explanations to facilitate understanding.
  5. Some concerns about the experiment results. The main goal of this work is to handle the cold-strat issue in recommendation. That is, improving the recommendation performance on cold-start items. However, Table 3 shows that the proposed SHE method does not achieve better performance in the cold-target setting. This result raises concerns about the effectiveness of SHE in addressing the very problem it is designed to solve. The authors should investigate this issue and provide further analysis to explain why the proposed method fails to demonstrate improvements in this setting.
  6. The Introduction should be carefully revised to clearly articulate the limitations of existing methods, the key insights motivating this work, and the specific research gap the proposed method aims to address. The authors should also explain how their approach differs from existing solutions and why these differences are important.
Questions for authors

Please refer to the Weakness.

1 comment

Reviewer iP6L

Thank you for the detailed comments. Numbers refer to the revised PDF, which contains tags for the weaknesses, e.g., "(alignment gap: iP6L W2)".

Summary of results. In the revised paper, SASRec + SHE is the most accurate method over the full test set on both datasets (Table 2, "All"), and with the new content fine-tuning (Remark 1) it is also the best cold-target method on Amazon (details under W5).

W1. Novelty. (tag: "novelty") We agree that composing a cold item from its content neighbors has precedents (App. B). Our contribution is the training objective (induction loss), the way the embeddings are composed (softmax hull), and their joint usage. In the new version, the paper also contributes a content fine-tuning procedure derived from the theoretical principles guiding the induction loss and SHE (Remark 1).

To show that the induction loss and SHE are most effective when combined, we added an ablation (Table 4) that applies SHE to a pre-trained backbone without induction-loss training:

HR@10, SASRecAmazon warmAmazon coldOnion warmOnion cold
Inference-only composition.0487.0292.0491.0058
SHE (trained).0779.0401.3448.0980

This shows that the accuracy gains do not come from the well-established composition heuristic.

W2. Motivation and Alignment. (tag: "alignment gap") Alongside references, the Introduction and motivation now point to our own evidence on the gap between content and behavioral space alignment (Sec. 4.2; App. C.13); furthermore, we show that the proposed content fine-tuning approach reduces this gap (Remark 1). Induction loss (tag: "methodological choices"): Sec. 2.1 now states the rationale behind the induction loss: a cold item must be represented by a constructor at inference, and SHE uses the same constructor during training (train–test matching; Vinyals et al., 2016).

W3, W6. Introduction and excessive related work. (tag: "novelty") We rewrote the introduction, shortening the discussion of existing methods to one paragraph, moving the full discussion to App. B, and giving more emphasis to the novelty of our work. The rest of the introduction hence states the limitation of prior works (they assume a content–behavior connection despite the gap), our insight (cold start as generalization), and contributions C1–C3.

W4. Eq. 1. (tag: "methodological choices") Sec. 2.1 now introduces the constructor before the equation, explains Ei←gE_{i\leftarrow g} and each of the two terms in words, states the role of λ\lambda, and makes assumptions explicit.

W5. Cold-target results. (tag: "accuracy goals") Goal. Our goal is a recommender that is accurate on cold items without harming accuracy on warm items. On both datasets, the baseline that is best on cold targets is below the ID backbone on warm targets (MoRec .0515 vs. .0568 on Amazon; SID two-tower .1497 vs. .3375 on Onion). To make this explicit, Table 2 now reports accuracy over all test sequences ("All" column, defined in App. C.8): SASRec + SHE is the most accurate method on both datasets, with and without fine-tuning. Furthermore, it is either best (Amazon) or statistically on par (Onion) with the best cold-target models, while being substantially better on warm targets.

Why SHE was not ahead on cold targets in the previous paper version. We attribute it to the misalignment between content and behavioral spaces (Props. 1 and 3). The revision therefore adds a fine-tuning of the content encoder to improve the alignment (Remark 1; Sec. 3, "Fine-Tuned Content Space"), reaching the best cold-target accuracy.

Reviewer_tRwHOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SHE, an item cold-start generalization Recommender. An induction loss reconstructs ID embeddings from item content and the embeddings of other items. Softmax-Hull Embedder (SHE) represents an item embedding as a softmax-weighted convex combination of its content-nearest warm items. Experiments are conducted on Amazon and Onion (audio content), with SASRec and MF backbones, evaluating warm-target and cold-target metrics.

Critical strengths

SHE has a principled interpretation. The softmax-hull constructor is the exact minimizer of a kernel-weighted least-squares reconstruction objective. This is principled than a linear mapper.

Critical weaknesses

1.The method does not beat SOTA cold-target baselines, which undermines the central cold-start claim. The paper’s main claim is that SHE effectively addresses item cold start. However, on cold targets, SASRec+SHE is not the best method on either Amazon or Onion dataset. The paper’s claim of offering the best balance between warm and cold accuracy is not supported by a defined combined metric; it is a post-hoc interpretation. For a cold-start paper, failing to outperform strong cold-start baselines on cold targets is a major weakness.

2.The method’s effectiveness is conditional on content–behavior alignment; when alignment is weak, the method does not solve cold start. The paper’s own analysis shows that SHE’s gains depend strongly on content–behavior alignment. But that is precisely the limitation: the method works when the content space already reflects behavioral similarity. In real cold-start scenarios, content and behavior are often weakly aligned (e.g., new domains, noisy metadata, multimodal content). The paper does not propose a mechanism to improve alignment or to remain effective under weak alignment. A cold-start method that only works when the cold-start problem is already easy is of limited value.

Questions for authors

1.SASRec+SHE is below MoRec on Amazon and below SID two-tower (RQ-Kmeans) on Onion. Why should the community adopt a cold-start method that is not the best cold-start method on either dataset? Can the authors define a combined warm/cold metric under which SHE is demonstrably best?

1 comment

Reviewer tRwH

Thank you for the suggestions provided in the review. The main points of criticism were addressed in the new paper version:

  • an evaluation on the full test set, combining warm and cold accuracy
  • a content fine-tuning mechanism to improve alignment.

Reply points in the paper are tagged in purple with a topic name and the weakness, e.g., "(alignment gap: tRwH W2)".

Cold-start accuracy. In the revised paper, SHE no longer loses on cold targets. On Amazon, with the new content fine-tuning (Remark 1), SASRec + SHE is the best cold-target method (HR@10 .0401 vs. .0370 for MoRec; paired test on nDCG@10, p=0.013p = 0.013). On Onion, its gap to the SID two-tower is not statistically significant. Neither of the baselines mentioned in the review (MoRec, SID two-tower) outperforms SHE on both cold and warm targets: MoRec has a HR of zero on Onion cold targets, and the SID two-tower reaches about a third of SHE's cold-target accuracy on Amazon (.0141 vs. .0401) and less than half of its warm-target accuracy on both datasets. SHE is the most accurate method on the full test set, while remaining competitive on cold targets.

W2. Dependence on alignment. (tag: "alignment gap") We are grateful to the reviewer for the comment, which guided further development of a method to improve on the alignment between content and behavioral spaces (Remark 1 in Sec. 2.3; setup in Sec. 3, "Fine-Tuned Content Space").

We fine-tune the content encoder to minimize the embedding reconstruction error that bounds the induction loss. Results are in Sec. 4.1 and in Sec. 4.2, "Alignment" (Fig. 2). In summary:

AmazonAlignmentWarm HR@10Cold HR@10
SASRec + SHE, raw → fine-tuned0.49 → 0.53.0745 → .0779.0335 → .0401 (+20%)
MF + SHE, raw → fine-tuned0.28 → 0.31.0379 → .0417.0280 → .0361 (+29%)

Content fine-tuning improves the alignment between the two spaces, and correspondingly leads to a marginal improvement on warm targets, and a substantial improvement on cold targets.

W1 / Q1. Cold- and warm-target combined evaluation. (tag: "accuracy goals") Table 2 ("All" column, HR@10) and Table 9 (nDCG@10, HR@100) report accuracy over all test sequences:

HR@10Amazon AllWarmColdOnion AllWarmCold
SASRec.0455.0568–.2711.3375–
MoRec.0486.0515.0370.0012.0015.0000
SID two-tower, RQ-Kmeans.0252.0280.0141.1425.1497.1132
SASRec + SHE.0663.0745.0335.2962.3448.0980
SASRec + SHE, fine-tuned content.0704.0779.0401–––

SASRec + SHE is the most accurate method over all test sequences on both datasets, with and without content fine-tuning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.