- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes ClueRec, an LLM-based recommendation framework that learns user preferences by comparing clicked items with co-exposed alternatives. The authors argue that existing LLM-based recommendation methods mainly focus on positive interactions while overlooking useful preference information from unclicked alternatives.
The authors evaluate ClueRec using industrial data from five million users for generative recommendation and CTR prediction. The model outperforms several baselines on offline prediction metrics. A one-week online A/B test also reports a 0.571% improvement in CTR and 1.024% improvement in revenue per mille (RPM).
The main claimed contribution is using hierarchical semantic differences between clicked and co-exposed alternatives to improve preference modeling in LLM-based recommendation.
Critical strengths
The paper addresses an important recommendation problem, and the use of hierarchical semantic representations to incorporate co-exposed alternatives is potentially useful. The framework supports both generative recommendation and CTR prediction.
I appreciate that the paper evaluates the model on large-scale industrial data and includes an online A/B test. The reported improvements across multiple metrics are encouraging. The ablation studies also provide some evidence about the contribution of different components.
Critical weaknesses
- Limited novelty. The main idea has been extensively studied in the economics and marketing literature for more than a decade.
My biggest concern is the limited contribution relative to existing research. The paper motivates its method by arguing that recommendation models focus on what users click but overlook the alternatives they choose not to click, and therefore fail to understand why users prefer one item over another. However, this is not a new insight.
The economics literature on consumer search and discrete choice has long studied how consumers' choices among available alternatives reveal their preferences. For example, De los Santos, Hortaçsu, and Wildenbeest (AER 2012) use browsing and purchasing behavior to model consumer search and preferences. Ursu (Marketing Science 2018) and Ghose, Ipeirotis, Li (Management Science 2019) examine consumer search and purchase decisions while accounting for endogenous product rankings and interactions with social media. Over the past decades, several related research has explicitly distinguished exposure, consideration, and choice. These studies have already recognized that understanding preferences requires examining not only selected products but also the alternatives consumers encounter, while accounting for the search and exposure process.
The authors do reference some prior exposure-aware recommendation methods, including ExpoMF and RACP, but they do not sufficiently engage with this established economics literature. More importantly, I do not see clearly what new conceptual understanding ClueRec adds beyond applying this established insight to LLMs.
The technical novelty mainly involves representing item differences through hierarchical SIDs and injecting them into an LLM using gating and attention. These are reasonable engineering choices, but I find the methodological advance incremental.
I am not convinced that these relatively small improvements establish a substantial new contribution, especially given the extensive prior literature on exposure-aware preference modeling.
- Insufficient identification of user preferences and limited validation of the proposed mechanism.
My second concern is that the paper assumes a clicked item is preferred over all co-exposed alternatives. However, non-clicking does not necessarily indicate lower preference. Users may not have noticed or considered all alternatives, and their decisions are influenced by ranking positions, presentation, and the platform's exposure policy. The exposure set itself is also endogenous, since the platform determines which products users see based on existing recommendation algorithms.
These issues have been extensively recognized in economic search models, which distinguish consideration, search, and choice. Yet ClueRec directly treats differences between clicked and unclicked items as preference evidence without adequately addressing this distinction. As a result, the learned representations may capture the platform's historical exposure and ranking patterns rather than users' underlying preferences.
The empirical evidence mainly demonstrates improved prediction of observed clicks, not that the model learns why consumers prefer certain products. I think these are quite different claims.
Questions for authors
- Novelty relative to economic search models.
How is the central preference-learning idea different from the established economics literature that uses observed search, chosen alternatives, and non-chosen alternatives to estimate consumer preferences? In particular, what new understanding does ClueRec provide beyond implementing this existing idea within an LLM architecture?
- Identifying preference versus exposure bias.
How can the authors establish that the clicked–exposed differences represent actual user preferences, rather than the platform's ranking and exposure policy? Would the results remain under randomized exposure, position-controlled comparisons, or a change in recommendation policy?
- Incremental value of the architecture.
Can the authors compare ClueRec with simpler pairwise preference-learning or exposure-aware methods using the same LLM backbone and computational budget? Given the relatively small improvement over the w/o SPE ablation, what is the specific advantage of the hierarchical SID difference mechanism?
- Evidence of preference learning beyond prediction.
Is there any independent validation that the learned representations capture stable user preferences, rather than simply improving predictions under the existing recommendation policy? For example, can they predict choices under different consideration sets or exposure policies?