acceptodds
Under review as a conference paper at ICLR 2027

Extrapolating User Preferences via Compositional Atomic State Transitions

Abstract

Sequential recommendation models typically encode user dynamics as continuous latent vectors. However, these monolithic representations conflate distinct preference dimensions, limiting their ability to generalize compositionally to novel configurations of known elements. To address this, we propose ompositional tomic tate ransitions, a framework that reconceptualizes preference evolution as state transitions within a discrete, compositional semantic space. Rather than allowing unconstrained latent shifts, CAST decomposes user profiles into facet-specific semantic atoms and models temporal dynamics via typed atomic operations. This design constrains future extrapolation to recombinations of known atoms, yielding interpretable semantic trajectories and soft transition signals for downstream candidate ranking. To evaluate this capability, we introduce a temporal extrapolation protocol with strict chronological and combination holdouts, isolating compositional generalization from mere entity recognition. Experiments demonstrate that CAST recovers a non-trivial subset of unseen preference configurations and improves candidate ranking across multiple sequential backbones. Ultimately, CAST provides an interpretable inductive bias for modeling compositional user dynamics.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
User Representation Learning, Sequence Recommendation, Dynamic Preferences, State Transitions, Compositional Modeling, Multi-Interest Representations

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qbeHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes CAST, a framework for representing future user-preference changes as compositional transitions over interpretable semantic atoms. It predicts ADD, STRENGTHEN, or NONE operations over reusable preference atoms and evaluates generalization to unseen combinations of known atoms. Validation covers semantic-state recovery, downstream recommendation, strict compositional generalization, case studies, and lightweight integration with multiple sequential recommenders.

Critical strengths
  1. The paper presents a coherent story for modeling future preference changes through compositional and human-readable state transitions.
  2. The experiments address the paper’s three main research questions and include strict compositional tests, downstream ranking, case studies, significance tests, and lightweight integration across several backbones.
Critical weaknesses
  1. The paper does not adequately justify why Theme, Style, and Setting are the appropriate decomposition, whether they are sufficiently distinct or complete, or how sensitive the results are to alternative facets. Because this ontology defines the prediction task, the omission weakens the methodological foundation.
  2. The experiments rely primarily on OpenOneRec-RecIF dataset and an LLM-assisted annotation pipeline. Although multiple backbones are tested, evaluation on more datasets, independently annotated labels, or a standard benchmark datasets would make the generalizability substantially more convincing.
Questions for authors
  1. How were Theme, Style, and Setting selected? Were alternative facet decompositions considered and evaluated, and how sensitive is CAST to the number and definition of facets?
  2. How much of the improvement comes from the proposed transition modeling, rather than simply providing the recommender with an LLM-generated future user profile?
  3. What fraction of the AI-generated annotations was rejected or corrected during human filtering, and what were the annotation protocol and inter-annotator agreement?
  4. What were the main considerations for the authors to evaluate on OpenOneRec-RecIF? Have the authors considered other datasets?
Reviewer_nWKyOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper identifies a limitation of existing sequential recommendation models: they typically represent users' dynamic preferences using continuous latent vectors. Such representations may conflate distinct aspects of user preferences and limit the models' ability to generalize compositionally to novel combinations of previously observed elements.

To address this limitation, the paper re-conceptualizes preference evolution as state transitions within a discrete, compositional semantic space. Specifically, the proposed model, CAST, represents user preferences using a set of semantic atoms, with novel preference directions constructed by recombining underlying basis elements identified from historical interactions.

Critical strengths
  1. The paper is well motivated and addresses an important problem. Given that user preferences evolve dynamically, developing a mechanism to construct future preference predictions from known basis elements could capture evolving user interests while also providing interpretability into the underlying factors driving user preferences.
  2. The paper is well written and easy to follow. Each component is described in detail and supported by illustrative figures, which greatly facilitate understanding of the key ideas and the overall methodology.
  3. The proposed model, CAST, achieves clearly higher performance than the baselines on large-scale industrial datasets, demonstrating its empirical effectiveness.
Critical weaknesses
  1. Learning multiple facets of user preferences for recommendation has been extensively studied in the literature on disentangled and multi-interest recommendation [a, b, c]. Although CAST introduces the idea of using LLMs to extract atomic preference bases and compose them for future preference modeling, its methodological novelty relative to these existing approaches remains unclear. In particular, the paper should more clearly articulate how its compositional semantic space and preference evolution mechanism fundamentally differ from existing methods that disentangle user preferences into multiple factors or interests. Without a clearer distinction, the proposed approach appears to be an incremental extension of existing multi-interest recommendation methods through the incorporation of LLM-based preference atom extraction and composition.

[a] Controllable Multi-Interest Framework for Recommendation. KDD 2020.

[b] Learning Multi-Faceted Prototypical User Interests. ICLR 2024.

[c] Disentangled Multi-Interest Representation Learning for Sequential Recommendation. KDD 2024.

  1. The choice of baselines is not sufficiently convincing. CAST explicitly represents user preferences through multiple facets, which may provide an inherent advantage over conventional sequential recommendation models such as BERT4Rec, SASRec, and Mamba4Rec that do not necessarily employ comparable multi-interest representations. Therefore, a more appropriate evaluation should include disentangled and multi-interest recommendation models [a, b, c, d, e, f] that represent users through multiple aspects of their preferences. Such comparisons are necessary to determine whether CAST offers meaningful improvements over existing multi-interest modeling approaches, rather than merely outperforming models with different representational assumptions. Without these comparisons, it is difficult to establish the relative effectiveness and advantages of the proposed method.

[d] Re4: Learning to Re-Contrast, Re-Attend, Re-Construct for Multi-Interest Recommendation. TheWebConf (WWW) 2022.

[e] User-Aware Multi-Interest Learning for Candidate Matching in Recommenders. SIGIR 2022.

[f] LLM-Driven Dual-Level Multi-Interest Modeling for Recommendation. https://arxiv.org/pdf/2507.10917.

  1. The paper provides insufficient analysis of the scalability and training costs of the proposed method. In particular, as the number of preference facets and semantic atoms increases, the paper does not systematically analyze how these factors affect model performance, computational overhead, or scalability. Such an analysis is important for understanding the practical trade-offs involved in the proposed design and assessing whether its benefits justify the additional complexity. Moreover, the method relies on a large model, Qwen3-30B-Instruct, to extract preference atoms. Consequently, the observed performance gains may be attributable, at least in part, to the capabilities of the underlying LLM rather than to the proposed compositional modeling mechanism itself. Further diagnostic experiments are therefore needed to disentangle the contribution of the LLM from that of CAST's core methodology and to provide a clearer understanding of the model's behavior.
Questions for authors

Please see the review.

Reviewer_UYW2OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Compositional Atomic State Transitions (CAST), a framework designed to overcome the limitations of monolithic continuous representations in sequential recommendation by modeling preference dynamics as transitions within a discrete, compositional semantic space. Rather than predicting unconstrained latent trajectory shifts, CAST factorizes user histories into typed semantic atoms across distinct facets (Theme, Style, and Setting) using an LLM and prototype codebooks, subsequently forecasting future preferences through typed operations such as adding or strengthening active atoms. The authors' primary contributions consist of formalizing user preference evolution from a compositional generalization perspective, proposing the CAST architecture that maps predicted soft transition deltas into candidate-aware residual ranking scores for frozen recommendation backbones, and establishing a temporal extrapolation protocol with strict seen-parts/unseen-whole holdouts to isolate compositional reasoning from simple memorization. Empirical validation on the OpenOneRec-RecIF benchmark demonstrates that CAST markedly outperforms holistic textual profiles in future-state recovery metrics, successfully reconstructs unseen joint state combinations with an average exact match rate of 17.84% against a 5.86% chance baseline, and yields consistent NDCG@10 improvements as a plug-and-play module across diverse sequential architectures (BERT4Rec, SASRec, FMLP-Rec, Mamba4Rec, and HSTU) under both standard and out-of-distribution evaluation protocols.

Critical strengths
  • The paper addresses a fundamental limitation in sequential recommendation by reframing user preference dynamics as discrete, typed state transitions over factorized semantic atoms rather than unconstrained continuous latent vectors. By decomposing profiles into explicit facets (Theme, Style, and Setting) and modeling discrete operations such as adding or strengthening preferences, the framework provides an interpretable and auditable view of evolving user intents.
  • The experimental evaluation is thoughtfully constructed to isolate genuine compositional generalization from simple sequence memorization. Enforcing a strict seen-parts/unseen-whole holdout alongside informative baseline controls, such as randomly shuffled facets, cleanly verifies that the recovered unseen state configurations stem from structured semantic reasoning rather than random combinatorial chance.
  • The proposed framework is modular and functions effectively as a plug-and-play residual adapter across diverse recommendation backbones, including Transformer, selective state-space, frequency-domain MLP, and modern transducer architectures. It achieves consistent performance gains across both standard and out-of-distribution settings while maintaining operational efficiency, introducing only roughly 408K trainable parameters during the re-ranking stage with minimal GPU training overhead.
Critical weaknesses
  1. The proposed framework essentially adapts the concept of Semantic IDs, a technique widely explored for item tokenization in generative recommendation, to the user side, yet does so through a rigid and heuristic formulation. Rather than learning an adaptive, hierarchical discrete representation data-drivingly (such as through learned vector quantization or residual codebooks), CAST enforces a handcrafted three-facet ontology (Theme, Style, and Setting) and partitions each facet into an arbitrary, flat codebook of 256 centroids using MiniBatchKMeans. This ad-hoc discretization inherits well-known quantization drawbacks: flat geometric clustering on frozen text embeddings risks boundary distortion, codebook under-utilization, and semantic collision, all while lacking an end-to-end reconstruction objective to guarantee that these user tokens preserve rich contextual nuances.
  2. CAST decouples preference modeling into several independent steps: LLM atom extraction, offline prototype clustering, MLP transition forecasting, and residual ranking adapter training. Because upstream modules are kept frozen during downstream optimization, hallucinations or semantic omissions during initial LLM extraction directly distort the discrete ground-truth transition actions (such as ADD and STRENGTHEN). These cascading errors cannot be backpropagated or corrected by the recommendation objective.
  3. CAST extracts preference atoms by prompting a frozen LLM to autoregressively decode textual summaries of past user interactions. As demonstrated by related work such as SPECTRA [1], direct autoregressive sequence generation is vulnerable to severe head-frequency bias and search truncation, which provably suppresses minority and long-tail user intents. Whereas SPECTRA reads full, calibrated preference distributions directly from the model softmax to preserve niche interests, CAST funnels historical interactions through a sparse generative bottleneck, discarding subtle, low-frequency preference signals before they ever reach the prototype codebook.

[1] Luyang Zhang et al. 2025. SPECTRA: Revealing the Full Spectrum of LLM-Internal Preference Distributions. arXiv preprint arXiv:2509.24189.

Questions for authors
  1. In Stage 1, a frozen LLM autoregressively extracts only two to three textual atoms per facet. Have you ever examined the long-tail effect of user intents?
  2. How robust is the framework to alternative facet definitions or to domains where Theme, Style, and Setting are less natural (such as e-commerce or news recommendation)? Additionally, what guided the choice of flat MiniBatchKMeans with a fixed 256 codebook size over learned vector quantization schemes (such as residual or hierarchical VQ) that could adaptively mitigate cluster collision and boundary distortion?
  3. Because the LLM atom extractor and the transition predictor are frozen prior to optimizing the ranking adapter, upstream classification errors or noisy transition supervision (such as misassigned ADD versus STRENGTHEN operations) cannot be corrected through the ranking objective. To what extent does error accumulation across these decoupled stages hurt downstream ranking, and did you experiment with joint or end-to-end fine-tuning?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.