acceptodds
Under review as a conference paper at ICLR 2027

Recommendation World Models for Future-State Control

Abstract

Sequential recommendation optimizes which items to rank, while each displayed slate also shapes subsequent feedback and user state. We study how a trained ranker can support decisions about these future consequences. We introduce UA-TWM, a utility-anchored world-model interface that constructs nearby slate actions, estimates their target-relevant consequences, and selects an alternative subject to utility constraints. The reference slate serves as a fallback when no alternative qualifies. A logged-replay instantiation combines utility and target-gain estimates with calibrated failure-risk prediction; a closed-loop instantiation uses one-step state-action prediction and updates its decisions after observed feedback. We evaluate transfer across twelve sequential backbones on MovieLens-25M and KuaiRand-Pure, and repeated target-directed interaction in KuaiSim. Attaching the interface improves Recall@20, NDCG@20, and future-state alignment for every matched logged backbone. Selection ablations reveal the utility and risk costs of aggressive target pursuit, while closed-loop diagnostics isolate the contribution of action-conditioned prediction. Local consequence modeling thus enables target-aware selection around a trained sequential ranker.

Primary area:
learning on time series and dynamical systems
Keywords:
Recommender Systems, Constrained Decision-Making

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_dEFoOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes UA-TWM, a utility-anchored interface that adds target-directed slate selection to an existing recommendation system. It separates candidate generation, consequence evaluation, and constrained selection, retaining the original recommendation when no alternative qualifies. The logged implementation combines utility and target-alignment estimates with learned failure-risk prediction, while the closed-loop implementation uses a learned one-step state–action model.

Evaluation combines chronological replay across twelve recommendation backbones on MovieLens-25M and KuaiRand-Pure with repeated interaction in KuaiSim. The reported results show consistent improvements in ranking and historical future-state alignment, alongside simulator-based target improvements under conservative utility checks. Importantly, the paper distinguishes retrospective alignment from action-induced changes in the simulator.

Critical strengths
  1. The modular, utility-anchored design is practically meaningful. Rather than replacing a trained ranker, the method adds a decision layer that explicitly separates target improvement from utility preservation and can decline an intervention. I find this a sensible contribution: it addresses how to introduce controllability without assuming that every target-directed change is beneficial. Reporting selection coverage together with failure rates also makes the consequences of conservative fallback more transparent.

  2. The breadth of the matched evaluation is a substantive strength. Improvements across twelve frozen backbones and two datasets suggest that the decision layer is not narrowly tied to one recommendation architecture. The same-support ablations further examine the utility, target, and risk trade-offs, while KuaiSim provides complementary evidence beyond static replay. Together, these experiments support the usefulness of the framework, even though they do not establish general long-horizon control capabilities.

Critical weaknesses
  1. The connection between the horizon-level objective and the implemented controller remains insufficiently validated. Equation (3) concerns trajectory-level target loss and cumulative utility, whereas the closed-loop selector uses one-step target predictions and an immediate-click check, without multi-step planning or a continuation-value estimate. This is a reasonable local approximation, not an inherently invalid approach. However, the current experiments do not establish when it adequately handles delayed trade-offs, and the conceptual discussion of related reinforcement-learning approaches is not accompanied by a matched goal-conditioned or constrained-RL comparison. This limits the evidence for its advantages beyond short-term constrained selection.

  2. The incremental contribution of learned consequence prediction needs cleaner isolation. The graph-semantic proposal rules are described at a high level, yet Figure 6 shows that they supply all 64 non-anchor selections, with no local-edit candidate selected. The state-only ablation is informative, but removing action inputs also prevents the predictor from distinguishing candidate slates at the same state. An action-aware heuristic selector using identical candidates and utility gates would therefore provide a stronger test of what learned dynamics adds beyond candidate construction. More precise proposal specifications are also important for reproducing this central component.

Questions for authors
  1. How are horizon-level targets, particularly cumulative coverage, translated into the one-step score DstepD_{\mathrm{step}}? What evidence supports using immediate utility checks to approximate the cumulative-utility constraint? A comparison with a long-horizon value learner or multi-step planner under matched candidates, data budgets, and utility tolerances would directly inform my assessment.

  2. Could you specify the scoring functions, weights, and click-floor variants defining the six graph-semantic proposals, including what “graph” denotes operationally? Why are the two anchor-local candidates never selected in the reported decision trace?

  3. Which target components depend on stochastic user feedback, rather than being directly computable from the current history and candidate slate? For the directly computable components, how would a selector using those statistics compare with the learned predictor when candidate support and utility gates are held fixed? This would help distinguish learned behavioral forecasting from exposure-statistic optimization.

Reviewer_Tq2iOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The authors argue that a good next-item ranking does not necessarily tell us how a recommendation will affect later user behavior. They therefore aim to steer future-state statistics toward a specified target while preserving recommendation utility. The proposed UA-TWM adds a decision layer around a frozen recommender. It generates alternative slates, estimates their target improvement and utility cost, and falls back to the original slate when no candidate passes the checks. Experiments cover ranking and future-statistic alignment on two logged datasets, and closed-loop target control in KuaiSim.

Critical strengths
  • The method can be attached to an existing recommender without retraining the backbone. This is a practical feature, and the experiments cover a broad set of backbones.
  • Reporting utility, intervention coverage, and failure rate together makes the selection behavior easier to understand.
Critical weaknesses
  1. My main concern is actually the problem itself. I find it a bit boring as currently presented. A recommender already uses history to predict what the user may want next. So saying existing methods focus on current ranking does not really explain what is missing. If the future target represents future preference, why is it a separate objective? If it is an externally specified goal, this sounds like an existing goal-conditioned recommendation problem.
  2. There are already works on future outcomes, long-term objectives, and controllable recommendation, including GoalRec and MocDT. The paper cites them, but I still cannot see the key difference beyond freezing a ranker, searching nearby slates, and adding constraints and fallback. This can be a practical design, but what is the new research contribution? Comparing with many standard ranking backbones does not answer this.
  3. I also feel the world model part is mostly a label here. The logged version uses historical preference statistics, backbone scores, and a risk classifier. It does not explicitly model how an action changes the future state. The closed-loop version does have a one-step predictor, but several targets are exposure statistics that can at least partly be computed directly. The paper has not convinced me why learning dynamics is needed for this task.
Questions for authors
  1. What exactly does the future target represent? What important need does it capture that existing preference prediction or goal-conditioned recommendation does not already address?
  2. What is the main technical difference from GoalRec and MocDT? What limitation of these methods does the proposed interface actually solve?
  3. Can the authors show a case where the learned dynamics makes a better decision than directly computing slate statistics under the same utility constraints and fallback rule? This would help explain what the world model is actually contributing.
Reviewer_yrMmOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies how to use a trained sequential recommender to select a slate close to a reference anchor slate that steers a future-state statistic toward a target while keeping utility within a tolerance of the anchor. The authors formulate this as selective slate-action control and propose UA-TWM, which (i) generates anchor-local candidate slates, (ii) predicts each candidate's utility gain, target gain, and failure risk, and (iii) selects the admissible candidate with the largest predicted target gain or falls back to the anchor. In logged replay, candidates are relevance-plus-target-affinity reranks of the top-200 pool, using closed-form margin heads for utility/target gains and an L2-regularized logistic head for risk. In closed-loop simulation (KuaiSim), a ridge one-step state-action model evaluates candidates around a diversity-rerank anchor. The method is evaluated across 12 backbones on MovieLens-25M and KuaiRand-Pure in logged replay, alongside ablations, risk diagnostics, and a multi-scenario study in KuaiSim.

Critical strengths

A well-structured selective-control formulation with explicit abstention accounting. The paper cleanly separates candidate generation, prediction, and constrained selection (section 3.2, Algorithm 1). Crucially, reporting coverage alongside invalidity prevents a policy from appearing artificially safe simply by always defaulting to the anchor (Tab. 1). This conceptual framing is broadly reusable and allows the comparisons across backbones and datasets to be read meaningfully as a risk–coverage trade-off rather than a single aggregated score.

Critical weaknesses

W1. Unexplained numerical regularities in the main logged results and missing variance/seed details (Tab. 1, Tab. 6).

  • While reviewing the logged evaluation, I noticed several striking numerical patterns in Tab. 1 and Tab. 6 that are not explained in the text, alongside an absence of reported seeds or run-to-run variance. Across both MovieLens-25M and KuaiRand-Pure, the same six backbones share the exact same ΔRecall@20 and ΔS values in Tab. 1 and nearly identical 95% CIs in Tab. 6, despite having widely different base performances and ΔNDCG@20 values (and, on MovieLens-25M, coverage rates ranging from 41.5% to 94.2%).

  • Across all 24 pairs in Tab. 1, the relative gain in NDCG@20 matches the relative gain in Recall@20 (i.e., ΔNDCG@20 equals ΔRecall@20 × Nbase/RbaseN_{\text{base}}/R_{\text{base}} within rounding). Seeing such exact proportionality between a rank-sensitive and a rank-insensitive metric is unexpected, especially since it holds for only 9 of the 24 Aggressive WM rows in Tab. 6.

  • All confidence intervals in Tab. 6 are perfectly symmetric around the point estimate, and the differences between the Constrained and Aggressive policies are completely uniform across six backbones on both datasets.

The paper does not state how the CIs in Tab. 6 were computed, and the released code does not appear to contain any mechanism that would cause gains or intervals to coincide across different backbones or force the relative gains of Recall@20 and NDCG@20 to match so closely. Because the main offline claims rely heavily on Tab. 1, it is difficult to evaluate this contribution without a clear explanation of how these results were generated and access to the underlying per-user outcome files.

W2. Gap between the "future-state control" framing and what the experiments establish.

In logged replay, the target is the user's own prefix tag profile, and the gain heads serve as observational proxies for agreement with held-out positives.. Because logged positives cannot identify the transition distribution of an unshown slate, Tab. 1 effectively measures profile-affinity reranking with an abstention gate rather than true counterfactual state control. Indeed, in Fig. 3, simple target-only reranking achieves a comparable ΔN@20 and most of the ΔS of UA-TWM, albeit at a higher unsafe rate. In the closed-loop study, RFSS-U reaches a much lower target L1 than UA-TWM with zero invalid claims. Although the text notes that RFSS-U has lower click utility than the anchor, none of its claims fail the utility constraint (Eq. 16), whose tolerance ϵu\epsilon_u is omitted. Without ϵu\epsilon_u, per-seed variances, and tabulated ablations, it is difficult to evaluate UA-TWM's advantage under the paper's own admissibility criteria.

Questions for authors

Q1. Could the authors explain the numerical regularities observed in Tab. 1 and Tab. 6 (as described in W1)? Specifically, what causes the identical Δ\DeltaRecall@20 and ΔS\Delta S values across multiple distinct backbones, the exact scaling of Δ\DeltaNDCG@20 with ΔRecall@20×Nbase/Rbase\Delta\text{Recall@20} \times N_{\text{base}}/R_{\text{base}} across all 24 pairs, the perfectly symmetric confidence intervals, and the uniform differences between the Constrained and Aggressive policies? In addition, how many independent runs underlie each cell, how were the 95% CIs computed, and could you provide the per-user outcome files (decisions.jsonl) for all 24 pairs?

Q2. Could you clarify the exact setup used to produce Tab. 1? Specifically, did the reported runs use the closed-form margins in Eq. 10–11 or the ridge regression heads implemented in the released code (cli.py:84–89)? Please also provide the numeric values for the thresholds in Eq. 13 and Eq. 16 (δZ,ϵuser,ϵu,δz\delta_Z, \epsilon_{\text{user}}, \epsilon_u, \delta_z), and clarify why the code searches a smaller gate grid than the one described in Tab. 3. Regarding the closed-loop results in Tab. 2, what value of ϵu\epsilon_u allows all RFSS-U claims to be valid despite having a lower mean click than the anchor, and under that tolerance, why is UA-TWM preferred over RFSS-U? Finally, please report the per-seed variances for Tab. 2.

Q3. Which backbone is shown in Fig. 3, and could you provide a table of the operating points for all compared alternatives and ablations? Also, is the full selector's improvement over simple target-only reranking (T) larger than run-to-run variation, and which component primarily drives this gain?

Q4. In the released code, a candidate slate containing the exact same items as the anchor in a different order is counted as a non-anchor claim. Since a pure reordering leaves Recall@20 and SS unchanged and is automatically valid when δZ=0\delta_Z = 0, were such reorderings counted as claims in Tab. 1? If so, what fraction of the reported coverage on KuaiRand do they account for?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.