acceptodds
Under review as a conference paper at ICLR 2027

Generative Contextual Ranking: Request-Level Global Optimization of Recommendation Slates

Abstract

Modern recommender systems typically score user–item pairs independently, even though users consume recommendations as ordered slates. This mismatch prevents rankers from modeling how an item’s utility depends on earlier recommendations and can place redundant content close together, reducing diversity and degrading user experience. To solve this problem, we propose Generative Contextual Ranking (GCR), which replaces independent scoring with autoregressive, request-level slate construction. At each step, GCR selects an item conditioned on the user's history and the items already placed, capturing inter-item effects such as topical redundancy, complementarity and position dependence. GCR is a lightweight architectural change that can be inserted at any ranking stage without a full-stack rewrite. Its principal cost is the latency of autoregressive decoding, which we offset with a set of serving optimizations that improve throughput by over 10. Experiments on a large-scale short-form video platform and on a public benchmark demonstrates GCR can improve offline prediction quality and deliver statistically significant online gains.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Recommendation Systems, Contextual Ranking, Sequence Modeling, Slate Generation

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 2 reviews, above 9% of the others.
Reviewer_nS73OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Generative Contextual Ranking (GCR), which is an autoregressive slate-ranking framework for recommendation systems that replaces pointwise user-item scoring with context-aware, request-level slate construction. Rather than generating intermediate semantic IDs, GCR decodes directly over real candidate item IDs using causal attention over previously placed items. To meet strict production latency constraints, GCR embeds business and diversity policy constraints directly into GPU-resident tensor operations within the model graph and applies serving optimizations (KV caching, batched inference, dynamic pagination, and ahead-of-time compilation) to achieve a 10x throughput improvement. The framework is evaluated offline on an internal dataset and the open-source KuaiRand benchmark, as well as via an online A/B test on a large-scale short-form video platform.

Critical strengths

The paper validates its design through a large-scale online A/B test against a strong industrial baseline (a generator-evaluator framework), demonstrating statistically significant engagement gains (+0.60% watch time, +4.85% follow). Showing that autoregressive slate generation can completely replace a production late-stage ranker within existing GPU capacity constraints provides strong applied value.

Critical weaknesses
  1. On the KuaiRand dataset, GCR is compared exclusively against a pointwise HSTU baseline. It omits evaluations against established slate-aware or generative reranking baselines (e.g., SetRank, List-GR). This makes it impossible to determine whether GCR's offline performance gains stem from its specific formulation or simply from upgrading an independent pointwise model.
  2. Limited novelty, where most of techniques are well-established previously, such as KV caching, batched inference, and PyTorch ahead-of-time compilation.
Questions for authors

Can you provide offline benchmark comparisons on KuaiRand against established context-aware or generative reranking models?

1 comment

Thank you for your time and review, and for your positive comments on the online A/B test and the production deployment.

On the KuaiRand dataset, GCR is compared exclusively against a pointwise HSTU baseline. It omits evaluations against established slate-aware or generative reranking baselines (e.g., SetRank, List-GR). This makes it impossible to determine whether GCR's offline performance gains stem from its specific formulation or simply from upgrading an independent pointwise model.

We added Seq2Slate, a slate-aware generative reranker, to the KuaiRand experiments (Section 5.3, Table 4), adapted to use the same features, prediction head and loss as HSTU and GCR. Both GCR and HSTU outperform it on all five tasks across both datasets, which we attribute mainly to its LSTM backbone.

That said, our main contribution is not the model architecture alone. It is the combination of a model that predicts several engagement outcomes for each real candidate given the items already placed, configurable business and diversity rules applied inside the model at every step, and a compiled GPU-resident serving design that runs both within production capacity. An offline AUC comparison only tests the first part; the others are tested by the online A/B test and the serving results. We clarified this in Section 2.2.

We did not run SetRank or ListGR because neither can be faithfully adapted to KuaiRand. SetRank reranks a full candidate set, including candidates that are never shown, and scores them jointly in one pass. KuaiRand contains only served items, not the candidates that were considered but not shown, so SetRank's actual input is not available. Running it on the served list would mean ranking an already-ranked output, with each item scored using knowledge of the items served after it, and a causal version would lose the permutation invariance that defines it. Neither would be a faithful adaptation of SetRank. ListGR is a generative retrieval model: given a text query, it generates semantic document IDs built by clustering and is trained to match a relevance-ordered list. Evaluating it on KuaiRand would mean replacing the query with user history, building a semantic ID vocabulary for the videos, and turning its likelihoods into engagement predictions. That would be a different model, and it relies on the semantic IDs that GCR is designed to avoid. Seq2Slate's decoder already works left to right, and with its encoder made causal over the candidates (Appendix A.3) it fits our protocol of predicting each item from the items before it.

These public results should also be read with KuaiRand's limits in mind. KuaiRand-1K has about 4.37 million videos but only 11.7 million interactions, fewer than three per video on average (Appendix A.1), which limits what item-ID embeddings can learn. KuaiRand also has no request IDs or display positions, so slates must be reconstructed. Our evaluation on internal data uses billions of samples (Section 5.2.1).

Limited novelty, where most of techniques are well-established previously, such as KV caching, batched inference, and PyTorch ahead-of-time compilation.

We agree these techniques are standard, and we don't claim them as contributions. On the serving side, our aim is making autoregressive decoding, which multiplies the cost of every request by the slate length, practical in a production recommender serving a large-scale platform. A direct implementation of GCR ran roughly an order of magnitude below the production system's throughput (Section 4.5), so without this work the model could not be deployed within production capacity.

The difficulty was not the individual techniques but making them work with a policy that runs at every step. In LLM decoding, the next token comes from a fixed vocabulary. In GCR, each step applies business and diversity rules that are configured per request, depend on the items already placed, and run over a candidate set whose size varies by request. Keeping these rules outside the model would add a CPU or service round trip at every step (Section 4.4), and compiling them required branch-free rule evaluation and bounded dynamic shapes (Section 4.5.4). As a result, optimized GCR reaches throughput comparable to the production reranker on the same traffic and serving machines while meeting the production latency requirement (Section 5.2.3).

Reviewer_BNywOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes generative contextual ranking, which is to not care about the single next item prediction quality, but emphasizing the importance of consider how the next list of items should be placed (like not listing similar contents too closely). To serve this model, the authors adapt recent serving techniques from the LLM community to serve the GCR model. Both offline experiments on public datasets and online experiments on industrial systems demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Both offline experiments on public datasets and online experiments on industrial systems demonstrate the effectiveness of the proposed method.
  2. The idea of "Branch-free rule evaluation" that vectorizes policy like rule filtering to GPU operation to optimize efficiency is both interesting and practical.
Critical weaknesses
  1. Novelty. The idea itself doesn't seem novel to me. Dated from Bello et al. "Seq2Slate: Re-ranking and Slate Optimization with RNNs." to recent papers like Huang et al. "Listwise Preference Diffusion Optimization for User Behavior Trajectories Prediction.", people have discussed this kind of "next list recommendation" in different ways already. Per my current understanding, I don't really get the point why GCR is conceptually novel considering of these methods.
  2. Overclaim. The paper claims the proposed method to be "Request-level global optimization". However, the proposed method still requires greedy sequential decoding, i.e., when generating early items, the model doesn't persevere future slots by design, but mostly depending on the given context only.
  3. It's good to have a section discussing strategies to optimize serving efficiency. However, the introduced techniques are quite common and widely used in the LLM community. Deploying these techniques here doesn't require scenario- or task- specific adaptation, so it's at least not substantial contribution to me.
Questions for authors

Why GCR is conceptually novel considering existing methods like Seq2Slate, and some listwise recommendation methods.

2 comments

Thank you for your time and careful review, and for your positive comments on the online results and branch-free rule evaluation.

The paper proposes generative contextual ranking, which is to not care about the single next item prediction quality, but emphasizing the importance of consider how the next list of items should be placed...

A small clarification: GCR does not trade per-item accuracy for list arrangement. It predicts each item's engagement like a standard ranker (Eq. 4), but conditioned on the items already placed, and this improves per-item prediction quality (Table 1). The order of the slate then comes from selecting the best-scoring candidate at each step, not from a model trained to output lists.

Why GCR is conceptually novel considering existing methods like Seq2Slate, and some listwise recommendation methods.

We agree that selecting items one at a time based on earlier selections is not new, and we don't claim it. GCR differs in three ways. Seq2Slate directly outputs the next item, while GCR predicts multiple engagement outcomes for each remaining candidate given the items already placed. A configurable, non-learned policy layer then applies business and diversity rules at every step. And this policy runs as GPU tensor operations inside the same compiled graph as the model, so the whole slate is built on the GPU with no external calls per step. The same distinction applies to listwise methods: our contribution is not generating a list, but this combination of contextual prediction, in-model policy and GPU-resident serving. We revised Section 2.2 to state this directly: "Our focus is integrating this contextual prediction and policy execution into a compiled GPU-resident ranking system."

We also added Seq2Slate to the KuaiRand experiments (Section 5.3, Table 4). It uses the same features, prediction head and loss as HSTU and GCR, but has lower AUC than both on all five tasks across both datasets. We attribute this mainly to the backbone: Seq2Slate is LSTM-based, while HSTU and GCR use attention-based HSTU layers.

The paper claims the proposed method to be "Request-level global optimization". However, the proposed method still requires greedy sequential decoding...

You are right that GCR selects greedily and does not plan ahead for later positions. Each choice uses the items already placed, but the model does not consider how that choice affects what comes next. We revised the paper to describe this accurately. The Introduction now frames the contribution as request-level contextual slate construction. After Eq. 3, Section 3.3 now states that the score "does not explicitly evaluate how selecting that candidate affects subsequent items." We also reworded the Conclusion to describe GCR as constructing the slate through prefix-conditioned engagement predictions, rather than optimizing the request as a whole.

Deploying these techniques here doesn't require scenario- or task- specific adaptation...

We agree that caching, batching and compilation are standard, and we don't claim them as new. But applying them to GCR did require task-specific work. In LLM decoding, each step picks the next token from a fixed vocabulary. In GCR, each step must also apply business and diversity rules that are configured per request, depend on the items already placed, and run over a candidate set whose size changes from request to request. Ahead-of-time compilers assume fixed shapes and no data-dependent branches, so this logic does not compile as is, and running it outside the model would add a CPU or service round trip at every step (Section 4.4). We made it compilable by writing the rules as branch-free tensor operations and bounding the dynamic shapes (Section 4.5.4). Together with the other serving optimizations, this closed a gap of roughly an order of magnitude to production throughput (Section 4.5). Optimized GCR now reaches throughput comparable to the production reranker on the same traffic and serving machines while meeting the production latency requirement (Section 5.2.3).

Thank you for taking the time to address my concerns. The revisions generally improve the writing and clarity of the paper. However, my main concern remains the novelty of the contribution. While I do value the practical aspects of the work (which I highlighted as strengths in my original review), after reading the rebuttal I am still not convinced that the overall contribution is substantial enough for a full research paper. I will maintain my current rating.

Comments on the paper

1 comment

Thank you for your time and detailed review, and for your positive comments on the online comparison against a production slate reranker and on the capacity-matched HSTU comparison.

Does the score in Equation 3 account for the effect of choosing an item on engagement with later items, or only its current predicted engagement? Please clarify how the training objective and greedy selection relate to the claimed optimization of total slate utility.

Only its current predicted engagement plus policy adjustments. Both depend on the items already placed, so earlier choices shape how later candidates are scored. But the model does not consider how picking an item changes engagement with the items after it. Eq. 4 trains each item's prediction given the logged prefix, and greedy selection picks the best candidate at each step. Neither directly optimizes total slate utility, so you are right that our earlier wording overstated this.

We changed the wording to match. The Introduction now describes the contribution as request-level contextual slate construction, in which GCR updates each candidate's predictions using the selected prefix "and constructs the slate greedily." After Eq. 3, Section 3.3 now states that the score "does not explicitly evaluate how selecting that candidate affects subsequent items."

What is the main methodological advantage over existing autoregressive slate methods such as Seq2Slate?

We agree that selecting items one at a time is not new, and we don't claim it. GCR differs in three ways. Seq2Slate directly outputs the next item, while GCR predicts multiple engagement outcomes for each remaining candidate given the items already placed. A configurable, non-learned policy layer then applies business and diversity rules at every step. And this policy runs as GPU tensor operations inside the same compiled graph as the model, so it adds no external calls per step. Because GCR is trained to predict engagement given a prefix rather than to imitate the logged order (Section 4.3.1), the same model works under different per-request value weights and rules. We revised Section 2.2 to state this contribution directly.

We also added Seq2Slate to the KuaiRand experiments (Section 5.3, Table 4), using the same features, prediction head and loss as HSTU and GCR. It has lower AUC than both on all five tasks across both datasets, which we attribute mainly to its LSTM backbone.

Is there a representative comparison or an online ablation that isolates contextual prediction while keeping the decoding procedure and business policies fixed?

Offline, the closest test is the HSTU-GCR pair in Table 4. The two models differ only in whether candidates attend to preceding items. This isolates context for prediction quality, but it does not involve decoding or policies. On our internal dataset, Table 1 points the same way: the NE gain grows deeper into the slate, as the model sees more of the items that would be shown before each candidate. It knows only which items those are, not how the user engaged with them.

Online, no. Our A/B test evaluates the complete system, so it does not separate contextual prediction from the rest. Offline, however, when we disabled candidate-to-candidate attention in GCR, its predictions were not substantially better than those of the existing ranker. That is why we did not take this configuration to online testing, and it suggests that most of GCR's offline gain comes from contextual prediction.

Under the same hardware, candidate and slate distributions, and latency requirement, how do optimized GCR and the production baseline compare in sustainable throughput and p95 or p99 latency?

We agree that a 10x gain over naive GCR does not by itself show comparable capacity. We compared optimized GCR with the production reranker directly and added the result to Section 5.2.3: "On the same production traffic and serving machines, optimized GCR achieved throughput comparable to the production reranker while meeting the production latency requirement." Because both systems served the same traffic, they saw the same candidate and slate distributions. We cannot share absolute QPS, latency values or candidate and slate sizes for confidentiality reasons, which is also why Table 3 is normalized to naive GCR.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.