acceptodds
Under review as a conference paper at ICLR 2027

Evolving Supervision through Learning Feedback for Generative Recommendation

Abstract

Generative recommendation models commonly learn target-item distributions through next-token prediction (NTP) on historical interactions. Yet observed interactions are incomplete proxies for users' latent preferences, rather than ground-truth descriptions of their preferences. Supervision can be enriched by extending click-based targets to purchases or incorporating behavioral signals from other contexts. The usefulness of additional supervision depends both on what information it adds and on what the current model has already learned. This motivates us to jointly consider supervision construction and feedback-guided learning. We propose a feedback-driven supervision evolution framework with two nested cycles. The outer Supervision Evolution Cycle (SEC) uses large language models (LLMs) to analyze updated performance feedback, diagnose supervision needs, and guide subsequent supervision construction. Within each round, the inner Feedback-Guided Learning Cycle (FGLC) updates the policy for incorporating constructed supervision based on its observed learning effects. We apply the framework to expand supervision in a recommendation retrieval system. Multi-round offline experiments on industrial recommendation retrieval show that the framework improves retrieval performance under matched total added supervision. Online A/B testing also shows improvements in ad clicks and revenue. The system has been fully deployed on a large-scale industrial e-commerce platform serving hundreds of millions of active users.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Generative Recommendation, Self-evolving supervision

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BtwnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a feedback-driven supervision evolution framework for generative recommendation. The framework consists of two components: the Supervision Evolution Cycle (SEC), which uses LLMs to identify and construct additional supervision based on model feedback, and the Feedback-Guided Learning Cycle (FGLC), which dynamically adjusts the learning order of the constructed supervision through short training probes. The framework is instantiated in an industrial recommendation system using intent-based supervision expansion. Offline experiments and online A/B testing demonstrate improvements in retrieval performance and business metrics.

Critical strengths
  1. The paper addresses a relevant problem in generative recommendation, where observed user interactions provide incomplete supervision of user preferences. Adapting supervision construction and learning based on model feedback is a reasonable and practically meaningful direction.

  2. The paper provides extensive offline experiments, including controlled comparisons that examine the contributions of SEC and FGLC separately. The results consistently demonstrate performance improvements over the considered baselines.

  3. The framework is evaluated on a large-scale industrial recommendation system, with online A/B testing showing improvements in ad clicks and revenue. The reported deployment further supports its practical relevance.

Critical weaknesses
  1. The proposed framework largely builds on existing ideas in feedback-driven data generation and adaptive curriculum learning. The authors themselves acknowledge these connections when introducing SEC and FGLC in Sections 4.1 and 4.2, but their technical distinctions from prior approaches could be better clarified. The intent-based supervision expansion in Section 4.3 also appears conceptually related to existing LLM-based recommendation augmentation methods. Overall, the methodological novelty seems somewhat limited despite the practical value of integrating these components.

  2. It seems that LLM-based diagnosis plays a central role in SEC by determining which categories should receive additional supervision. However, the paper provides limited details on how these decisions are made in practice. Although Appendix A includes the diagnosis prompt, concrete examples of the inputs and resulting decisions would be helpful. Moreover, the comparison with the low-recall baseline in Table 3 does not fully establish the benefit of using an LLM over simpler heuristic-based approaches that consider the same feedback information.

  3. At each stage, FGLC performs separate, relatively short training runs for all remaining supervision batches, followed by evaluations on the full decision set to select the next batch. This process is repeated after each model update, potentially introducing substantial computational overhead. While the authors acknowledge this overhead in Appendix C.2, the experiments use matched committed-update budgets rather than matched total computational budgets, making it difficult to assess the gains relative to the additional computational cost.

  4. Sections 4.1 and 4.2 introduce a general framework, whereas Section 4.3 presents its recommendation-specific instantiation through intent-based supervision expansion. Although this distinction is briefly mentioned, the transition feels somewhat abrupt, particularly given the section titles.

Questions for authors

NA

Reviewer_37VvOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the limitations of next-token prediction (NTP) supervision in generative recommendation systems, where historical user interactions provide incomplete signals of users' underlying preferences. The authors propose a feedback-driven supervision evolution framework that dynamically determines both what additional supervision to construct and how to incorporate it into model training. The proposed framework consists of two nested cycles:

  • Supervision Evolution Cycle (SEC): An LLM-based diagnosis mechanism that uses current model performance, historical feedback, and supervision coverage to identify categories requiring additional training supervision. These requests guide an intent-based supervision construction pipeline that generates new context–item training pairs.
  • Feedback-Guided Learning Cycle (FGLC): An adaptive learning mechanism that uses short training probes to estimate the utility of candidate supervision batches. Rather than fixing their learning order in advance, FGLC repeatedly reevaluates the remaining batches as the model evolves. The framework is evaluated on a large-scale industrial e-commerce recommendation system using a Qwen3-0.6B generative retriever. The experiments include multi-round offline evaluations, controlled comparisons of supervision construction and learning strategies, and online A/B testing. The complete framework achieves a category-macro Recall@1500 of 76.73%, compared with 72.41% for the fixed-request, initial-order baseline. An online evaluation over five evolution rounds reports relative improvements of 0.70% in ad clicks and 0.92% in revenue. Overall, the paper demonstrates the potential of jointly adapting supervision construction and learning strategies based on model feedback in industrial generative recommendation.
Critical strengths

S1. Joint consideration of supervision construction and adaptive learning. The paper addresses two closely related but often separately studied questions: what supervision should be added and how the resulting supervision should be incorporated into training.

S2. Well-designed controlled experiments supporting the main components. The experimental design is a notable strength. Table 1 compares four combinations of fixed/updated supervision requests and initial-order/feedback-based learning. The full framework achieves 76.73% Recall@1500, outperforming the other three configurations.

Critical weaknesses

W1. Computational efficiency and cost-effectiveness are insufficiently established. A major concern is the computational overhead introduced by FGLC. At each learning stage, the method probes every remaining candidate batch for 1,250 training updates before selecting the next batch. Since the remaining batches are reevaluated after each committed update stage, the total number of discarded probe updates can grow quadratically with the number of candidate batches.

W2. The specific contribution of LLM-based diagnosis remains insufficiently isolated. The SEC relies on an LLM to analyze model performance, historical feedback, and supervision coverage before selecting categories for expansion. However, the experimental comparisons mainly include a fixed initial plan and a simple low-recall selection heuristic. These baselines do not fully establish whether the gains arise from the LLM's reasoning capabilities or simply from using updated performance statistics and historical supervision information.

Questions for authors

Q1. Computational cost and efficiency Could the authors report the total training computation of each learning policy, including discarded probe updates, and provide a comparison under matched total computational budgets rather than only matched committed-update budgets? In particular, how does FGLC compare with Joint and Initial-order training when the baselines are allowed to use the computation saved by avoiding repeated probes?

Q2. Necessity of LLM-based supervision diagnosis How much of SEC's improvement can be attributed specifically to LLM-based diagnosis rather than the availability of updated performance feedback? Could the authors compare SEC with a non-LLM adaptive prioritization baseline that combines category-level recall, historical improvement, and existing supervision coverage? It would also be useful to provide examples of diagnosis decisions and explain why the LLM selected particular categories.

Reviewer_j9kXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a feedback-driven framework for generative recommendation, combining an outer loop that uses LLMs to decide which supervision to construct with an inner loop that dynamically orders the resulting category-specific batches. Its main contribution is connecting supervision construction and adaptive training through repeated model feedback.

Critical strengths
  1. The motivation is sensible: the usefulness of additional supervision should depend on the model's current learning state. The factorial and controlled experiments help distinguish the contributions of updated construction decisions and adaptive learning order.
  2. The industrial-scale evaluation and reported online improvements provide evidence of practical value beyond offline retrieval metrics.
Critical weaknesses
  1. I am not convinced that FGLC adequately handles harmful updates or forgetting. Previously learned additional batches are not replayed, and every batch must eventually be incorporated, even if all remaining probe gains are negative. Mixing historical data and scoring overall recall provide some protection, but neither guarantees retention of newly learned preferences. Aggregate results do not show whether improvements come at the expense of particular categories.
  2. The evidence does not yet establish the framework's value under a comparable total compute budget or across settings. Repeated probing adds computation beyond the matched committed updates, and the paper provides no comparison with Joint training under equal total compute. All reported experiments also rely on proprietary industrial data which limits independent verification and broader conclusions.
Questions for authors
  1. What happens when all remaining probe gains are non-positive, or when the selected full-stage update reduces performance? Please clarify any stopping or rollback rules and provide category-level forgetting measurements, ideally including a baseline that replays earlier additional batches.
  2. Does FGLC still outperform Joint training when total computation, including discarded probes, is matched?
  3. How do FGLC and Initial-order compare with learning the same batches in a random order under identical stage budgets and historical-data mixing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.