acceptodds
Under review as a conference paper at ICLR 2027

QD-Rec: Open-Ended Evolution of Recommendations via Quality–Diversity

Abstract

Open-ended search based on foundation models has produced faster algorithms, new results on open mathematical problems, and creative discoveries. We extend this paradigm to decision-making, where the set of possible choices is fixed but the reasons for choosing them are not. Examples are choosing which company to acquire, which paper to read, or which customer to target. In high-stakes decisions, comparing diverse, well-reasoned options can be more useful than a single ranked list. We formalize open-ended recommendation as building a diverse, high-quality set of recommendations, each an item together with a hypothesis, a natural-language reason why the item fits the needs of a decision maker. We propose QD-Rec, a quality-diversity method that selects items with a contextual bandit and generates hypotheses with a foundation model from earlier ones in an archive. A new recommendation is kept in the archive if it is novel or better than its nearest neighbor. Novel recommendations can thus serve as stepping stones for later hypotheses. Across M&A, customer targeting, and paper and book recommendation, QD-Rec gives the highest mean quality and builds the largest archive in most settings.

Primary area:
foundation or frontier models, including LLMs
Keywords:
open-endedness, quality-diversity, evolution

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_hvZXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formulates open-ended recommendation as a search over hypothesis pairs drawn from a fixed item pool. QD-Rec maintains an embedding-based archive: an LLM first assesses novelty, and a non-novel entry then competes against its nearest neighbor in a blind pairwise duel. Linear Thompson sampling selects items, while exemplars sampled from the archive guide hypothesis generation through mutation, crossover, or exploration. The authors evaluate the approach on Goodreads, M&A target discovery, synthetic-customer marketing, and paper-abstract exploration, using two proposer models. Comparisons include ablations and feedback-free MMR/DPP rerankers on Goodreads. Overall, QD-Rec improves mean quality, archive size, and coverage in most LLM-scored settings.

Critical strengths

1.Recommendation is usually treated as ranking a fixed set of items; this paper highlights that the space of reasons for recommending them remains open-ended and connects it to quality–diversity and novelty search.

2.The algorithm is simple and interpretable.

3.The experiments cover four different task types and produces quality and coverage gains.

Critical weaknesses
  1. The evaluation is circular.*The bandit, replacement decisions, and final quality metric all rely on the same GPT-5.6-Terra evaluator, so the method may optimize for the evaluator’s preferences rather than independently assessed human quality. Apart from Goodreads, there is no independent human evaluation or evaluation by a model uninvolved in the optimization process to validate the practical quality of the proposed method in M&A, marketing, and paper recommendation. The authors should provide evidence of the alignment between human evaluations and LLM-as-a-judge evaluations.

  2. The MMR/DPP comparison is not a controlled ablation of item selection. The QD methods receive up to 1,000 instances of real feedback and update the bandit, whereas the baselines receive no feedback. Although the authors also argue that online learning and archive machinery justify their computational cost, this claim should still be treated with caution. Additional baselines should include a quality-only bandit, randomized feedback, a fixed item order, and methods with matched feedback budgets.

  3. Archive size nn may reward novel but otherwise unremarkable hypotheses. Admission is not governed by an explicit distance threshold, and new entries can accumulate within already-covered regions. Therefore, a larger archive does not necessarily imply more useful diversity. Deduplicated semantic or behavioral coverage, human-assessed utility, and quality–diversity Pareto frontiers would provide more informative measures.

  4. The baseline comparisons appear insufficient. Beyond MMR/DPP on Goodreads, the evaluation should include a broader range of baseline models to better establish the effectiveness of the proposed method. The two ablations do not fully isolate the respective contributions of the bandit, exemplars, and LLM-based novelty assessment.

  5. The M&A and marketing applications raise concerns about factual errors, discriminatory targeting, and unfair competition. The ethics statement mentions human verification but does not analyze how incorrect hypotheses might affect acquisition screening or how recommendation diversity might affect fairness toward different demographic groups.

Questions for authors

See Weekness

Reviewer_k59qOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper applies foundation model based open-endedness for recommendations. Here the set of choices is fixed but the reason for making a particular choice isn’t and hence is open-ended. The authors propose a quality-diversity method called QD-Rec where the items are selected using a contextual bandit along with a diversity metric. An LLM generates a justification for making the item choices and an LLM judge or a human scores them. An archive of the recommendations is maintained and a selected item is added to the archive only if it passes a novelty or a local quality comparison test. They also discuss two variants to ablate different aspects of the proposed method. One where the items are selected at random instead of using a bandit and another where the justifications are generated zero-shot along with random item selection. The proposed methods are evaluated in marketing, M&A, and paper and book recommendation domains. The performance is measured using four metrics - quality, diversity, coverage, archive size and QD-Rec has the best performance in most of the settings and in others, is not significantly worse than the best performance.

Critical strengths
  1. The recommendation framework comprising of the item-hypothesis pairs makes the recommendation process more interpretable. Using a bandit and a diversity measure to select the items and generating hypotheses based on past selections is a well rounded way of generating the pairs. The novelty/quality comparison step acts as an secondary check to measure the quality and diversity of the item along with the hypothesis.

  2. The paper also discusses ablations of the proposed method for item selection and the hypothesis generation. The observed performance improvements, wherever applicable, are impressive and the experiments cover different domains.

Critical weaknesses
  1. The paper doesn’t compare the proposed method of generating hypotheses against different methods, apart from the zero-shot generation. Also, the importance of the novelty/local comparison quality rule is not studied.

  2. Performance is not compared to any other learning baselines. The experiments involve comparisons only against ablations of the proposed method and a no learning baseline on the book recommendation.

Questions for authors
  1. What does the performance look like when β=0\beta = 0 i.e. no diversity measure applied?

  2. How would different ways of generating the exemplars and the hypotheses compare against the procedures adopted in the paper?

  3. What are the results if you remove the archive retention rule and instead retain all the generated item-hypothesis pairs?

  4. In the fig 2(b) mean quality metric plots, why does Novelty + evo perform better than QD-rec initially for ML Papers and similarly, Novelty better than Novelty + evo for Marketing?

Reviewer_23DqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The article defines open-ended recommendations. With a fixed number of items to choose from; the goal is to produce a set of high-quality, distinct recommendations that are recommended as being a good match for the individual making the decision. For each item recommended there will be a natural language explanation as to why the item would be appropriate. During each step of the process, QD-Rec (the suggested approach) selects one of the available items based upon Thompson sampling with a diversity bonus. After selecting an item, a large language model (LLM) creates a new explanation by modifying existing explanations in the archive through mutation, crossover, or exploration. An LLM judge determines whether or not the newly created explanation is original. Original explanations are added to the archive while explanations that are not original are only included in the archive if they defeat the nearest explanation in a blind pairwise comparison. There were four areas of experimentation: books recommended by Goodreads members, identifying companies for mergers & acquisitions, targeting customers for marketing purposes, and recommending papers related to machine learning. Comparisons were made between QD-Rec and two simplified versions of QD-Rec. Evaluation was done primarily through the average quality of explanations as rated by the LLM, the diversity of explanations in the archive, the amount of content covered by the archive, and the total size of the archive.

Critical strengths

This paper provides a new viewpoint and an easily understood method. It states that, even though the candidate items are constant, the list of reasons why we would recommend those items will remain infinite. Therefore, this allows Quality-Diversity search to be used in the area of recommendations. There are three parts to this method: item selection, hypothesis generation, and local archive updates. Each part of the method is well-documented in Algorithm 1 and Table 1 as well as the two ablations, thus making the method simple to comprehend and recreate.

The experiments are conducted in several domains, which are documented clearly. There are four domains; Books, Mergers & Acquisitions (M&A), Marketing, and Papers. Two different proposer models are used in each domain. A judge from a different model family than the proposer models is employed to control for self-preference bias. The reported findings include means and standard errors for five seeds, API cost per run, the sensitivity of the embedding model, and all of the prompts. In the Goodreads domain, the experimenters use actual human ratings as their reference signal.

Critical weaknesses

The “embedding retrieval + LLM-written rationale” baseline is only evaluated on one dataset, and only in the appendix, so we cannot assess whether QD-Rec's many model calls per run is worth the cost in the other domains. The method optimizes for the judge whose scores it uses to measure quality, and the authors do not check if it agrees with humans.

Questions for authors
  1. Why is the "embedding retrieval + LLM-written rationale" baseline not present in the other domains?
  2. Why this value of β\beta and how the results change with other β\beta's?
Reviewer_bnaHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper formalizes open-ended recommendation as searching for diverse pairs of candidate items and natural language hypotheses explaining their suitability for a given profile. It proposes QD-Rec, combining contextual bandit selection, hypothesis generation from archived examples, and archive updates through novelty checks and local quality comparisons. Experiments across four domains and two proposer models compare QD-Rec with component ablations under matched evaluation budgets and five random seeds. Results show the highest mean quality in most settings evaluated by an LLM and generally larger archives. Additional experiments use historical human ratings for book recommendation and assess sensitivity to the embedding model.

Critical strengths
  1. Clear problem formulation and coherent methodology. The paper formulates recommendation as a search over candidate items and their supporting hypotheses, motivating a coherent integration of contextual bandits, hypothesis evolution, and local archive competition.

  2. Broad and controlled empirical evaluation. Experiments span four domains and two proposer models, with matched evaluation budgets, component ablations, and five random seeds. Historical human ratings for books and an embedding sensitivity analysis provide complementary evidence.

Critical weaknesses
  1. Insufficient independent validation of rationale quality. Although using the search evaluator for final assessment is consistent with the stated protocol, the paper does not report an independent human evaluation of the generated rationales. It therefore remains unclear whether higher model scores correspond to more accurate or useful explanations. Independent validation, for instance through blinded evaluation by domain experts, would strengthen this interpretation. The Goodreads experiment does not directly address this gap, since historical ratings assess books rather than the generated rationales.

  2. Limited comparison with alternatives using equivalent feedback. MMR and DPP receive no rating feedback, whereas the QD methods use up to 1,000 evaluations. Although explicitly acknowledged, this difference limits conclusions about the framework’s incremental value beyond feedback learning. A contextual bandit without the diversity bonus, using the same generator, archive rules, and evaluation budget, would provide a more informative comparison.

Questions for authors
  1. Human validation of rationale quality. Have any generated rationales been independently assessed by human evaluators? If so, do human preferences between QD-Rec and its ablations agree with the LLM scores? This concerns validation of the current results rather than extending the method to incorporate human feedback during search.

  2. Contribution of the diversity bonus. Are results available for the contextual bandit with β = 0, using the same generator, archive rules, and evaluation budget as QD-Rec? Comparing its Quality and Diversity with β = 1 would help distinguish the contribution of the diversity bonus from that of learning which items to evaluate.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.