- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper formulates open-ended recommendation as a search over hypothesis pairs drawn from a fixed item pool. QD-Rec maintains an embedding-based archive: an LLM first assesses novelty, and a non-novel entry then competes against its nearest neighbor in a blind pairwise duel. Linear Thompson sampling selects items, while exemplars sampled from the archive guide hypothesis generation through mutation, crossover, or exploration. The authors evaluate the approach on Goodreads, M&A target discovery, synthetic-customer marketing, and paper-abstract exploration, using two proposer models. Comparisons include ablations and feedback-free MMR/DPP rerankers on Goodreads. Overall, QD-Rec improves mean quality, archive size, and coverage in most LLM-scored settings.
Critical strengths
1.Recommendation is usually treated as ranking a fixed set of items; this paper highlights that the space of reasons for recommending them remains open-ended and connects it to quality–diversity and novelty search.
2.The algorithm is simple and interpretable.
3.The experiments cover four different task types and produces quality and coverage gains.
Critical weaknesses
-
The evaluation is circular.*The bandit, replacement decisions, and final quality metric all rely on the same GPT-5.6-Terra evaluator, so the method may optimize for the evaluator’s preferences rather than independently assessed human quality. Apart from Goodreads, there is no independent human evaluation or evaluation by a model uninvolved in the optimization process to validate the practical quality of the proposed method in M&A, marketing, and paper recommendation. The authors should provide evidence of the alignment between human evaluations and LLM-as-a-judge evaluations.
-
The MMR/DPP comparison is not a controlled ablation of item selection. The QD methods receive up to 1,000 instances of real feedback and update the bandit, whereas the baselines receive no feedback. Although the authors also argue that online learning and archive machinery justify their computational cost, this claim should still be treated with caution. Additional baselines should include a quality-only bandit, randomized feedback, a fixed item order, and methods with matched feedback budgets.
-
Archive size may reward novel but otherwise unremarkable hypotheses. Admission is not governed by an explicit distance threshold, and new entries can accumulate within already-covered regions. Therefore, a larger archive does not necessarily imply more useful diversity. Deduplicated semantic or behavioral coverage, human-assessed utility, and quality–diversity Pareto frontiers would provide more informative measures.
-
The baseline comparisons appear insufficient. Beyond MMR/DPP on Goodreads, the evaluation should include a broader range of baseline models to better establish the effectiveness of the proposed method. The two ablations do not fully isolate the respective contributions of the bandit, exemplars, and LLM-based novelty assessment.
-
The M&A and marketing applications raise concerns about factual errors, discriminatory targeting, and unfair competition. The ethics statement mentions human verification but does not analyze how incorrect hypotheses might affect acquisition screening or how recommendation diversity might affect fairness toward different demographic groups.
Questions for authors
See Weekness