Personalized Query Enhancement via Reward-Guided Feedback Descent

Users in real-world interactive systems often submit vague queries that only partially reflect their latent intent. Query enhancement aims to make such queries more informative by adding context that better captures the user’s underlying needs. However, a key challenge is that enhancement quality is difficult to measure: direct satisfaction feedback is sparse, while abundant indirect signals are only noisy proxies. In this paper, we study personalized query enhancement under sparse feedback. We formulate user satisfaction through a hierarchical model that links sparse direct satisfaction signals with denser indirect behavioral feedback, yielding a reward model for evaluating query enhancements. Using this reward model, we define objectives for generating and refining personalized enhancements. To iteratively improve the enhancements, we propose reward-guided feedback descent, a novel text-feedback-based query refinement algorithm. A key novelty of our approach is that it uses the reward model and the enhancement objective as a grounding mechanism for determining the preferred candidate numerically and synthesizing textual feedback, which in turn guides the LLM enhancer toward generating higher-quality enhancements. We evaluate our method on both public benchmarks and real-world e-commerce data, demonstrating that the proposed algorithm consistently improves user satisfaction through enhanced queries.