It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning

Recent work increasingly trains large language models (LLMs) for retrieval. Yet existing methods predominantly optimize generation solely on the query side, leaving item representations fixed. This one-sided adaptation restricts mutual alignment and caps overall retrieval performance. We introduce , a co-evolving reinforcement learning framework that jointly trains both query-side and item-side LLM generators. Each generator produces a compact set of discrete keywords for retrieval, preserving full compatibility with existing keyword-based infrastructure. first employs supervised fine-tuning to establish a warm-started, aligned keyword space. It then applies co-evolving reinforcement learning, alternately optimizing each generator against the opposite side’s frozen index. Both sides optimize the same query-to-item $F_1$ objective: the query side receives $F_1$ directly, while the item side receives a counterfactual marginal reward designed to isolate each item's impact on retrieval performance. Evaluated against 10 representative sparse, dense, and generative retrieval baselines, achieves the strongest overall retrieval performance on both an internal APP Marketplace dataset and the public WANDS benchmark, improving $F_1$ over the strongest baseline by $10.9\%$ and $36.1\%$, respectively. Further analysis demonstrates stable co-evolution dynamics and the emergence of more specific, balanced keyword representations.