Explore Widely, Exploit Wisely: Harnessing Text–Image Cost Asymmetry for Unified Multimodal Reinforcement Learning

Prompt Expansion (PE) enhances text-to-image generation by transforming user prompts into explicit, visually grounded descriptions, but jointly optimizing prompt rewriting and image generation remains computationally expensive when large candidate pools are explored. We observe a pronounced cost asymmetry between text and image generation: expanding the number of PE candidates incurs relatively modest decoding overhead, whereas image generation scales approximately linearly with the number of evaluated candidates. Motivated by this asymmetry, we propose **PEAR (Prompt Exploration for Adaptive Joint Reinforcement Learning)**, a framework that enlarges the textual exploration space while selectively allocating image-side evaluation budget. PEAR first oversamples PE candidates for each prompt and estimates their relative rewards using low-cost proxy image generation with reduced precision and fewer denoising steps. It then measures the utility of each prompt by the reward separation between selected top-\(k\) and bottom-\(k\) candidates, and uses bootstrap-based expected utility gain together with a variance-based noise discount to dynamically allocate the remaining evaluation budget across prompts. After the proxy budget is exhausted, the selected high- and low-reward PE candidates are regenerated under the full generation configuration and used to jointly optimize the text and image policies. Experiments show that PEAR evaluates only 62.5% of the full candidate pool, reducing proxy screening time by 35.1% and end-to-end training-step time by 21.1% compared with exhaustive PE screening. Meanwhile, it improves the average score across WISE, HPSv2, GenEvalv2 VQAScore, and PickScore from 0.60145 to 0.60985, and outperforms a static allocation baseline under the same proxy evaluation budget. These results demonstrate that adaptive candidate evaluation can substantially reduce PE screening cost while preserving, and potentially improving, generation quality.