- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies a practical problem that image generators struggle with new entities that didn't appear in the training data and might appear alongside other entities because they are relatively new. The paper contributes a benchmark named Savanna of 500 prompts across 16 subcategories. These prompts are challenging and directly retrieving evidence from the internet is likely to fail to retrieve the correct ones to facilitate the image generator. The authors thus propose a paradigm named Locate-Corroborate-Isolate, where agents verify the retrieved evidence with multiple sources, then crop the reference images for use by image generators. The paper also shows that training models on the trajectories following this paradigm can outperform the harnesses with frontier models and yield the best generation quality.
Critical strengths
S1. This paper identifies a challenging and practical problem of using agents to retrieve new entities for image generators. The challenges in this setup are also summarized into three aspects and lead to a solid benchmark where frontier models also struggle to retrieve correct evidence in this setup.
S2. This paper contributes a benchmark and a paradigm to improve agent performance in evidence retrieval and follow-up improvement in image generation with well-retrieved evidence. The methods are well-motivated and the performance improvement is effective, outperforming a wide range of baselines and generalizing well.
Critical weaknesses
W1. The paradigm is well-motivated and its effectiveness is validated via extensive experiments. The effects of different components or stages might need further exploration, for example, how performance varies if applied without cropping or without verification, although the additional computational costs of such ablation studies may be a consideration.
W2. The Corroborate stage appears to be a potential bottleneck in the proposed bottleneck. The current paradigm uses cross-modal verification and cross-image agreement, which might not be reliable in the proposed setup. The example in Figure 4 also presents a subtle entity, which might be challenging for VLMs to correctly identify as the desired evidence. Is there any measure to control or examine the quality at this stage?
Questions for authors
Overall I think the strengths outweigh the weaknesses and the major concerns are listed in the weaknesses. Minor question: the paper claims that the generation is bounded by reference quality, and it might be interesting to visualize the correlation between reference quality and generation quality for the same generator, and it will better support this claim.
3 comments
Thank you for your response; my confusion has been resolved, and I will maintain my positive rating.
Glad to hear our response resolved your questions! Thanks again for taking the time to engage with our rebuttal and for your support of our work.
Response to Reviewer YERL
Thank you for your feedback and valuable comments. Below is our response to your concerns.
W1 [How much does each LCI stage exactly contribute?]
(1) Isolate cropping ablation (Sec. 4.4, Fig. 7): On the 102 prompts where Ranger-8B calls
crop_image, passing the cropped target region instead of the full retrieved image raises target identity scores by +6.3 points (28.1→34.4) on Qwen-Image-Edit-2511 and +3.8 points (47.8→51.6) on Nano Banana Pro.(2) Controlled 5K SFT stage ablations (Sec. 4.5, Tab. 3): We randomly sample 5,000 trajectories from Ranger-15K and remove the corresponding steps while keeping the original trajectory's final result using Gemini-3.1-Pro (w/o Locate searches images directly without prior text search; w/o Corroborate directly selects the target image without verification steps). We use these trajectories to fine-tune Qwen3-VL-8B (see the table below; Sec. 4.5, Tab. 3):
W2 [How reliably does Corroborate select the right image from noisy candidates?]
We check all candidate images returned by
image_searchon 100 sampled prompts to measure how often the agent picks the right reference once a correct target image appears in the search results (see the table below; App. M, Tab. 12). Ranger-8B correctly selects 67 out of 74 candidates (90.5%), while Gen-Searcher with the same backbone only selects 39 of 76 candidates (51.3%). This confirms that Corroborate helps the model select the target image more accurately.Minor Question [How strongly does retrieved reference quality affect final image generation?]
We show this in three ways in our paper:
(1) With vs. without Isolate on the same retrieved image (Sec. 4.4, Fig. 7): Fixing the generator and the retrieved source image on the 102 cropped prompts, cropping out surrounding distractors raises target identity scores by +6.3 points (28.1→34.4) on Qwen-Image-Edit-2511 and +3.8 points (47.8→51.6) on Nano Banana Pro.
(2) Same generator with references from different methods (Sec. 4.2, Tab. 1): Fixing Nano Banana Pro, generation scores increase consistently with reference Search scores from 26.1 (built-in search) and 31.6 (Unify-Agent, 23.2 Search) to 47.3 (Gen-Searcher, 43.5 Search) and 58.9 (Ranger-8B, 67.2 Search). Generation scores on Qwen-Image-Edit-2511 follow the same trend, rising from 24.9 to 46.8.
(3) Visual comparisons in the Appendix (App. J, Fig. 9 & App. L, Fig. 11): We also provide side-by-side visual results for the same generator in the Appendix. App. J (Fig. 9) shows Nano Banana Pro outputs using references retrieved by different methods across all five domains, and App. L (Fig. 11) shows paired outputs generated from full vs. cropped references.
We hope this addresses your concerns and are happy to discuss further.