- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents RISA, a training-free multimodal agent for synthesizing specified visual anomalies while preserving unrelated content in the source image. The agent first plans a target region, an editable region, and a context crop. It then applies localized image editing and composites the result against the original image. After each attempt, a visual evaluator scores faithfulness, locality, realism, and mask alignment and provides a textual diagnosis. RISA can respond by regenerating content, recompositing a previous candidate, stopping early, or selecting an earlier candidate rather than the last one.
The paper also introduces SQAE-300, a set of 300 anomaly-editing tasks derived from 66 source images across industrial inspection and safety-monitoring settings. RISA is compared with direct editing, Best-of-K, and a tool-augmented agent without recursive repair. On the authors' metrics, it improves faithfulness and realism while preserving high locality, and the gains are generally consistent across nine combinations of reasoning models and agent harnesses. A downstream experiment trains a U-Net on the synthetic images and masks, where RISA gives the best reported image-level and pixel-level AUROC.
Critical strengths
1.The spatial execution design fits the problem well. RISA separates the target, editable, and context regions, and every candidate is composited against the original image. This gives the method a concrete way to preserve unrelated content instead of asking the image editor to preserve it implicitly. The large improvements in locality and mask alignment over direct editing are consistent with this design.
2.Retaining the full candidate history is useful. RISA can reuse content from an earlier candidate, tighten its composition, and still select that earlier result if later generations get worse. The toothbrush trajectory in the supplement makes this behavior easy to inspect. This is a meaningful advantage over one-shot editing and naive schemes that always return the final iterate.
Critical weaknesses
1.The main evaluation is not independent of the signal used for refinement. GPT-5.5 provides the diagnoses that guide RISA and also produces the four headline final scores. This creates a risk that RISA is learning to satisfy one judge within each task rather than improving human-perceived fidelity or realism. P-SSIM is not fully independent either, because its pass-rate multiplier uses thresholds on the GPT-5.5 faithfulness, realism, and mask-alignment scores. Local IS does not measure instruction correctness. Without a blinded human study or held-out evaluators from different model families, I do not think the central quality claim is adequately established.
2.The empirical scope and novelty comparison are not yet strong enough for the broader claims. SQAE-300 contains 300 tasks but only 66 source images, yet the paper reports no source-clustered confidence intervals or repeated-run variance. The downstream result uses one training seed and an unspecified selected subset of the MVTec AD test split. The paper also does not compare against MIRA, RefineEdit-Agent, or EditRefiner, even though these systems already use iterative multimodal feedback and localized image repair. The current evidence shows that RISA beats its own ablated pipeline, but it does not establish a clear advantage over the closest existing approaches.
Questions for authors
-
Is the GPT-5.5 evaluator used for Table 1 identical to the evaluator used online during recursive repair, including the prompt, rubric, and model snapshot? Are the reported scores stored online scores or fresh held-out evaluations? Please explain what prevents within-task overfitting to this judge.
-
Do you have blinded human ratings or results from held-out evaluators from different model families for faithfulness and realism? Please report the protocol, sample size, agreement, and paired uncertainty. If the gains remain under independent evaluation, this would materially improve my assessment.
-
Which MVTec AD categories and test images are used in Table 2, how many test samples are included, and how is disjointness from the 30 normal source images used in SQAE-300 guaranteed? Please provide results over several training seeds if available.
-
What is the substantive difference between RISA and MIRA, RefineEdit-Agent, and EditRefiner beyond the anomaly domain and tool implementation? If a direct comparison is infeasible, a capability-level comparison of localized diagnosis, iterative repair, and candidate reuse would help determine the contribution.