acceptodds
Withdrawn from ICLR 2027

RISA: Recursive Improvement for Scene-Consistent Visual Anomaly Synthesis

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Visual anomaly synthesis provides a practical way to supplement scarce real-world anomaly data for industrial inspection and safety monitoring. Although instruction-guided image editing enables controllable anomaly generation from natural-language descriptions, existing methods typically rely on one-shot generation or loosely constrained iterative refinement, making it difficult to correct synthesis errors without disturbing the spatial structure and unaffected content of the original scene. We introduce RISA, a multimodal agent framework for Recursive Improvement for Scene-Consistent Anomaly Synthesis. RISA formulates anomaly synthesis as a progressive process of observation, diagnosis, and repair. At each iteration, the agent evaluates the synthesized image at both global and local scales, identifies discrepancies in anomaly appearance, scale, placement, and integration with the surrounding scene, and adaptively revises its editing strategy based on feedback from previous attempts. To maintain scene consistency throughout refinement, RISA combines multimodal scene understanding with spatially constrained image manipulation, including cropping, localized editing, and compositing, enabling the agent to jointly determine what anomaly to generate, where to place it, and how to integrate it into the original scene while minimizing unintended changes. This recursive inference-time improvement process requires no parameter updates to the underlying models. Experiments across industrial inspection and safety monitoring scenarios demonstrate that RISA consistently improves anomaly fidelity while better preserving the structural and visual consistency of the original scene.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Anomaly Synthesis, Recursive Improvement

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_cbihOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents RISA, a training-free multimodal agent for synthesizing specified visual anomalies while preserving unrelated content in the source image. The agent first plans a target region, an editable region, and a context crop. It then applies localized image editing and composites the result against the original image. After each attempt, a visual evaluator scores faithfulness, locality, realism, and mask alignment and provides a textual diagnosis. RISA can respond by regenerating content, recompositing a previous candidate, stopping early, or selecting an earlier candidate rather than the last one.

The paper also introduces SQAE-300, a set of 300 anomaly-editing tasks derived from 66 source images across industrial inspection and safety-monitoring settings. RISA is compared with direct editing, Best-of-K, and a tool-augmented agent without recursive repair. On the authors' metrics, it improves faithfulness and realism while preserving high locality, and the gains are generally consistent across nine combinations of reasoning models and agent harnesses. A downstream experiment trains a U-Net on the synthetic images and masks, where RISA gives the best reported image-level and pixel-level AUROC.

Critical strengths

1.The spatial execution design fits the problem well. RISA separates the target, editable, and context regions, and every candidate is composited against the original image. This gives the method a concrete way to preserve unrelated content instead of asking the image editor to preserve it implicitly. The large improvements in locality and mask alignment over direct editing are consistent with this design.

2.Retaining the full candidate history is useful. RISA can reuse content from an earlier candidate, tighten its composition, and still select that earlier result if later generations get worse. The toothbrush trajectory in the supplement makes this behavior easy to inspect. This is a meaningful advantage over one-shot editing and naive schemes that always return the final iterate.

Critical weaknesses

1.The main evaluation is not independent of the signal used for refinement. GPT-5.5 provides the diagnoses that guide RISA and also produces the four headline final scores. This creates a risk that RISA is learning to satisfy one judge within each task rather than improving human-perceived fidelity or realism. P-SSIM is not fully independent either, because its pass-rate multiplier uses thresholds on the GPT-5.5 faithfulness, realism, and mask-alignment scores. Local IS does not measure instruction correctness. Without a blinded human study or held-out evaluators from different model families, I do not think the central quality claim is adequately established.

2.The empirical scope and novelty comparison are not yet strong enough for the broader claims. SQAE-300 contains 300 tasks but only 66 source images, yet the paper reports no source-clustered confidence intervals or repeated-run variance. The downstream result uses one training seed and an unspecified selected subset of the MVTec AD test split. The paper also does not compare against MIRA, RefineEdit-Agent, or EditRefiner, even though these systems already use iterative multimodal feedback and localized image repair. The current evidence shows that RISA beats its own ablated pipeline, but it does not establish a clear advantage over the closest existing approaches.

Questions for authors
  1. Is the GPT-5.5 evaluator used for Table 1 identical to the evaluator used online during recursive repair, including the prompt, rubric, and model snapshot? Are the reported scores stored online scores or fresh held-out evaluations? Please explain what prevents within-task overfitting to this judge.

  2. Do you have blinded human ratings or results from held-out evaluators from different model families for faithfulness and realism? Please report the protocol, sample size, agreement, and paired uncertainty. If the gains remain under independent evaluation, this would materially improve my assessment.

  3. Which MVTec AD categories and test images are used in Table 2, how many test samples are included, and how is disjointness from the 30 normal source images used in SQAE-300 guaranteed? Please provide results over several training seeds if available.

  4. What is the substantive difference between RISA and MIRA, RefineEdit-Agent, and EditRefiner beyond the anomaly domain and tool implementation? If a direct comparison is infeasible, a capability-level comparison of localized diagnosis, iterative repair, and candidate reuse would help determine the contribution.

Reviewer_eSK3OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RISA, a multimodal agent framework for scene-consistent visual anomaly synthesis. It combines scene-grounded planning, spatially constrained editing, and feedback-driven recursive refinement to improve anomaly generation while preserving unrelated image content. The authors introduce SQAE-300, a benchmark containing 300 editing tasks across industrial inspection and safety monitoring. Experiments demonstrate improvements over direct editing, Best-of-K, and a tool-augmented agent baseline. The authors also evaluate the utility of synthesized anomalies through downstream anomaly detection and segmentation on MVTec AD.

Critical strengths
  1. The paper presents a coherent agent-based framework that integrates scene understanding, spatially constrained editing, and iterative refinement. The explicit spatial constraints and candidate reuse mechanism are practically useful for preserving non-target image content.

  2. The experiments cover multiple reasoning models and agent harnesses, demonstrating consistent improvements in faithfulness and realism over the tool-augmented baseline. The inclusion of downstream anomaly segmentation experiments is also valuable for assessing the practical utility of synthesized anomalies.

Critical weaknesses
  1. The reported P-SSIM values appear inconsistent with the computation described in Appendix B. Specifically, P-SSIM is defined as 100 × ρ × s, where ρ is the quality-gate pass rate and s is the mean protected-region SSIM. Since s ≤ 1, the reported P-SSIM of 59.54 for Direct Editing implies a pass rate of at least ρ ≥ 0.5954. Furthermore, the quality gate requires each passing sample to have a mask-alignment score M ≥ 0.55. Therefore, even assuming that all failing samples have M = 0, the average mask-alignment score must be at least 0.55 × 0.5954 = 0.32747, corresponding to 32.75 on the 0–100 scale. However, Table 1 reports an average M of only 9.84 for Direct Editing. Assuming these metrics are computed over the same samples and scoring records, these values appear mathematically inconsistent. This raises concerns about the correctness or clarity of the evaluation protocol, and the authors should clarify whether different sample sets or scoring procedures were used. Furthermore, since P-SSIM incorporates a quality-gate pass rate rather than measuring structural similarity independently, its reported improvements are difficult to interpret without separately reporting the original SSIM and pass rate.

  2. Unlike general-purpose image editing, anomaly synthesis is particularly valuable for improving downstream anomaly detection and segmentation. However, the paper primarily compares general editing strategies without evaluating representative anomaly-specific synthesis methods, such as AnoGen (ECCV 2024) and AnomalyDiffusion (AAAI 2024). Although downstream AUROC results are reported, the limited experimental protocol and lack of per-category evaluations make it difficult to establish the competitiveness and robustness of the method. Moreover, the construction and semantic accuracy of the generated anomaly masks remain unclear. The reported Mask Alignment evaluates consistency with image changes rather than the actual anomalous regions, which is particularly important for pixel-level anomaly segmentation.

Questions for authors
  1. Could the authors clarify the apparent inconsistency between the P-SSIM computation in Appendix B and the results in Table 1? Specifically, given the quality-gate requirement of M ≥ 0.55, a P-SSIM of 59.54 for Direct Editing would imply an average mask-alignment score of at least 32.75, assuming the same evaluation samples and scoring records. However, Table 1 reports only 9.84. Could the authors explain this discrepancy and report the original SSIM and quality-gate pass rate separately?

  2. Could the authors provide comparisons with representative anomaly synthesis methods, such as AnoGen (ECCV 2024) and AnomalyDiffusion (AAAI 2024), under a comparable downstream anomaly detection and segmentation protocol? Since the primary motivation of anomaly synthesis is to benefit downstream tasks, such comparisons are important for establishing the practical advantages of RISA. In addition, could the authors provide per-category results and multi-seed evaluations to demonstrate the robustness of the reported AUROC improvements?

  3. Could the authors clarify how the output anomaly masks are generated for each compared method? The current Mask Alignment metric measures consistency between the generated masks and image changes, rather than the semantic accuracy of anomalous regions (e.g., cracks or damaged components within an edited region). How do the authors ensure that the generated masks accurately represent the actual anomalies rather than the entire modified area? A direct evaluation of anomaly-mask accuracy would help clarify whether RISA provides more reliable supervision for downstream segmentation.

Reviewer_dBELOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The goal of this paper is to generate visual anomaly data for industrial inspection and safety monitoring. A multimodal agent framework, i.e., RISA, is proposed for scene-consistent anomaly synthesis via a recursive improvement process of observation, diagnosis, and repair. Experimental results show the effectiveness of the proposed method.

Critical strengths

The idea of considering anomaly synthesis as a progressive process of observation, diagnosis, and repair, is reasonable without parameter updates to the underlying models.

Critical weaknesses
  1. The technical contribution is relatively limited. The paradigm of iterative refinement via multimodal agent feedback has already been widely explored and adopted across various tasks. This paper directly applies a similar agentic loop to visual anomaly synthesis without introducing new insights.
  2. The related work section only discusses general image generation, editing, and recursive self-improvement. A thorough discussion of existing visual anomaly detection and synthesis methods is necessary.
  3. The current evaluations lack comparisons against existing state-of-the-art visual anomaly synthesis methods. In addition, the results are potentially biased, as both the proposed recursive refinement loop and the evaluation metrics rely on GPT-5.5 using similar criteria.
  4. For downstream anomaly segmentation evaluation, training a single standard U-Net is insufficient to validate the usefulness of the generated data. A more robust evaluation across diverse anomaly segmentation architectures using the generated data is necessary.
  5. As the proposed method relies on iterative refinement, it would be better to conduct computational cost, token overhead, and latency analysis.
Questions for authors
  1. How to make sure that the reported performance gains are not merely an artifact of evaluator bias?
  2. How about the comparison of the proposed method against existing anomaly synthesis methods in terms of visual quality and downstream segmentation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.