- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MirrorRefine is an inference-time multi-agent framework for mirror-reflection generation and completion with frozen generation models. It combines Source–Mirror–Reflection reasoning to infer reflected content with an offline-trained Reflection Spatial Geometry (RSG) module that predicts coarse reflection locations. A VLM-based evaluator diagnoses geometry, appearance, and illumination errors, selects synthesis actions, and verifies candidates within a budget of 8 synthesis attempts. Experiments show complementary contributions from reflection context and RSG. The online configuration leads the reported GLAM-Bench scores, while the local configuration achieves the highest PSNR on MSD and CC-Mirror; both configurations trail PhysReflect in PSNR and IoU on MirrorBenchV2.
Critical strengths
-
Complementary contributions of reflection reasoning and spatial guidance MirrorRefine uses Source–Mirror–Reflection reasoning to infer reflected content and RSG to predict reflection locations. In Table 3, removing reflection context causes larger drops in appearance and lighting scores than removing RSG, whereas removing RSG causes larger drops in geometry scores, IoU, and BF1. This ablation supports the use of both components to address different aspects of reflection consistency.
-
Completion quality on real-image datasets With frozen generation models, MirrorRefine-local achieves the highest PSNR among the compared methods on MSD and CC-Mirror, and the best SSIM and LPIPS on CC-Mirror. These gains in reference-based metrics support improved completion quality on real images, independently of the VLM-based evaluation.
Critical weaknesses
-
Limited evidence of advantages over mirror-specific methods Table 1 highlights scores within groups, placing MirrorRefine alongside general-purpose inference-time refinement methods. This presentation does not emphasize that PhysReflect outperforms both MirrorRefine configurations on PSNR, LPIPS, IoU, and BF1 on MirrorBenchV2. MirrorRefine itself uses reflection-trained RSG and Source–Mirror–Reflection reasoning and is evaluated in single-planar-mirror settings, so comparisons with mirror-specific methods are particularly relevant. Its advantages over these methods in Table 1 lie mainly in GLAM-Bench scores. However, GLAM-Bench uses GPT-5.6 Sol, also the online configuration’s controller and evaluator, which may bias the evaluation in favor of that configuration. Its geometry scores favor MirrorRefine-online over PhysReflect, contrary to the IoU and BF1 rankings. The user study includes only five participants and reports neither agreement among participants nor confidence intervals, limiting the strength of this evidence. Computational costs are also not reported, as discussed in Weakness 2. PhysReflect performs better on reference-image and annotation-based geometry metrics on MirrorBenchV2, while the evidence favoring MirrorRefine relies mainly on a potentially biased VLM evaluation and a small user study without uncertainty estimates. These results do not provide a strong reason to choose MirrorRefine over PhysReflect for this completion task.
-
Insufficient isolation of the agent loop’s contribution and cost-effectiveness Appendix Table 5 compares MirrorRefine-local with its standalone inpainting tool, FLUX.1-Fill-dev, only on selected geometry metrics, with limited and inconsistent gains. PSNR, LPIPS, and GLAM-Bench comparisons are missing. Table 3 supports the contributions of reflection context and RSG, but all variants retain the iterative loop and use FLUX.1-Fill-dev. Even “Without both” retains action selection and candidate retention. These ablations therefore do not isolate the full loop’s benefit over a single inpainting call. Table 1 also varies backbones, synthesis budgets, and candidate-selection procedures. A controlled comparison using the same generation model and matched computational budgets is needed against simpler alternatives: generating multiple FLUX.1-Fill-dev candidates and selecting one with a VLM, or repeatedly regenerating images with prompts revised from VLM feedback. Neither comparison is provided. The synthesis budget counts only image-generation attempts. VLM call counts, token usage, and execution time for the controller and evaluator are not reported. The reflection-specific agent loop is the paper’s central contribution, but comparisons with the standalone tool are limited, and controlled comparisons with simpler alternatives at the same cost are missing. Therefore it remains unclear whether the added complexity of the proposed framework provides meaningful benefits.
Questions for authors
A. Questions that could materially affect the assessment
- (Weakness 2) Could you report the results of a single standalone execution of FLUX.1-Fill-dev on the main metrics in Table 1, including PSNR, SSIM, LPIPS, IoU, BF1, and GLAM-Bench?
- (Weakness 2) Using the same generation model (FLUX.1-Fill-dev) and the same total computational budget, could you provide comparisons against the following simpler alternatives? (e.g. Generating multiple candidates and using a VLM to select one / Having a VLM identify corrections without the proposed agent structure, incorporating its feedback into the prompt, and repeating generation)
- (Weaknesses 1 and 2) Could you report the number of VLM calls made by the controller and evaluator, token usage, and execution time per sample? Comparisons with mirror-specific methods such as PhysReflect and other inference-time methods would be helpful.
- (Weakness 1) Could you provide GLAM-Bench results rescored by a model that is not used in any MirrorRefine configuration?
- (Weakness 1) Could you provide evidence supporting the reliability of the user-study results, such as agreement among participants (inter-rater agreement) and confidence intervals? Could you also clarify how participants were recruited, including their relationship to the authors, without compromising review anonymity?
B. Clarification requests
- Are the controller’s Analysis Controller, Reflection Content Inference, and Prompt Rewriter implemented as separate VLM calls or combined into a single call? Are they executed again at every round? Could you provide the exact prompts and example outputs for each stage?
- What algorithm is used to construct M^{err} and M^{target} in practice?
- Which generation models are used for the TIR, T2I-Copilot, RAISE, and Noise Scaling baselines?
- How are reflected-object masks extracted from generated images when computing IoU and BF1?
10 comments
## W1 / A4 / A5-2: Advantages over mirror-specific methods and evaluation reliability
Human evaluation reliability (A5). We conducted a separate blinded, geometry-focused pairwise study with 15 participants and 200 image pairs (3,000 valid judgments), comparing MirrorRefine-online against PhysReflect.
| Subset | Cases | Online preference | 95% CI | Pairwise agreement |
|---|---|---|---|---|
| All | 200 | 68.8% | 62.40–75.20% | 74.49% |
| Real (MSD + CC-Mirror) | 100 | 83.4% | 80.85–86.15% | 82.90% |
The study includes 50 single-target, 50 multi-target, 50 MSD, and 50 CC-Mirror cases. Participants assessed reflected-object position, scale, orientation, occlusion, and source–reflection correspondence. They were recruited through university advertisements, excluding authors, their students, and research collaborators. Method identities were hidden, presentation order randomized, and GT reflections withheld. All participants completed the full evaluation without missing or invalid responses.
Confidence intervals use 20,000 category-stratified two-way bootstrap resamples of participants and images (Owen, 2007). Pairwise agreement measures the proportion of participant pairs selecting the same method for each image.
The original six-method study separately involved five participants evaluating 200 cases across three perceptual criteria, yielding 3,000 valid selections.
Metric discrepancy. As illustrated in Appendix B.3, IoU/BF1 measure reference-object alignment, whereas GLAM-GEO assesses broader spatial coherence and may overlook object displacement. Closer reflected-object alignment can coexist with inconsistencies in surrounding scene structure; these metrics therefore capture different aspects of reflection quality and are interpreted separately.
We clarify PhysReflect's cross-group advantages in Table 1 and correct Appendix Table 5, where some entries inadvertently retained single-object results during appendix assembly. The expanded table also includes GPT Image 2 and MirrorRefine-online.
W2 / A1 / A2 / A3-1: Agent-loop contribution and computational cost
We provide complete standalone Fill results, controlled same-backend comparisons against simpler alternatives, and measured computational costs, demonstrating quality and efficiency benefits of the integrated reflection-guided refinement procedure.
Standalone and controlled comparisons (A1–A2). All experiments use the same 500 MirrorBenchV2 samples. Independent best-of-N generates Fill candidates and selects the highest-scoring output using the Qwen3.8-27B evaluator (Geometry/Lighting/Appearance weights: 40%/30%/30%). The same weighted score is also used in the stopping procedure of MirrorRefine-local and the matched iterative controls. Candidate retention follows the verification rule described in Appendix A.3. Fixed-mask feedback performs iterative VLM critique, prompt revision, and full-mirror inpainting, retaining the same reflection context, RSG, candidate verification, retention, and stopping conditions as all-Fill. The same-backend controls share FLUX.1-Fill-dev and an eight-attempt synthesis cap.
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ | CLIP ↑ | IoU ↑ | BF1 ↑ | Geo ↑ | Light ↑ | App ↑ | Attempts |
|---|---|---|---|---|---|---|---|---|---|---|
| Standalone Fill | 13.4465 | 0.7487 | 0.2021 | 26.6650 | 0.4024 | 0.1468 | 56.55 | 69.78 | 68.83 | 1.00 |
| Fill best-of-8 | 16.0304 | 0.7482 | 0.1713 | 26.0434 | 0.4125 | 0.1607 | 68.65 | 71.16 | 71.65 | 8.00 |
| Fill best-of-4 | 15.8014 | 0.7447 | 0.1726 | 25.8167 | 0.4098 | 0.1598 | 68.33 | 70.87 | 71.41 | 4.00 |
| Fixed-mask feedback | 16.2121 | 0.7489 | 0.1688 | 25.9033 | 0.4137 | 0.1610 | 68.77 | 71.42 | 72.05 | 6.34 |
| MirrorRefine all-Fill | 16.2365 | 0.7508 | 0.1632 | 26.1810 | 0.4269 | 0.1621 | 69.08 | 71.83 | 72.30 | 4.24 |
| Full MirrorRefine-local | 16.5389 | 0.7538 | 0.1641 | 26.1756 | 0.4273 | 0.1673 | 69.54 | 72.73 | 75.66 | 4.06 |
All-Fill improves eight of nine metrics over standalone Fill, increasing PSNR by 2.79 dB, IoU by 0.0245, BF1 by 0.0153, and Geometry by 12.53 points, while reducing LPIPS by 0.0389. Standalone Fill retains higher CLIPScore.
All-Fill also improves all nine metrics over best-of-8 and fixed-mask feedback. Compared with best-of-8, IoU increases by 0.0144 and LPIPS decreases by 0.0081, using 47.0% fewer synthesis attempts. Relative to fixed-mask feedback, IoU increases by 0.0132, while attempts decrease from 6.34 to 4.24.
These comparisons support the integrated iterative procedure over independent selection and adaptive repair localization over fixed-mask feedback. Full additionally uses FLUX.1-Kontext-dev for targeted editing; all-Fill provides the strictly same-backend comparison.
## W1 / A4 / A5-1: Advantages over mirror-specific methods and evaluation reliability
PhysReflect's stronger reference-based geometry results on MirrorBenchV2 are obtained with mirror-task training on SynMirrorV2, whose held-out scenes form MirrorBenchV2, and rendered ground-truth depth outside the mirror as an additional generation input. In contrast, MirrorRefine performs training-free refinement using frozen pretrained synthesis backbones with a single mirror-erased RGB image and mirror mask, without depth input or mirror-task fine-tuning of the synthesis backbones.
Training and input differences. MirrorRefine's RSG is trained separately on Mirror-Reflect-Inference and remains frozen during refinement. We additionally retrained PhysReflect and MirrorVerse without depth inputs, retaining their architectures and original training settings.
PhysReflect's IoU/BF1 decrease from 0.6932/0.4228 to 0.4611/0.1393 without depth, narrowing its IoU advantage. Both MirrorRefine configurations achieve higher BF1 and GLAM-Geometry than the depth-free baselines, while PhysReflect retains higher IoU. These results suggest that access to additional rendered ground-truth depth contributes substantially to the stronger MirrorBenchV2 geometry performance of depth-conditioned mirror-specific methods. Such accurate depth is difficult to obtain in real-world scenes, where MirrorRefine achieves better reflected-object IoU/BF1 than the evaluated specialized baselines.
Objective geometry on real images. We additionally evaluated reflected-object IoU and BF1@2px on MSD and CC-Mirror using the same evaluation procedure across methods (B4).
Online achieves the highest IoU and BF1@2px among these methods on both real-image datasets, including against PhysReflect fine-tuned on MSD. These measurements provide reference-based geometry evidence beyond VLM evaluation.
Independent VLM evaluation (A4). We rescored the same 500 MirrorBenchV2 final outputs using DeepSeek-V4.1-Flash-1017 and GPT-6.1 Sol, neither of which is used in MirrorRefine's configurations.
Both independent judges rank Online above PhysReflect and the Qwen-controller hybrid across all three dimensions, with Geometry margins of 6.35 and 4.16 points over PhysReflect. This reduces the concern that the reported GLAM advantage is specific to the original GPT-5.6 Sol judge.
For the original real-image GLAM evaluation, the valid/unscored counts per method and dimension are 200/0 on MSD and 199/1 on CC-Mirror. The single unscored CC-Mirror case resulted from a content-policy refusal and was excluded from the common comparison set.