acceptodds
Under review as a conference paper at ICLR 2027

After a Decade: Bringing Shadow Removal into the Real World with Agentic Training Data

Abstract

Shadow removal looks nearly solved on established benchmarks, yet remains brittle in the real world. Models have advanced; the paired training data they rely on have barely changed in nearly a decade. The reason is simple: obtaining a shadow-free target requires removing the occluder while keeping the scene, camera, and illumination otherwise unchanged, making diverse paired data difficult to capture. Meanwhile, large shadow detection datasets already contain diverse real-world images and masks, but no shadow-free targets. To turn this abundant but incomplete data into paired supervision, we propose an offline agentic workflow combining physics-motivated generation, failure detection, feedback-driven retry, candidate selection, and deterministic correction. Using this workflow, we construct AgenticShadow, a dataset of 17,138 image-mask-target triplets spanning general scenes, faces, and remote sensing. Our construction workflow reduces Color Distribution Difference by 50.5% over previous shadow removal work, while training existing shadow removal models on AgenticShadow reduces cross-domain LAB RMSE by 19.7–37.5%.

Primary area:
datasets and benchmarks
Keywords:
Shadow Removal, Agentic Data Construction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3NVcOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces AgenticShadow, a dataset of 17,138 image-mask-target triplets constructed through generative editing, automated evaluation, and regional correction. It studies the resulting supervision across several restoration models and introduces a semantic-and-depth Prior-Fusion baseline with a mask-free variant.

Critical strengths
  1. The dataset expands supervision to diverse general scenes, faces, and remote-sensing imagery.
  2. Improvements across several restoration architectures support the usefulness of the constructed training data.
  3. The appendix provides useful construction statistics, costs, runtime measurements, and failure analyses.
Critical weaknesses
  1. Evaluation relies heavily on generated references, and some benchmark inputs undergo generative preprocessing.
  2. Better boundary-color consistency coincides with larger changes outside shadow regions.
  3. The benefits of adaptive retries and the additional depth branch need stronger controlled comparisons.
Questions for authors
  1. Most gains in Table 4 are measured against pseudo-targets generated by the training-data workflow. Beyond ISTD+, please validate the trained models on independently captured pairs or through blinded human assessment in the new domains.
  2. Appendix B.3 uses GPT Image 2 to normalize benchmark inputs, which may itself change image content. Please evaluate on original images with ordinary resizing and quantify changes introduced by generative normalization.
  3. Despite improved CDD, Appendix B.5 reports non-shadow RGB RMSE of 23.13, versus 3.44–18.39 for previous methods. Please include this preservation issue in the main results and check whether trained models inherit these unwanted edits.
  4. Only 2.07% of images trigger feedback-guided retries (Table 6). Please compare against a fixed multi-candidate pipeline using the same prompts, selector, and correction under a matched editing budget to isolate the benefit of adaptive control.
  5. In Table 3(b), adding depth to semantic priors improves LAB RMSE only from 5.585 to 5.577. Please report variation across training seeds and the additional inference cost to establish whether the depth branch is worthwhile.
4 comments
AuthorsOpenReview ↗

Responses to Reviewer 3NVc’s Questions and Concerns

Thank you for recognizing the broader supervision, improvements across architectures, and construction analysis.

Does normalization change image content, and do gains persist with ordinary resizing? (W1, Q2)

We agree that generative normalization can alter image content and introduce a potential confound. And yes, the gains persist on original images with ordinary resizing.

(1) To quantify the changes, we resize the original images to 512 × 512 and compare the differences:

SourceImagesSSIMLPIPS
SBU4,6090.70620.2037
CUHK10,4220.64290.2547

(2) Testing the trained models on the original testing images give the following improvements:

ModelSourcePSNR gain (dB)SSIM gainLAB reduction
HomoFormerSBU+1.28+0.02211.15
HomoFormerCUHK+2.00+0.04952.11
PFSBU+1.54+0.03451.26
PFCUHK+2.64+0.07152.75

We have added the normalization and original-input analyses in Appendices B.3 and D.3. The human assessment and captured-reference evaluations below provide complementary evidence.

Do gains hold on independently captured pairs and in blinded human assessment? (W1, Q1)

Yes. We validate the gains on independently captured pairs and through blinded human assessment on original inputs.

  1. Independently captured pairs: On all 1,100 held-out WSRD+ pairs, training with ISTD+ and AgenticShadow improves PhaSR’s PSNR by 2.84 dB and reduces LAB RMSE by 30.1% compared with ISTD+-only training. These pairs were captured under controlled indoor, close-range conditions. Since WSRD+ provides no annotated shadow masks, we evaluate frozen mask-free models:

    Model / training dataPSNR ↑SSIM ↑LAB RMSE ↓
    PhaSR / ISTD+18.450.73008.61
    PhaSR / Combined21.290.76106.02
    PF-MF / Combined20.800.75566.39
  2. Blinded human assessment: Combined-trained PF scores 3.11/4 versus 1.89/4 for ISTD+-only PF, an average paired improvement of 1.22 points. We conduct a blinded study on 80 AgenticShadow cases using original inputs, resized for SBU/CUHK. Raters assess removal quality using a four-point scale: 1 (Very poor), 2 (Poor), 3 (Good), and 4 (Very good).

We have added the WSRD+ evaluation in Section 5.4 and the human study in Section 5.5. Generalization to unseen domains and improvements in human-rated removal quality support AgenticShadow as a valuable resource for the shadow removal community.

AuthorsOpenReview ↗

Responses to Reviewer 3NVc’s Questions and Concerns (cont’d)

Non-shadow preservation: do trained models inherit unwanted edits? (W2, Q3)

We acknowledge that non-shadow preservation remains a limitation.

On original AgenticShadow test images, resized for SBU/CUHK, non-shadow input–prediction RGB RMSE increases from 5.93 to 16.58 for HomoFormer and from 5.78 to 18.65 for PF after combined training. On all 540 ISTD+ test images, non-shadow input–prediction RGB RMSE changes from 3.47 to 3.01 for HomoFormer and from 3.29 to 3.59 for PF. These results indicate increased non-shadow changes on AgenticShadow after combined training, while ISTD+ shows little change.

However, we also observe that non-shadow differences occur in the reference targets, even for captured datasets. Mean non-shadow input–target RGB RMSE is 10.83 on SRD and 20.29 on WSRD+, using available SRD masks and difference-based pseudo masks for WSRD+. For workflow-generated targets, the corresponding values are 13.83 on all 540 ISTD+ test images and 27.27 across all 17,138 AgenticShadow pairs. This suggests that preserving non-shadow content during generative construction is more challenging in complex real-world scenes. We have expanded the preservation analysis in Appendix B.5 and summarized the limitation in Section 5.1.

Adaptive control versus fixed multi-candidate generation (W3, Q4)

The 2.07% statistic measures only how often the major-failure retry branch is triggered, while candidate selection and regional correction still apply when the initial probe passes. Fixed multi-candidate generation alone cannot reliably replace corrective feedback, because repeated generation can retain shadow structure or produce different hallucinated content.

For four rejected-probe examples, we compare five and ten independent generations using the original prompt with three feedback-guided candidates per case, keeping inputs, masks, and model settings unchanged:

GenerationCandidates per caseMean CDD (×10³) ↓Best CDD (×10³) ↓
Without feedback10170.01425.384
Without feedback5160.61626.153
With feedback31.1210.307

The visual comparisons show repeated hallucinations or retained shadow structure without feedback, while feedback produces more consistent removal. We have added the numerical and visual comparisons in Appendix B.4, including Figure 13.

Depth prior's incremental gain and inference cost (W3, Q5)

We agree that depth’s incremental gain over semantic-only PF is marginal: LAB RMSE improves from 5.585 to 5.577, and from 5.607 to 5.604 with a second training seed. Depth alone still improves over the HomoFormer backbone from 5.912 to 5.769. We report full PF as the best observed configuration and provide both single-prior variants and their costs for comparison.

Inference costs are measured on RTX A6000 GPUs with 512 × 512 inputs and batch size 1, including online prior extraction where applicable:

ConfigurationAgenticShadow macro LAB RMSE ↓Latency (ms/image) ↓Peak allocated memory (GiB) ↓
HomoFormer5.912505.20.922
Depth-only5.769718.72.186
Semantic-only5.585572.32.073
Full PF5.577956.23.330

We have added the single-prior efficiency measurements in Appendix C.3.

Reviewer_3NVcOpenReview ↗

response

Based on the author's response, I believe most of my concerns have been addressed; therefore, I am inclined to increase my rating.

Thank you for your constructive feedback and positive reassessment of our work. We are glad that our responses have addressed most of your concerns.

Reviewer_cLNROpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a new workflow to convert existing shadow images and masks into paired training data. Specifically, GPT Image 2 is used to restore shadowed areas to their appearance under normal lighting, and GPT-5 mini is then used to verify whether the results meet the standards; if they fail, the image is redrawn based on the feedback, and one image is selected from multiple attempts, followed by a final color correction. This process yields AgenticShadow, comprising a total of 17,138 pairs covering everyday scenes, faces, and remote sensing imagery. The authors then used this dataset to train several existing shadow removal models, as well as a baseline model incorporating semantic and depth priors.

Critical strengths
  1. The dataset is larger than previous paired datasets, and the scenarios extend beyond controlled data collection to include facial images and remote sensing data.
  2. The constructed shadow-free images are close to the corrected ground truth on ISTD+;
  3. After training on this dataset, the cross-domain performance of some existing methods has improved;
Critical weaknesses
  1. Essentially, it still involves having the image editing model generate a shadow-free image, and then having the VLM determine whether the image meets the criteria. Once both the editing model and the VLM have improved, the core of this task will shift to clearly defining the problem using prompts. The difference between physical prompts and regular prompts in the paper is essentially just stating more explicitly that “shadows are caused by light being blocked, not by a change in material”; The improvements gained from subsequent retries and selection are limited. Moreover, the vast majority of images pass on the first try; only a small number actually require re-generation.

  2. The VLM’s evaluation criteria are narrow, focusing only on two types of failures: shadows being mistaken for objects and left in the image, or new content being drawn within the mask.

  3. These shadow-free images were generated by the editing model itself, and during testing, these generated images were used as the ground truth. The cross-domain improvement stems more from the small model approximating this set of generated results rather than approximating actual shadow-free photographs; on ISTD+, which uses real-world photographs as ground truth, the changes are minimal. Therefore, I do not believe this dataset is suitable for fine-tuning image editing models, as that would merely cause the model to relearn its own output; the approach taken in the paper simply has the smaller models imitate the editing model.

Questions for authors

It is unclear whether GPT Image 2 had already been exposed to public datasets such as ISTD, SRD, SBU, and CUHK during training. If it had, the claim in the paper that “ISTD+ ground truth was not used during construction, yet the results are close to the ground truth” might simply mean that the model remembered the original paired samples, rather than that this process actually removed the shadows. AgenticShadow uses public datasets as its source data and faces the same issue.

4 comments
AuthorsOpenReview ↗

Clarification of Our Data Contribution

Thank you for recognizing AgenticShadow’s broader coverage and the downstream improvements enabled by its supervision.

Your review raises three main concerns about the workflow, and we readily acknowledge that our workflow is not perfect. Even so, to our knowledge, it is the first of its kind dedicated to constructing paired supervision for shadow removal, and its impact is already visible in what existing methods can do. The dataset we built with this workflow enables those methods to achieve far better results across a much wider range of scenes. Before our work, much of the shadow removal research focused on a relatively narrow setup in which the shadow was in the image but the object casting it was kept outside the frame. With our data, the very same methods begin to perform reasonably well across all four domains we evaluated, including faces, remote sensing imagery, and street scenes. We believe the field’s ability to move beyond controlled benchmarks and into everyday use depends on breaking this data bottleneck. We hope this clarifies the broader significance of addressing the data bottleneck.

AuthorsOpenReview ↗

Responses to Reviewer cLNR’s Questions and Concerns

As image editors and VLMs improve, will a well-written prompt be enough for shadow removal? (W1)

Possibly. However, with current models, prompting alone can leave shadows or alter scene content, so we evaluate and correct the generated edits before using them as training targets.

What does the physics-motivated prompt add beyond making material preservation more explicit? (W1)

That explicit physical framing is the intended benefit. It distinguishes illumination changes from material changes and guides preservation of surface details. This reduces mean CDD on SRR from 12.07 to 9.51 (Table 2(b), CDD values scaled by 10³, lower is better). Figure 14 in Appendix B.4 provides a qualitative comparison with generic prompting.

Why retain probing, and how much do retries and selection help? (W1)

Only 2.07% of initial probes are flagged for major-failure retry, but these failures can persist across repeated generations without corrective feedback (Appendix B.4, Figure 13). Selection still assesses overall quality when the probe passes. On SRR, retry reduces CDD from 9.51 to 7.78, visual selection further reduces it to 7.45, and Retinex-LoG guidance to 7.07.

Does VLM evaluation cover only two failure modes? (W2)

No. The two criteria apply to major-failure evaluation. A separate VLM selection stage compares passing candidates on overall removal quality, scene preservation, and visual artifacts, supported by Retinex-LoG diagnostics. We have clarified these distinct evaluation roles in Section 3.

Does AgenticShadow improve removal quality beyond matching generated targets? (W3)

Yes. The benefit of training specialized removal models on AgenticShadow extends beyond matching generated targets, as supported by:

  1. Human assessment: We conduct a blinded study on 80 AgenticShadow cases, using original inputs with ordinary resizing for SBU/CUHK. On a four-point scale of 1 (Very poor), 2 (Poor), 3 (Good), and 4 (Very good), combined-trained PF scores 3.11/4 versus 1.89/4 for ISTD+-only PF, an average paired improvement of 1.22 points.
  2. Held-out generalization: We evaluate on all 1,100 held-out WSRD+ pairs with captured shadow-free references from controlled indoor, close-range scenes. Compared with ISTD+-only training, adding AgenticShadow improves PhaSR’s PSNR by 2.84 dB and reduces LAB RMSE by 30.1%. Video comparisons provide additional qualitative evidence of broader applicability.

We have added the WSRD+ evaluation in Section 5.4 and the human study in Section 5.5.

Performance on ISTD+ has largely saturated. Its controlled captures cover a narrow range of scenes, so improvements on ISTD+ alone provide an incomplete assessment of broader real-world applicability.

Could the foundation model have encountered these public datasets during pretraining? (Q1)

We agree that exposure to public datasets during VLM pretraining is possible, and we cannot verify or exclude it. We have revised the ISTD+ statement to clarify that the references were not supplied during our construction, without implying that they were absent from VLM pretraining.

However, the public shadow detection datasets used for AgenticShadow provide shadow images and masks, but no paired shadow-free references. For general real-world scenes, the desired shadow-free target is often a counterfactual rather than an available captured image that the VLM could have encountered during pretraining. Our workflow constructs approximations of these missing targets.

Thank you for the detailed response. Although the author answered some of my questions, I am not entirely convinced by the author’s responses. In addition, I believe the author should respect existing research works and engage in the necessary discussion, including but not limited to:

  1. Des3: Adaptive attention-driven self and soft shadow removal using vit similarit
  2. Recasting Regional Lighting for Shadow Removal
  3. Dc-shadownet: Single-image hard and soft shadow removal using unsupervised domain-classifier guided network
  4. Bijective mapping network for shadow removal
AuthorsOpenReview ↗

Follow-up response

“The author should respect existing research works and engage in the necessary discussion.”

We thank the reviewer for suggesting additional references [1,2,3,4], which we have incorporated into the revised manuscript with corresponding discussion. We also include the revised discussion at the end of this message. Besides, we notice that both [1] and [3] explicitly address soft shadow removal. We have also added qualitative comparisons on soft shadow examples in Figure 19. While we cannot run [1] and [3] at this late stage, we have included HomoFormer and PhaSR alongside SID, ShadowFormer, Inpaint4Shadow, and ShadowDiffusion in the comparison. Simply put, for soft shadow removal, our simple PF baseline trained with AgenticShadow produces more complete removal than the compared SOTA models trained on ISTD+.

[1] DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity

Yeying Jin, Wei Ye, Wenhan Yang, Yuan Yuan, Robby T. Tan - AAAI 2024

[2] Recasting Regional Lighting for Shadow Removal

Yuhao Liu, Zhanghan Ke, Ke Xu, Fang Liu, Zhenwei Wang, Rynson W. H. Lau - AAAI 2024

[3] DC-ShadowNet: Single-Image Hard and Soft Shadow Removal Using Unsupervised Domain-Classifier Guided Network

Yeying Jin, Aashish Sharma, Robby T. Tan - ICCV 2021

[4] Bijective Mapping Network for Shadow Removal

Yurui Zhu, Jie Huang, Xueyang Fu, Feng Zhao, Qibin Sun, Zheng-Jun Zha - CVPR 2022

“I am not entirely convinced by the author's responses.”

We have made a good-faith effort to address all technical concerns. In our previous responses, we provided quantitative ablations and qualitative evidence addressing the effectiveness of physics-motivated prompting, probing, retries, and selection (W1); clarified the distinct roles of VLM evaluation and selection (W2); added a human study and held-out WSRD+ evaluation to demonstrate improvements beyond generated targets (W3); and explicitly acknowledged the possibility of dataset exposure during foundation-model pretraining (Q1).

While we understand that the reviewer remains unconvinced, it is unclear to us which specific concerns remain unresolved in light of these additional analyses and revisions. We would greatly appreciate it if the reviewer could state which aspects of our responses they find insufficient, so that we can address them directly.

Revised discussion in Section 2:

Shadow Removal. Early methods used illumination invariance (Finlayson et al., 2002), matting (Wu et al., 2007), texture consistency (Liu and Gleicher, 2008), and shadow-lit region relationships (Guo et al., 2013). Learning-based approaches introduced contextual reasoning (Qu et al., 2017), joint learning (Wang et al., 2018; Zhu et al., 2022), hierarchical aggregation (Cun et al., 2020), and shadow-image or illumination-reflectance decomposition (Le and Samaras, 2019, 2022; Liu et al., 2024). Recent methods exploit transformers, inpainting, and diffusion models (Guo et al., 2023; Xiao et al., 2024; Li et al., 2023; Guo et al., 2023; Xu et al., 2025), incorporate semantic or geometric priors (Lee et al., 2026), or explicitly address soft shadows (Jin et al., 2024). Semi-supervised and unpaired methods (Ding et al., 2019; Hu et al., 2019; Jin et al., 2021) acknowledge the limited availability of paired training data and reduce reliance by using unlabeled or unpaired images. AgenticShadow complements these advances by expanding paired supervision across diverse real-world scenes.

Reviewer_RMjHOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The work proposes a shadow removal pipeline. It takes the input image with shadow and its shadow area mask. The inputs are evaluated using visual language models for probe evaluation and generation of the desired shadow-removed image. The model utilizes ChatGPT 2026 for generation and ChatGPT Mini for checking generated probes for hallucination and probe selection. Then, a physics-motivated agentic workflow for noise removal and color correction.

Critical strengths
  1. It claims to propose the first large-scale shadow-removal benchmark based on real-world scenarios, with 17,138 triplets.
  2. Well-inspired workflow for shadow removal with a preprocessing pipeline of ChatGPT 2026 and GPT Mini for probe selection.
  3. Integrating the proposed AgenticShadow supervision significantly improved performance in terms of PSNR, SSIM, and other evaluation metrics.
Critical weaknesses
  1. The proposed work relies heavily on VLM (ChatGPT 2026 image 2) generated images for ground truth without any human-centric cross-validation for false outputs.
  2. The mentioned GPT mini probe selection is insufficient as it can also hallucinate without any human-guided metric for evaluation.
  3. The proposed Prior Fusion is HomoFormer with frozen DINOv2 and added depth features. The paper doesn’t justify the fusion choices or compare against alternatives.
  4. Gains are measured on test targets made by the same pipeline. No held-out real paired benchmark is used, such as SRD or WSRD+.
  5. The paper has several presentation and technical clarity issues. There are frequent grammatical errors and awkward expressions, such as unclear sentence structure.
  6. The pipeline mostly assembles existing parts: a generic generate, evaluate, retry, and then select loop, a GPT image 2 editor, and a GPT -5 mini judge, classical Retinex LoG, and median-based color correction. The paper’s own related work cites similar generate-and-validate data construction work and the main novelty is applying this loop to shadow data
Questions for authors
  1. Justify the claim of sufficient ground truth validation without humans in the loop. How did you eliminate the hallucination of GPT mini for probe evaluation?
  2. The contribution states a benchmarking dataset proposal but the mentioned dataset count comes from just preprocessing the existing dataset. Justify the contribution in the dataset proposal other than preprocessing the existing datasets.
6 comments
AuthorsOpenReview ↗

Clarification of Our Focus and Contributions

Thank you for your time and feedback.

Based on your summary, we believe that some aspects of our work may not have been communicated clearly and we would first like to make the stakes clearer. For a decade, dozens of shadow removal methods have been proposed, yet much of their training and testing has been confined to relatively simple scenes due to challenges in shadow removal data collection. Despite demand from industry and academia, reliable shadow removal in everyday workflows remains an unmet need. The fundamental bottleneck has always been the data. Our main contribution is to demonstrate a way past this data bottleneck, and the proposed workflow is a means to that end. We show that realistic pseudo-paired data can be generated at scale and, more importantly, that training existing shadow removal methods with this data enables them to handle complex, real-world scenes that were previously beyond their capabilities. These results demonstrate that substantial progress in practical shadow removal can be achieved by addressing the data bottleneck, even without developing new model architectures.

Accordingly, we hope our work is judged by the following contributions:

  1. Workflow: We introduce a practical, scalable way to generate approximate shadow-free targets from real images. As more shadow images become available, our workflow will continuously expand the training data and support shadow removal in increasingly diverse and challenging scenes.
  2. Training Data and Benchmark: In its current form, AgenticShadow is already the largest and most diverse real-world shadow removal dataset for training and evaluation. It includes high-quality paired shadow and shadow-free images across general scenes, faces, and remote-sensing imagery at a scale far beyond that of existing datasets.
  3. Validation and Baseline: We carefully validate the quality and usefulness of our generated data and establish AgenticShadow as a reliable benchmark for training and evaluation. We show that broader and more diverse supervision generally improves existing shadow removal methods, especially on challenging real-world scenes. We also demonstrate mask-free shadow removal, video applications, and generalization to unseen domains. Our simple PF baseline achieves the best overall macro performance on the benchmark and provides a strong reference point for future work.
AuthorsOpenReview ↗

Responses to Reviewer RMjH’s Questions and Concerns

What do the dataset and workflow contribute beyond preprocessing? (Q2, W6)

Our workflow constructs the shadow-free targets missing from existing shadow detection datasets, instead of simply preprocessing shadow images. It transforms image–mask pairs into image–mask–target triplets, providing paired supervision for shadow removal at scale across diverse real-world scenes.

The non-trivial workflow addresses incomplete shadow removal, altered scene content, and lighting inconsistencies through major-failure retry, candidate selection, Retinex-LoG diagnostics, and regional correction. These stages reduce CDD from 9.51 with physics-motivated prompting alone to 6.96 (Table 2(b)). Appendix B.4 illustrates each step’s contribution.

How are targets validated through human assessment and automated evaluations? (W1, W2, Q1)

We have conducted a blinded human study to provide human-centric cross-validation of the constructed targets. In the study, 97.6% of generated-target ratings were Good or Very good, supporting their quality as approximate shadow-free supervision.

The study used 45 ISTD+ cases. Raters viewed the shadow input alongside two anonymous candidates, the generated target and captured reference, and scored each using a four-point scale: 1 (Very poor), 2 (Poor), 3 (Good), and 4 (Very good). Generated targets scored 3.68/4, compared with 3.76/4 for captured references. We have included this human assessment in Section 5.5 and its protocol in Appendix C.4.

Probe evaluation screens for two task-specific major failure modes identified through manual inspection of initial SRR outputs. Passing candidates undergo broader VLM visual judgment supported by human-vision-inspired Retinex-LoG diagnostics. Appendix B.4 illustrates these evaluation roles. The complete workflow still cannot eliminate every error, and Appendix B.5 documents the remaining failure cases and limitations.

Do the gains extend to captured pairs outside the construction data? (W4)

Yes, the gains extend to the held-out real paired benchmark WSRD+. Across all 1,100 pairs, PhaSR trained with ISTD+ and AgenticShadow improves PSNR by 2.84 dB and reduces LAB RMSE by 30.1% compared with ISTD+-only PhaSR.

WSRD+ contains controlled indoor, close-range scenes with captured shadow-free references. Since it provides no annotated shadow masks, we evaluate frozen mask-free models:

Model / training dataPSNR ↑SSIM ↑LAB RMSE ↓
PhaSR / ISTD+18.450.73008.61
PhaSR / Combined21.290.76106.02
PF-MF / Combined20.800.75566.39

Why this Prior-Fusion design? (W3)

PF is a simple modification of HomoFormer to test whether semantic and depth priors provide additional gains with broader supervision. We choose these priors based on their demonstrated usefulness in recent shadow removal methods (StableSR and PhaSR), and report controlled comparisons of the different configurations in Table 3(b) and Appendix D.2. It serves as a new baseline for future shadow removal research.

Reviewer_RMjHOpenReview ↗

After a Decade: Bringing Shadow Removal into the Real World with Agentic Training Data

Thank you for clarifying the focus of the work. I agree that curating high quality training targets is the main contribution here and address a key bottleneck in shadow removal. I also appreciate the clarity on the workflow using GPT Image 2, GPT 5 mini, retinex-log, and regional color corrections. However, few concerns are here:

  1. The pipeline relies on two general-purpose models (GPT Image 2 and GPT-5 mini) which are not designed for shadow removal, and the choice of these two is not really justified. No alternative editors or judges, such as open-source image editors, are compared. Releasing the dataset helps, but the workflow itself remains tied to closed models and cannot be fully reproduced or extended.
  2. The "high-quality" claim is human-verified only on 45 ISTD+ cases, while the targets themselves in the new domains (faces, remote sensing, general scenes) are not human-rated. The non-shadow RMSE of 27.27 is also higher than captured datasets (WSRD+ 20.29, SRD 10.83), which suggest unwanted edits pass through the workflow and is likely why trained models show larger non-shadow changes.
  3. Since test targets come from the same pipeline, in-benchmark gains mostly measures agreement with the generator rather than true restoration. WSRD+ is the only evaluation against captured references, and it does not covers the new domains (faces, remote sensing).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.