- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a relation-indexed representation for selective shadow removal. The task is an interesting topic in shadow manipulation, and the authors identify that the current gaps are: 1) single caster, multiple receivers, and 2) shared overlap pixels between shadows. The authors formulate the caster-shadow-receiver relations and corresponding remove/preserve instructions, and propose a method to learn the correct reconstruction conditioned on the given relations and instructions. The authors evaluated the proposed method on both Kubric synthetic scenes and real DESOBAv2 images.
Critical strengths
-
The authors' definition of the task is clear. The formulation of caster-shadow-receiver relations is straightforward, with well-defined remove/preserve actions.
-
The evaluation strategy is clear. The authors first test and validate the proposed method on fully controllable synthetic data using Kubric. Then the method is evaluated on real images from DESOBAv2 data.
Critical weaknesses
-
The organization and presentation make the method section hard to follow. A constructed example from a Kubric scene would help explain Sec. 3.1 and Sec. 3.2. Fig. 2 also does not clearly show the data flow described in Secs. 3.2–3.5, especially the interaction between the appearance features and the conditioner and the predicted residual in (a). The largely sequential flow in (b) does not clearly reflect the interactions and dependencies described in the sections. Can the authors clarify the actual operations involved in the Action-CSR conditioner, its intermediate outputs, and its interactions with other components of the network?
-
The fairness of the comparison on real images is questionable. The proposed method is trained on DESOBAv2, but the authors have not specified whether the compared models are also retrained on the same data. If, as reported, the compared models use their official checkpoints, then the comparison is not fair, since the images in other shadow removal datasets, like ISTD+ and SRD, differ a lot from the DESOBAv2 data. Consequently, the fairness of the visual comparison is also affected. Can the authors clarify what datasets the compared models were trained on and report results where all models (or some representative compared models) are trained on the same DESOBAv2 dataset?
-
The authors' strategy is to verify the idea on fully controllable synthetic scenes. However, the experimental results do not show comparative visual evidence of the benefit of the proposed method on the two failure modes in Kubric scenes. Fig. 4 only shows an example of the proposed method on overlapping shadow pixels. Similarly, the authors do not provide clear comparative visual evidence in real scenes. The improvements in MAE scores do not show whether the improvements are visually meaningful. Can the authors provide qualitative results for the single-caster, multiple-receiver case and the shared-overlap case, including visual comparisons between existing methods, ablated versions of the proposed method, and the full proposed method on both synthetic and real scenes?
Questions for authors
-
Please see my question in Weakness 1.
-
Please see my question in Weakness 2.
-
Please see my question in Weakness 3.
-
How substantial are the reported improvements in MAE (in Table 2 and Table S12) for this task? The reported improvements are all below 0.1 on the original scale.
-
What is on the right side of each panel in Fig. 1? Is it just an illustration of the relations and actions, or is it some intermediate outcome? The authors should clarify this, as the layout and positions of the casters and shadows do not match the images on the left side. This could lead to misunderstanding.
-
I suggest that the authors add an illustrative figure (in the main paper or appendix) showing example rendered scenes from the Kubric synthetic dataset. It would help readers understand the setting, the coverage of the failure modes, and the complexity or insufficiency of the synthetic scenes.
5 comments
3. Two Kubric Failure Modes, Ablations, and MAE Scale (W3 / Q3 / Q4)
Concern. The two Kubric failure modes lack visual comparisons with ablations and existing methods. Moreover, the reported MAE improvements are below 0.1, making their practical significance unclear.
Response. We clarify the two failure modes using their respective measured action subsets and report the corresponding ablation results.
For receiver-specific attribution, one caster creates shadows on multiple receivers. The action selects which receiver to edit and which to preserve. This subset contains 10 scenes and 120 actions.
| Configuration | Target ↓ | Protected ↓ | Leakage ↓ |
|---|---|---|---|
| Caster–Shadow | 2.80 | 4.60 | 2.60 |
| + Receiver identity | 2.40 | 3.59 | 2.55 |
| + Action binding | 1.12 | 3.48 | 2.52 |
| + Protection | 1.06 | 1.82 | 2.48 |
| Full | 1.02 | 1.75 | 1.98 |
Adding receiver identity reduces Target and Protected errors by 14.3% and 22.0%, respectively. Overlap is N/A because this subset has no selected–protected intersection.
For shared-overlap attribution, shadows from different casters overlap on the same receiver. This subset contains 11 scenes and 154 actions.
| Configuration | Target ↓ | Protected ↓ | Overlap ↓ | Leakage ↓ |
|---|---|---|---|---|
| Full-information Spatial | 1.35 | 2.10 | 3.20 | 1.98 |
| Independent Tokens | 1.22 | 1.90 | 2.78 | 1.98 |
| Full Interaction | 1.08 | 1.70 | 2.40 | 1.96 |
Full reduces Overlap error by 25.0% compared with Spatial and 13.7% compared with Independent Tokens.
On the separate 352-action overlap-valid pool, the progressive C-S-R, +Action, +Protection, and Full configurations achieve Target/Protected/Overlap errors of 3.221/6.682/7.421, 1.080/6.550/7.280, 1.020/2.250/2.950, and 0.970/1.563/2.094, respectively. The equal-information Spatial/Independent/Full comparison gives Overlap errors of 2.850/2.420/2.094. These results use different ablation designs and are therefore reported separately.
Target, Protected, and Overlap are RGB MAE multiplied by on normalized images. On the 352-action Kubric pool, the Overlap reduction from 2.850 to 2.094 corresponds to a raw MAE decrease from 0.02850 to 0.02094 (0.00756, or about 1.93 levels on an 8-bit scale). The DESOBA reduction from 3.60 to 2.90 corresponds to 0.007 raw MAE (about 1.79 8-bit levels). These are regional averages.
Figure 1(a) shows Kubric ShadowEdit inference outputs alongside the corresponding renderer references and change/topology diagnostics; Figure 1(b) shows DESOBA model outputs under different remove instructions. Figure 4 presents DESOBA outputs and baseline comparisons on the left and Kubric caster-removal outputs on the right. We use the measured ablations and quantitative baseline comparisons as evidence for method comparisons; the figures provide qualitative examples of model outputs.
Revision locations. Main-paper Section 4.4, Table 2, and the new cross-reference to Supplementary Table S34; Supplementary section on Kubric failure-mode ablations; Supplementary Tables S34, S37, and S17.
4. Figure 1 Right Panel and Kubric Scene Coverage (Q5 / Q6)
Concern: The right side of Figure 1 does not spatially match the left images. The paper also needs rendered Kubric scene examples that show coverage of the failure modes.
Response. Response. Figure 1(a) shows Kubric ShadowEdit inference outputs alongside renderer references and change/topology diagnostics; Figure 1(b) shows DESOBA model outputs under different remove instructions. Figure 4 presents DESOBA outputs and baseline comparisons on the left and Kubric caster-removal outputs on the right. The full Kubric diagnostic contains 36 scenes and 472 action–scene pairs: 26 overlap-valid scenes with 352 actions and 10 receiver-specific scenes with 120 actions.
Revision locations. Main-paper Figure 1 and Figure 4 captions; Supplementary Appendix A.2 and Table S5 for the scene/action inventory; Supplementary section on Kubric failure-mode ablations and Table S34 for topology-specific measured results.
We have kept the renderer-based scene illustrations, the receiver-specific subset, the shared-overlap subset, and the 352-action pooled evaluation distinct. This avoids presenting different cohorts or evaluation protocols as if they were one experiment.
Dear Reviewer fctY,
Thank you again for your time and constructive feedback.
With only several hours remaining in the author–reviewer discussion period, we would greatly appreciate it if you could take a moment to review our rebuttal and revised manuscript. We have tried our best to address your concerns through comprehensive analyses, experiments, and clarifications.
We would be happy to clarify any remaining questions and look forward to your further feedback.
Thank you for your time and consideration.
Best regards, The Authors
Thank you for the concrete checks on the method figure, the DESOBA comparison protocol, and the visual scale of the selective gains. We answer W1–W3 and Q4–Q6 below.
1. Action-CSR Data Flow and Figure 2
Direct answer. We agree the original Figure 2 was hard to read as an execution path. The intended order is: compile supports → update relation tokens → assign them to pixels → gather appearance evidence → modulate only inside the permission gate → predict a residual → hard-compose with the input.
Given image X, relation set R, and action vector a, the compiler produces Mremove, Mprotect, and Mperm. Mremove marks the selected shadow support; Mperm is the only region where the network may write. Dense relation fields yield FR, and each valid caster–shadow–receiver unit becomes a labelled token ti0. After invalid tokens are dropped, the remaining tokens interact to form T′, so a selected relation can use context from protected or competing relations without losing its identity.
At decoder scale ℓ, image features query T′ to produce Zℓ. Zℓ assigns relation context to locations; it does not authorize writing. In parallel, frozen priors from X give action-independent appearance features P0. Action-dependent focus yields Pa, and the base branch combines X, Pa, and support cues into the appearance candidate Yg. Thus T′ / Zℓ identify the requested intervention, while P0, Pa, and Yg supply appearance evidence.
Image features Fℓ, Zℓ, and the aligned controls then predict scale/bias fields that modulate decoder features only inside Mperm, producing Feℓ. The residual refiner uses FR, T′, and Feℓ to predict Δ. The final output is
Y=(1−Mperm)X+Mpermclip(Yg+Δ).
Pixels outside Mperm are copied from X; only permitted pixels receive the reconstruction and residual correction.
Revision locations. Main-paper Section 3.1 and the Figure 2 caption; Sections 3.1–3.5, especially Section 3.5; Figure 2 caption; Eqs. (8)–(10); Appendix B.2.2 . Supplementary Appendix B.2.2. Figure 2 artwork is retained.
2. DESOBA Comparison Fairness (W2 / Q2)
Response. We revised the wording to distinguish a shared test-time evaluation protocol from matched training. External baselines were retrained to convergence on the same DESOBA v2 training split, using their original architectures and inference procedures. After prediction generation, all outputs were evaluated on the same fixed DESOBA test list, references, masks, metric-stage resize and preprocessing, and frozen scorer. The controlled within-method comparison is ShadowEdit-Mask versus ShadowEdit-Full under the matched ShadowEdit training and evaluation setup.
The controlled within-method comparison is ShadowEdit-Mask versus ShadowEdit-Full, which uses the matched ShadowEdit training and evaluation setup. ISTD+ and SRD use separately trained relation-free configurations and are reported as restoration-compatibility evaluation.
Revision locations. Main-paper Section 4.2 and Table 1 caption; Supplementary Appendix D.4, especially the cross-method comparison protocol and the comparison-scope table.