- Rating
- 2
- Confidence
- 1
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes a novel evaluation protocol (a transport lattice of illumination-occlusion combinations) and a learned operator that supports inverse restoration and forward re-shadowing. The approach is evaluated on the shadow removal task (Table 8) against related work on four different benchmarks and shows strong results.
It would be great to see a controlled transport lattice protocol evaluation also for real scenes, not just synthetic. Apart from the two contributions above, I struggle to identify which parts of the architecture are commonly used in previous work and which are novel contributions. This could be stated more clearly.
Critical strengths
- A novel evaluation protocol (a transport lattice of illumination-occlusion combinations). Formulating the problem as learning one "held-out" combination makes sense.
- A learned operator that supports inverse restoration and forward re-shadowing. Strong results in the shadow removal task
Critical weaknesses
I found the current draft terse and difficult to parse. The illustrations are hard to follow, notation is dense, with many terms not defined at first use (or left undefined). For that reason, I find it challenging to assess the technical soundness and novelty aspects of this paper. I feel that this submission would benefit from a thorough editing pass to clarify the method description, so unfortunately, despite the strong results, I will not recommend the current draft for acceptance at ICLR.
I also struggle to identify which parts of the architecture are commonly used in previous work and which are novel contributions. This could be stated more clearly.
It would be great to see a controlled transport lattice protocol evaluation also for real scenes, not just synthetic.
Several figures and illustrations are duplicated in the supplemental material. I would have liked to see more visual results.
Questions for authors
There are visible color shifts compared to GT in Fig 5. It may be worth a discussion. Can this be addressed?
How large is the network, how many diffusion steps are used and what is the inference cost of the proposed approach?
For visual results, I would recommend including additional visual examples in the supplemental material. Currently, Figure 5 and Figure 8 show the same examples, which are also reused in Figure 1. I would also recommend including visual examples of limitations to understand where there are remaining challenges.
How are the geometric condition features provided? Are they extracted from the input image, and if so, with what tools, or are they readily available (as gbuffers) in the synthetic dataset?
Fig 2 is confusing to me. "Transport residual" is included in many places, and it is unclear to me if that is a constant tensor that is being reused, or if it is updated in each pipeline stage. The "Shared Factorized Transport Renderer" are included twice, with arrows from the input. I assume the lower row is a "blow-up" of the upper red box, but having the split arrow from the input makes this confusing, it looks like two parallel blocks. What is "FOTA adapter" referring to? If this is a commonly known architecture from previous work, please add citations or additional details.
Fig 3 is very dense, and hard to parse. Make it bigger (fonts are hard to read). I would also recommend adding more text to explain how it fits into the overall pipeline.
Mixed up notation in Fig 1: It is unclear to me what O_A and O_B represent. In Fig 1, it looks like (L1,VA) and (L1, VB) differs only in the occluder, so I assuime O_A and O_B represents different occluders. This could be stated explicitly early in the paper.
What is "support m" and "context F" referring to (l:160)?
Define non-shadow anchor.
Section 3.2 is hard to parse. I would recommend introducing what projected appearance and geometry features are, and where in the pipeline they are introduced.
Section 3.3 starts abruptly, and I don't know where in Fig 2 or Fig 3 Eq. 4 fits in. Some additional text would help the reader navigate here.
It is unclear to me how large the network is, how many diffusion steps are applied, what diffusion model is used, and what the runtime cost of the proposed method is.