- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a method to recover the content from a monitor screen that is reflected onto a wall. The pipeline combines various hand-crafted features and prior biases (such as weighing low-frequency information higher in the prediction) and consists of two main components (LTOR, EBL) and an image-to-image backbone, which is used twice in a single inference. The technical contributions seem quite obscure due to the writing that mostly details low-level implementations with overwhelming notations. The validation is done on four content categories using standard metrics like PSNR, SSIM, LPIPS and compared against baselines designed for general image-to-image tasks. Ablation studies on the two components are provided and experiments on different resampled inputs and masked inputs. The reported scores outperform the baselines; however, the evidence is not convincing.
Critical strengths
-
An interesting and difficult problem that can potentially reveal privacy issues.
-
Extensive effort to include various baselines and numerous ablation studies, such as grid occlusion, replacements, prior variants, etc.
Critical weaknesses
- The experimental setups and results may only reflect memorization, not that the light transport is effectively learned. The proposed method is never evaluated across categories, and even within the same categories, it is unclear how much content overlaps between the training and testing frames. The results in Fig. 3 show accurate color reproduction and fine details that are extremely unlikely to be recovered from such a diffuse wall input. This seems to suggest that the method simply memorizes category-specific appearances and uses a minimal signal to retrieve those memorized visuals, rather than learning the inverse light transport or the wall-to-screen mapping.
- The method relies on relatively complex, hand-crafted heuristics, which mostly lack strong justification in the era of modern learning. For example, the two-time inference via the backbone is not well motivated, and the method could incorporate all input signals in one pass and still utilize confidence prediction and refinement without the two passes. The ordered band prior is simply hard-coded and not learned, and the learned correction might make it even irrelevant (Table A4 shows that reversed band prior even yields better PSNRs on all datasets than no prior at all). While handling scales, frequencies differently or utilizing local statistics sounds sensible, manually handling them seems unnecessary, even suboptimal, in the modern learning paradigm. Most critically, the framework does not model any physical relationships, such as the relative positions of the screen, wall, camera, and simply shares the input wall and output screen coordinates, leading the networks to learn a fixed mapping that fails to model generalizable light transport.
Questions for authors
- Were the baselines’ hyperparameters properly tuned or simply taken from the default values, which were meant for different tasks? Were they trained from scratch or fine-tuned? How were some baselines adapted for the task? The details on the training and evaluation of the baselines are insufficient, making the comparison unconvincing.
- Have statistical tests been performed? Are the improvements statistically significant, especially for the small gains from key components like LTOR, EBL (Table 2)?