acceptodds
Under review as a conference paper at ICLR 2027

ReGLaR: Learning Reflection-Guided Latent Representations for Hidden Screen Content Recovery from a Diffuse Wall-Reflection Image

Abstract

Can an ordinary camera recover a hidden screen's displayed content from a red, green, and blue (RGB) image of its reflection on a visible wall? Diffuse reflection spatially mixes and attenuates the screen signal, while wall texture, ambient illumination, and camera noise obscure the remaining content cues. Recovery therefore requires turning the wall observation into screen content while retaining the useful information carried by the reflection. We propose Reflection-Guided Latent Representation Learning (ReGLaR) to address these requirements. Measurement-to-Content Latent Representation Learning (MCLRL) aligns a transitional wall-side latent with a fixed clean-content reference, guiding intermediate features toward recovering screen content. Low-Order Reflection Consistency Supervision (LRCS) extracts low-pass RGB, luminance, and slow spatial variation from the observed wall. It compares them with a learned proxy generated from the reconstruction in a shared coarse space. Experiments cover interfaces and wallpapers, natural images, data visualizations, and games, together with changes in the wall observation. ReGLaR outperforms eighteen image restoration baselines across the five quantitative datasets and retains a comparative advantage across most observation variations. The recovered brightness, color distributions, regional layouts, and principal contours demonstrate the initial feasibility of hidden-screen recovery through an ordinary camera's view of wall reflections.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Screen Reconstruction, Deep learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_iCz8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of recovering the content of an occluded screen from diffuse wall reflections captured by an ordinary camera. The authors propose ReGLaR, where MCLRL transforms wall-side representations toward screen-content representations, while LRCS further constrains the final prediction using low-frequency color, luminance, and spatial variation information from the observed wall. Overall, the proposed method forms a relatively complete and task-specific framework, but there are still several issues that need to be addressed.

Critical strengths
  1. The problem itself is relatively novel and has clear research value.

  2. The experiments are extensive. In particular, the generalization and robustness studies are relatively comprehensive. The authors systematically vary factors such as distance and camera viewpoint, and conduct multiple wall-observation variation studies.

Critical weaknesses
  1. The paper is insufficiently positioned with respect to prior work on hidden-screen recovery. The Introduction provides limited discussion of related work, and the Related Work section does not clearly explain the advantages of the proposed method over existing approaches. Moreover, the experiments mainly compare against generic image restoration methods, while lacking direct comparisons with methods designed for the same task.

  2. The presentation still needs improvement. Figure 2 contains too much text and too many modules, making the framework appear more like a stack of multiple components and obscuring the key methodological novelty. In addition, the authors appear to have incorrectly used the ICLR 2026 template instead of the ICLR 2027 template.

Questions for authors
  1. What are the advantages of the proposed method over existing methods for the same task, and what concrete progress does it make beyond prior work?

  2. Can direct comparisons with methods designed for the same task be added?

  3. Can visualizations of the learned proxy be provided to further validate whether it captures measurement-related information?

Reviewer_2tHVOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper presents ReGLaR, a method for recovering occluded screen content from a single RGB image of its diffuse reflection on a wall captured by an ordinary camera. MCLRL transforms wall observation features into screen-content representations, supervised through alignment with clean-screen features. LRCS uses a learnable mapping to match the reconstruction to the low-frequency color, luminance, and spatial variation in the wall observation. The model is trained on paired wall and screen images and requires only a wall image at inference. Experiments across multiple datasets and observation conditions demonstrate its ability to recover screen color distributions, regional layouts, and principal contours.

Critical strengths

The experiments cover multiple datasets, various observation changes, and extensive ablation studies, showing improvements over the reported image restoration baselines in recovering color distributions, layouts, and coarse contours.

Critical weaknesses
  1. The practical value of the recovered information has not been sufficiently validated. In Fig. 3, the method mainly improves color distributions, window layouts, and coarse contours, while the actual content remains difficult to identify. Although the paper includes target matching experiments, these rely on the similarity of low-resolution RGB images and primarily reflect correspondence in coarse appearance. The experiments do not sufficiently establish what useful conclusions about screen content can be drawn from the reconstructions, leaving their practical value unclear.

  2. The core technical contribution is not clearly articulated, and the distinction from existing methods is insufficiently justified. MCLRL primarily combines gated residual transformations, clean-screen feature alignment, and local refinement. LRCS uses a learnable mapping to match the reconstructions to the low-frequency color, luminance, and gradient features of the wall observations. However, the paper does not sufficiently explain how these specific modules provide new solutions to the spatial mixing and information loss caused by diffuse reflection. In particular, existing methods such as NLOS-LTM also combine latent representation learning with reprojection constraints.

  3. The method figure lacks a clear correspondence with the text and contains specific inconsistencies that hinder understanding and reproducibility. Fig. 2 combines the imaging principles, overall architecture, training supervision, and module details without clearly highlighting the main inference path from input to output. The definitions of x,yx,y for screen content and wall observations are reversed relative to the main text. The LRCS branch shows channel dimensions of 3→16→63\rightarrow16\rightarrow6, whereas Appendix B.2 specifies 3→24→63\rightarrow24\rightarrow6. Some labels also contain unusual spelling or unclear wording. These anomalies raise questions about whether generative AI was used to assist in creating the figure, and the authors should clarify how it was produced and checked. The current manuscript also lacks an AI Use Statement, which is required by the ICLR 2027 author guidelines.

  4. Comparisons with directly relevant methods are missing, and some ablation visualizations do not adequately support the conclusions. Table 1 mainly compares general-purpose image restoration networks and networks designed for specific degradations. The cited methods NLOS-OT (TIP 2022), NLOS-LTM (TIP 2025), and IR4Net (2026) are not included in the experiments. IR4Net, in particular, also studies hidden-screen content recovery through diffuse reflection. The existing experiments therefore do not sufficiently establish the proposed method’s advantage over prior work in this field. Furthermore, Appendix Fig. 4 explicitly includes results from replacement models trained from scratch for only three epochs. Their differences from the full model may reflect differences in training progress and cannot be confidently attributed to module design alone.

  5. The data splits and generalization evaluation are insufficiently documented, making it difficult to assess recovery of unseen screen content. Appendix A.3 reports the sizes of the training, validation, and test sets but does not explain whether repeated captures of the same screen content, similar interfaces, or consecutively acquired samples are kept within a single split. Content overlap could substantially affect evaluation in a task that primarily recovers colors and coarse layouts. The current description does not adequately rule out the possibility that the model relies on familiar interface patterns for prediction. The authors should clarify how the data are split by screen content and how duplicates are removed. Moreover, although the paper evaluates various observation changes, generalization across rooms or independently acquired environments has not been sufficiently validated, leaving applicability to practical conditions unclear.

  6. The appendix contains lengthy mathematical formulations that provide limited insight into the core design and its justification. Appendix B expands standard operations such as convolution, normalization, sigmoid gating, average pooling, and Smooth L1 loss in considerable detail. While these descriptions offer some implementation guidance, they do not sufficiently explain the connection between the specific network design and the light transport model in Eq. (1), nor do they provide corresponding recoverability or stability analyses. Meanwhile, the decoder’s frequency, detail, and edge refinement operations are still described primarily through brief textual summaries. The paper should devote more attention to explaining its key design choices and their rationale, so that the level of mathematical detail reflects the technical focus of the work.

Questions for authors

refer to Critical Weaknesses

Reviewer_B5wDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper recovers coarse content of a hidden screen from one RGB photo of its diffuse reflection on a visible wall. MCLRL aligns an intermediate wall-side feature to a frozen clean-content encoder. LRCS requires the reconstruction to match the wall observation in low-pass color, luminance, and slow spatial variation. Across five content sets, it outperforms 18 generic image-restoration networks.

Critical strengths
  • The problem is concrete, and the claim is scoped honestly to color, layout, and principal contours rather than fine text.
  • The Screen gain is real: 26.28 dB PSNR and 0.1939 LPIPS, against 22.60 dB and 0.3449 for the best generic baseline, Restormer. Removing MCLRL or LRCS hurts.
  • Brightness, wall appearance, and partial occlusion are tested systematically, not only in the main table.
Critical weaknesses
  • All 18 baselines are generic restoration networks, not passive NLOS, computational-periscopy, or display side-channel methods. On Photo, LPIPS moves only from 0.8174 to 0.8167, so the five datasets are not uniformly improved.
  • The conclusion leaves other rooms, walls, and capture geometries to future work. The evidence supports feasibility on this capture setup, not transfer beyond it.
Questions for authors
  • What is the architecture and training data of the frozen clean-content encoder, and has it seen the test screen content?
  • Do train and test splits change room, wall material, or camera? If not, please state that these are single-setup results.
  • What fraction of cases fail on fine text or low luminance? Please show failures, not only the best visualizations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.