acceptodds
Withdrawn from ICLR 2027

Weak Light as Structural Evidence: Relay-State Hierarchical Representation Learning for Hidden-Screen Reconstruction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We investigate whether an ordinary camera can recover hidden display content from the weak light that the display projects onto a visible relay wall while direct viewing is blocked. Diffuse reflection, spatial spread, and propagation attenuation disperse and mix the brightness, edges, and spatial relations of screen structures, so the wall response carries these cues in a rearranged form. We present Relay-State Hierarchical Representation Learning (RSHRL), a two-pass framework that first organizes this structural evidence through Relay-Wall Invertibility Representation (RWIR) and then repairs residual damage through Damage-State Learning (DSL). RWIR rebuilds wall-to-screen structural correspondence by combining fine evidence, regional relationships, and coarse screen context. DSL estimates spatial and channel damage states from the preliminary reconstruction and its associated features; the second-pass consumer uses these states through its internal DACA, DASA, and SDR operations to refine the affected structures. Experiments on four paired screen-content datasets show that RSHRL leads the evaluated restoration baselines across all 12 PSNR, SSIM, and LPIPS comparisons. Removal and replacement studies demonstrate the complementary value of hierarchical representation and damage-guided repair. Across two acquisition setups and varied observation conditions, the reconstructions recover principal brightness distributions, text-bearing regions, icon contours, window boundaries, and overall interface layouts. These results establish the initial feasibility of recovering hidden screen content with an ordinary camera by observing only wall-reflected display light.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Passive relay-wall computational imaging, ill-posed inverse problems, hierarchical representation learning, state-conditioned image restoration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ETkVOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a framework that recovers hidden display content from the weak light reflected off a wall. More precisely, it presents a two-pass framework, Relay-State Hierarchical Representation Learning (RSHRL), which consists of a Relay-Wall Invertibility Representation (RWIR) and Damage-State Learning (DSL). The authors additionally contribute four self-collected paired datasets, 8,777 wall-observation and screen-image pairs in total, captured under two different acquisition setups.

Critical strengths

1.

I think the data contribution is substantial: four paired datasets totaling 8,777 pairs of a wall observation and the screen image that produced it, captured by the authors themselves under two acquisition setups. As far as I know, measured benchmarks for this task (recovering a hidden screen from the light it casts on a wall) are scarce, so this collection could be the basis for recovering hidden display content from the weak light.

Critical weaknesses

1.

To me, the central feasibility claim (that hidden screen content could be recovered by reading the wall-reflected light) does not seem separated from a simpler explanation: the model may largely have memorized what screens in each dataset typically look like, and draws that, rather than reading the signal of the wall. The setting seems favorable to such memorization: training and testing use the same acquisition setup (Appendix A) and the same family of screen content (desktop/UI, lock screen, or game/HUD; Table 5).

And as far as I could find, how the train/test split was made is not described (Table 5 gives counts only), so visually near-identical screens could exist both sides of the split.

I also could not find a control experiment that measures how much of the reconstruction comes from the wall signal versus this memorized prior. The paper's robustness table seems to show the problem: averaged over the clean condition and five Gaussian-noise levels, RSHRL's scores are almost unchanged from clean (14.52 to 14.53 PSNR on OSHome, 16.74 to 16.77 on OSLock; Tables 1 and 3). That is, if the input is heavily corrupted but the output score barely moves, it looks to me like the model is not really reading the input, which is the behavior a memorization-driven predictor would show. The regional-masking study (Appendix C.3.4) is qualitative and does not quantify this, as far as I can tell.

The conclusion's "initial feasibility of recovering hidden screen content" depends on showing that the wall-reflected light, not the model's memorization, drives the result. Thus, I think this weakness is the main reason for my recommendation.

2.

Second, the comparative evidence does not seem strong enough for its stated claim. As far as I checked, every number appears to come from a single training run with no variance estimate, and on three of the four datasets the margin over the best of 20 baselines is only 0.18 to 0.30 dB PSNR (14.52 vs. 14.34 on OSHome; 16.74 vs. 16.54 on OSLock; 18.45 vs. 18.15 on Game; Table 1).

In my experience, gaps of this size could easily flip with a different random seed. But the abstract claims "RSHRL leads the evaluated restoration baselines across all 12 PSNR, SSIM, and LPIPS comparisons." In my opinion, one run each for 21 models could not establish such a ranking. To claim a performance improvement with gaps of this size, I think at least three seeds are needed, with per-seed values and the variance reported.

In particular, given that training details such as the batch size or learning rate are not reported, it seems difficult to be confident that these gains are reliable.

Questions for authors
  1. In Table 2, the "w/o RWIR" and "w/o DSL" variants score 16.73 and 16.11 dB on Screen, below several standalone baselines in Table 1 (for example, AdaIR at 21.71 dB (2nd best)). In my understanding, a variant that merely lacks one proposed component should still behave like an ordinary restoration network (i.e., I would expect the "w/o RWIR" and "w/o DSL" variants to score at least around 21 dB, on par with the stronger standalone baselines). So when it falls about 5 dB below off-the-shelf methods trained on the same data, it looks to me as if the removal degraded the network as a whole rather than isolated the component. Measured against such a weakened reference, the +9.22 dB gap would mostly reflect recovering from that damage, not the value RWIR adds over a sensible alternative (against AdaIR, for instance, the margin is +4.24 dB). Could you explain what makes the removed variants fall this far below ordinary baselines (for example, lost capacity, or some functions the remaining modules depend on)?
  2. Could you report an input-blind control (for example, feeding an all-zero or constant gray image instead of the wall observation) or a mismatched-pair control (for example, evaluating the output for sample A's wall observation against sample B's target), with the resulting PSNR/SSIM/LPIPS?
  3. Could you provide the training configuration (optimizer, learning rate, schedule, iterations, batch size, parameter count, hardware) for RSHRL and for each retrained baseline (at least for RSHRL)? As far as I could find, none of this is reported.
  4. I was confused by several baseline numbers that are so low that, to me, they suggest those models did not train properly (DarkIR at 6.57 and 4.46 dB, SFHformer at 8.82 dB on Screen, DeepSN-Net at 10.48 dB; Table 1). Could you share evidence that these baselines converged under your protocol?
  5. How were the train/validation/test splits constructed for each dataset (Table 5 reports counts only)? And roughly what rate of near-duplicate screen content (same desktop layout, wallpaper, or HUD) appears across splits?
  6. Could you explain the margin asymmetry between Screen (+4.24 dB) and the other three datasets (+0.18~+0.30 dB; Table 1)?
  7. Could you clarify what this task is intended to be useful for (for example, a privacy or eavesdropping risk study in Kuhn (2002))? As far as I could find, the paper motivates it only as a feasibility question, and the significance of the result seems to depend on this.

Minor issues:

  • Several acronyms are used before they are defined: DACA, DASA, and SDR in the abstract, and CRT and NLOS in the introduction. Please spell them out at first use.
  • Figure 2 carries too much explanatory text inside the figure itself. In my opinion, the parts that repeat the caption or the main text could be removed so that only the essential labels remain.
  • Some passages repeat almost the same sentences. For example, Appendix C.5.1 contains two nearly identical paragraphs in a row ("On the same Screen example, the full configuration preserves..." followed by "On the same Screen example, the full model preserves..."). Removing such repetition could make the paper easier to follow.
Reviewer_73jLOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies passive hidden-screen reconstruction, which aims to recover screen content from the display light reflected off the wall. The authors propose RSHRL, a two-pass framework. In the first pass, a hierarchical representation module (RWIR) organizes the wall observation into multi-scale features and produces a preliminary reconstruction. A damage-state module (DSL) then estimates a scalar severity score from that reconstruction. In the second pass, the network uses the first-pass reconstruction, the RWIR features, and the severity score to refine the output.

Critical strengths

The paper studies an interesting and practically relevant problem, recovering screen content from wall-reflected light with an ordinary camera. The authors also built real-world capture setups and collected four paired datasets across two acquisition configurations, which may benefit the research community.

Critical weaknesses
  1. The results on the Screen dataset are surprisingly strong. On OSHome, OSLock, and Game, RSHRL beats the best baseline by only 0.18–0.30 dB PSNR. On Screen, however, it gains more than 4 dB (25.95 vs. 21.71 dB), and Figures 3 and 4 show it recovering fine-grained details and region shapes almost exactly from a nearly featureless wall observation. Can the authors explain why this happens? Is the Screen setting itself easier, or do its train and test sets share near-duplicate content?

  2. All the compared baselines are generic image restoration networks that are not designed for this task. NLOS-OT addresses a nearly identical setting and it also released a public dataset. It would be more convincing if the authors compared with NLOS-OT and evaluated on its dataset.

  3. Some baselines appear not to be properly trained. For example, in Figure 4, pix2pix and U-Net reconstruct almost nothing, and their PSNR in Table 1 is far below other baselines. Some training details of the proposed method and baselines are missing, which may hurt the reproducibility.

Questions for authors
  1. Can the author discuss why the results on Screen is much stronger than other datasets?
  2. If would be more convincing if the authors can provide comparison with NLOS-OT.
Reviewer_UPdXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies passive hidden-screen reconstruction: recovering what a display shows from the weak light it casts on a relay wall, using only an ordinary camera, even when the display is out of the camera’s direct view. The authors propose RSHRL, a two-pass framework in which RWIR builds a hierarchical representation and a preliminary reconstruction, and DSL estimates damage severity to condition a second-pass refinement. On four paired screen-content datasets (Screen, OSHome, OSLock, Game), it ranks first in all 12 PSNR/SSIM/LPIPS comparisons against 20 restoration baselines. The authors conclude that this establishes initial feasibility.

Critical strengths
  • The evaluation spans four paired datasets covering the blocker-assisted setup and the behind-screen wall-reflection setup, with content types including desktop, lock screen, and game screens.

  • The paper provides an image-formation model (Eq. 1) that explains why spatial spreading and attenuation make the inverse problem ill-conditioned.

Critical weaknesses
  • The proposed modules do not demonstrate the capabilities they are claimed to have. RWIR is described as rebuilding wall-to-screen structural correspondence, and DSL as estimating the damage state of the preliminary reconstruction. However, the operations behind these modules are standard (e.g., pooling and depthwise projection, coarse-to-fine synthesis with bounded residuals, channel and spatial attention), and nothing in the architecture or the loss ties them to the claimed roles. The image-formation model in Eq. (1) is not used in the method or the loss (only a masked ℓ1), and the main text does not explain what supervises DSL or how severity enters the refinement modules. The ablations do not isolate the proposed ideas either: the replacement variants are weak, internally defined alternatives, and there is no control, such as a standard multiscale decoder of similar capacity or a two-pass model without DSL, so the gains cannot be attributed to the design rather than to added capacity and a second pass.

  • This paper has limited practicality and generalizability. Each dataset is trained and tested separately, with no cross-dataset or cross-setup evaluation. Because the image-formation kernel depends on the camera, wall, blocker, and distances, a model trained for one configuration is not guaranteed to work when these change. Retraining on paired (wall observation, ground-truth screen) data for every new environment conflicts with the premise that the screen cannot be viewed directly, and the intended use case is never stated.

  • Many of the 20 baselines appear to be restoration models for pixel-aligned input-output degradation, whereas wall-to-screen is not a pixel-aligned problem, so these models may not be suited to this task in the first place. Passive NLOS methods cited in the related work are not compared.

Questions for authors
  • What does DSL actually output? Section 3.3 states that DSL outputs a single scalar s and introduces no additional state maps, whereas Figure 2 shows S_spatial, S_channel, and S_structure. Which is the final implementation?

  • Who supervises DSL? DSL is said to estimate the damage state, but the only loss appears to be a masked ℓ1 on the final output. Is there any analysis showing that s (or the S maps) is actually larger where the reconstruction is more damaged?

  • The image-formation kernel depends on the camera, wall, blocker, and distances. How does performance change when the camera-wall or screen-wall distance, the positions, the wall material, or the blocker placement vary? What happens when a model trained on one dataset is tested on another dataset?

  • Training requires paired data (wall observation and ground-truth screen), but in a real-world setting where the screen cannot be viewed directly, such pairs cannot be collected. How would the method be used in that case, and what is the intended use scenario?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.