acceptodds
Withdrawn from ICLR 2027

Progressive Light-Confidence Representation Learning for Hidden Screen Reconstruction from Wall Light Patterns

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can an ordinary camera recover what a hidden screen displays by observing only the light it casts onto a wall? The task is governed by two coupled difficulties: (i) screen emission, ambient illumination, secondary transport, wall texture, and sensor noise mix in the wall pattern, so screen-consistent evidence varies across space; (ii) diffuse propagation attenuates high-frequency structure, leaving fine detail weakly represented and increasing reconstruction ambiguity. We introduce Progressive Light-Confidence Representation Learning (PLRL), which addresses these difficulties through ordered restoration and confidence-calibrated detail recovery. Its Physics-Guided Progressive Learning (PPRL) organizes wall evidence into coarse, intermediate, and fine scales and reconstructs layout, structure, and detail through bounded residual updates. Its Light-Propagation Confidence Representation (LPCR) combines scale compatibility, edge state, and image-wide context to identify reliable detail and calibrate its contribution. Across four screen-content domains, PLRL recovers the hidden display's brightness distribution, regional layout, and major contours, achieving the lowest LPIPS on every domain when compared with 24 restoration methods. Experiments spanning illumination, wall response, distance, and viewing changes establish the initial feasibility of recovering principal screen content from wall light captured by an ordinary camera.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Screen Recovery, Deep Learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_c2g2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors claim to propose a method to recover screen-displayed content not observable in direct view, from the light it casts on a wall. While three major contributions are listed, none of them have been convincingly justified or validated. The write-up is far from being clear and the paper as a whole is very difficult to follow.

Critical strengths

None. The methodology section is mostly filler. None of the claimed contributions have been justified or validated clearly.

Critical weaknesses

1. Fig.2 (the main method figure) is filled with gibberish labels: 'Soud Prompt Bank', 'niex-global statistics', 'Schel edges', 'Patter confidence', 'Global State Taken', 'T112 proxy-bins', 'RPI border Taper', 'cfip'. Its evidence panel shows meaningless numbers ('P 92.23', '0.0-142'). Fig.~2 also gives impossible tensor sizes, e.g.\ an output of '2×304×0122\times304\times012' and an intermediate of 304×212304\times212, while every dataset in the paper is on a 512×384512\times384 canvas. Every cross-reference reads 'Equation equation A.11' or 'Equations equation 7--equation 9'. The text is padded with near-identical boilerplate. Nearly every section restates 'PLRL leads all NN ... comparisons' and ends with generic filler ('connects', 'establishes', 'supports').

  1. Section 3 (describing their main method) is mostly filler. The method that was actually implemented is minimal and is hidden in Appendix A.2. Section 3 never says that its formulations are replaced, so a reader of Section 3 will believe the model does things it does not. Two cases stand out. First, the final detail d^\hat d is defined in Section 3 (Eq. 16) as the solution of an optimization over the whole image. What is implemented is simply d^=q⊙d\hat d=q\odot d, the proposed detail multiplied by the confidence at each pixel, with no optimization and no interaction between pixels. Eq. 16 is also wrong as written. Its weight (1−q)/(q+ϵ)(1-q)/(q+\epsilon) becomes small where confidence is high, so trusted detail gets smoothed away, and becomes large where confidence is low, so untrusted detail is forced through. That is the opposite of the stated goal. Second, the confidence is said to draw context from a 'propagation-aware' nonlocal kernel (Eqs. 12--13). What is implemented instead averages the feature stack over the whole image and passes this single vector through an unspecified learned map. The rest of Section 3 is similar. Eq. 3, the 'physics-constrained' objective, is presented as the problem formulation ('we formulate reconstruction as a constrained optimization problem') but is never solved, and it would need the ground truth at test time. Eq. 4 expands the wall image in a basis with 'transport response coefficients', but the implementation just blurs the wall image with two fixed filters to split it into three frequency bands, and has nothing to do with transport. Eq. 5 describes recovery as a continuous-time ODE whose terms are never defined, while the model simply applies three correction steps. Eq. 6 defines an attention-like operator Θa\Theta_a whose discrepancy χ\chi is never defined; the implementation (Eq. A.10) only says the corrections come from unspecified networks, so it is unclear whether or how Θa\Theta_a is used. Eqs. 7--9, the three bounded correction steps, are the real core, but their bounds α,β,γ\alpha,\beta,\gamma are never reported. Eq. 10, a penalty on how much each stage changes the image, is defined but never used. Eq. 11's per-pixel feature stack is replaced by a different one, and its terms DbD_b change from 'consistency between stages' to 'difference between the wall image and the final estimate'. Eq. 14's confidence objective becomes a plain L1 loss on edge magnitude instead of gradient error, with no smoothness term. In short, the paper takes credit for a complex system but implements a much simpler one, and even for that simpler model it leaves out critical details, so the method is not reproducible.

  2. The origin of the datasets is never stated. Nowhere does the paper say the authors collected Screen-2000, Screen-Game, Screen-Figure, OS-Lock or OS-Home. The contributions list does not claim a dataset, nothing is cited as a source, and there is no release statement. Some passages suggest the authors captured the data themselves: Appendix A.1 describes a Sony A7S~II rig and two capture configurations, and Fig. 1(c) and Fig. 7 show photographs. Other passages suggest a pre-existing dataset: Appendix A.3 says 'we combine its original 2,121 training pairs and 124 validation pairs into a training set of 2,245 pairs', and A.4 refers to 'merging its source training and validation partitions'. That implies Screen-2000 came with its own splits from somewhere else, yet no source is named. Meanwhile, the one validation split that did exist was folded into training, and checkpoints were selected on the test set instead (see last point).

  3. The reported numbers for PLRL are internally inconsistent. Appendix A.4 says the external-comparison (Table 1), main-ablation (Tables 2--3) and supplementary-ablation (Table 26) suites were trained independently, yet many PLRL values match across all of them to the last reported digit (OS-Lock 17.088/0.6273, Screen-Figure LPIPS 0.4712, Screen-Game 18.378/0.6743/0.4736). Other values differ: Screen-Figure PSNR is 20.336 in Table 1 but 19.336 (exactly 1.000,dB lower) in Tables 2, 3 and 26, and Screen-Figure SSIM is 0.7457, 0.7094, 0.7147 and 0.7136 across Tables 1, 2, 3 and 26, even though Tables 2 and 3 share a suite. Independent runs would not match to the last digit, and a single shared model would not give different numbers, so neither reading is consistent.

  4. Checkpoints were selected on the test set. Appendix A.4 states: `checkpoint selection monitored LPIPS on the test split, and the selected checkpoints were evaluated on that same split. No separate held-out validation set was used.' Every number in every table is therefore tuned on the data it is reported on and is optimistically biased. Screen-2000 even shipped with a 124-pair validation split, which was merged into training rather than used for selection. It is also not stated whether the baselines received the same test-set selection, so the comparison may not even be biased equally. Combined with the previous point, none of the reported numbers can be taken at face value.

Questions for authors

See under weaknesses.

Reviewer_H5MGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates the problem of reconstructing hidden screen content from diffuse wall light captured by an ordinary RGB camera and proposes the PLRL framework. By combining coarse-to-fine progressive restoration with confidence-guided detail calibration, the method recovers brightness distributions, regional layouts, and major contours across four screen-content domains.

Critical strengths
  1. The paper connects passive indirect imaging with visual information exposure, providing real-world acquisition configurations and paired wall-screen datasets that make the task formulation clear and concrete.

  2. Reconstructing stable low-frequency layouts before high-frequency details while bounding unreliable residuals aligns well with the physical degradation of diffuse light transport, supported by explicit multi-scale intermediate supervision.

  3. PLRL achieves consistent quantitative and qualitative gains across all benchmarks.

Critical weaknesses
  1. The paper does not clearly explain how the light transport model, SVD analysis, and constrained optimization concretely determine the network architecture. In particular, Eq. (3) depends on C(y)C(y) of the unknown ground-truth screen yy, leaving its inference-time handling and the estimation of the transport operator AξA_\xi unclear.

  2. Applying ωq=(1−q)/(q+ϵ)\omega_q=(1-q)/(q+\epsilon) to ∥δ−d∥22\Vert{}\delta-d\Vert{}_2^2 assigns a larger weight when confidence qq is low, which forces the calibrated residual δ\delta to stay close to the raw detail proposal dd. This directly contradicts the stated goal of suppressing unreliable details in low-confidence regions and conflicts with the amplitude shrinkage formulation in Appendix Eq. (A.21–A.22).

  3. Training separate models for each content domain only validates in-domain supervised reconstruction. It remains unclear whether a trained model can generalize to unseen rooms, wall materials, displays, or screen-to-wall geometries without costly recalibration or retraining. Moreover, selecting checkpoints by monitoring LPIPS directly on the test split without a held-out validation set weakens the rigor of the evaluation.

Questions for authors
  1. How are the transport operator AξA_\xi, ambient term bb, and structural operator CC in Eq. (3) obtained or estimated in practice, and how is C(y)C(y) handled during inference when yy is unknown? Which physical constraints actively participate in training or inference to guide PPRL?

  2. In Eq. (16), a lower confidence qq increases ωq\omega_q, enforcing stronger fidelity to the raw detail proposal dd. How is this consistent with suppressing unreliable details, and how does it reconcile with the amplitude penalty in Appendix Eq. (A.22)? Which formula corresponds to the actual implementation?

  3. How does the model perform in out-of-domain settings (e.g., cross-dataset evaluation) and under unseen physical setups (new rooms, wall finishes, displays, or geometries) without retraining? Additionally, how does selecting checkpoints directly on the test split (Appendix A.4) affect the objectivity of the reported results?

Reviewer_fgDqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies how to recover a hidden screen from a single RGB image of the light reflected on a nearby wall. The proposed PLRL has two main components. PPRL progressively reconstructs the layout, structure, and detail, while LPCR predicts a spatial confidence score to control the detail correction. Figure 2 also shows a Transformer encoder-decoder producing an initial reconstruction before these modules. The authors train separate models for four screen-content domains and compare with 24 restoration architectures adapted to this task. They report better PSNR, SSIM, and LPIPS, with particularly large gains on Screen-2000. There are also module ablations, same-backbone comparisons, and experiments under different observation conditions. The main goal of this work is to recover the broad appearance and layout of the screen, rather than to reliably read small text or recover every displayed detail.

Critical strengths
  • Recovering hidden screen content from an ordinary RGB observation is a useful computational imaging problem. The paper provides physical acquisition configurations and evaluates several types of screen content. I also appreciate that the authors mostly limit their claim to large regions, brightness distribution, and major contours, rather than claiming that all text or fine details can be recovered. Even this level of reconstruction can reveal useful information about a hidden display. I would not consider the coarse outputs alone as a reason to reject the paper.

  • Recovering the main structure first and then controlling uncertain detail is a sensible approach for this severely mixed observation. On Screen-2000, the reported improvement over Restormer is substantial: PSNR increases from 22.601 to 27.341 dB, and LPIPS decreases from 0.3449 to 0.1678. If these numbers hold under an independent evaluation, this would be a strong result.

  • The authors also made considerable effort in the experiments. In particular, the supplementary material includes same-backbone comparisons and controls that change the frequency-supervision schedule while keeping the architecture. These are more useful for understanding the contribution than only adding more external baselines. I give credit to these controls, although the evaluation issue discussed below also limits how confidently I can interpret them.

Critical weaknesses
  • Figure 2 shows a Transformer encoder-decoder that produces an initial reconstruction before PPRL and LPCR. But eq. (7)-(10) and eq. (A.11) start the progressive reconstruction from x_l, the low-frequency wall observation. Where does the initial reconstruction enter these equations? Is it included inside a learned correction operator, or should it replace the initial state? A backbone can of course be included in an abstract learned operator, so not writing it as a separate symbol is not automatically wrong. The issue is that the paper does not explain this mapping. Without it, I cannot tell what the residual bounds apply to, or which part performs the main wall-to-screen reconstruction.

  • Appendix A.4 explicitly states that checkpoint selection monitored LPIPS on the test split, and the selected checkpoints were evaluated on the same split. There was no separate validation set. This is a concern for a paper whose main evidence is the reported reconstruction improvement. The reported test LPIPS is also the quantity used to choose the model. Reporting PSNR and SSIM on those selected checkpoints does not make the evaluation independent either. Even if all methods used the same selection rule, this would not make the test set independent. A comparison using validation-only checkpoint selection and an untouched evaluation set is needed.

  • The brightness results also need a more careful interpretation. On Screen-2000, PSNR drops from 27.341 dB at 300 nit to 15.312 dB at 250 nit, and to 20.272 dB at 350 nit. Being better than the other methods at these settings is different from being robust to the change. The drop is substantial. At the same time, it does not by itself prove template memorization or train/test overlap. The acquisition procedure and radiometric normalization need to be clear before drawing such a conclusion.

  • I would also recommend substantially shortening and reorganizing the paper, especially the appendix. The submission is 50+ pages including references and supplementary material. I do not consider this length alone a reason for rejection, since the main paper is nine pages. However, essential method and evaluation details are spread across different parts, while several robustness sections repeatedly describe similar findings through the table, curve, and example discussions. This makes careful review unnecessarily difficult without helping much to explain the main contribution. For a conference paper, one complete description of the forward computation, training losses, and model-selection protocol should be easy to find in the main text. Much of the repeated discussion in the appendix could be combined into shorter summaries and compact tables. This would also help the authors present a clearer technical story.

Questions for authors
  • Were the Figure 4 ablation models trained separately, or were the modules bypassed within the same trained checkpoint? It would be helpful to report the initial and final reconstructions from the same full-model checkpoint, together with the relevant retrained control. I do not expect a small residual bound to limit the difference between separately trained models. The purpose is to separate the direct effect of refinement from the effect that these modules and losses have on backbone training.

  • For the brightness, wall-response, and camera-position experiments, which conditions were physically recaptured and which were obtained by transforming recorded observations? For the change from 300 to 250 nit, please clarify the camera exposure, ambient-light treatment, and radiometric normalization. Does an appropriate gain-compensation control recover much of the lost performance? Please also state whether the occluder geometry in Screen-2000 is shared between training and testing. These details would help distinguish robustness within a fixed setup from generalization to a different physical acquisition condition.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.