acceptodds
Withdrawn from ICLR 2027

Evidence-Budgeted Light Transport Representation Learning: Wall-Reflected Screen Content Recovery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can an ordinary camera recover hidden screen content from the diffuse light that the screen casts onto a wall?We address this question with Evidence-Budgeted Light Transport Representation Learning (EBLTR). Wall transport preserves screen information unevenly across locations, scales, and frequency bands, making the reliability of surviving evidence central to reconstruction.Our Light Transport Observability Representation (LTOR) combines frequency-separated responses, local wall statistics, and multilevel features to estimate this evidence.Evidence-Budgeted Learning (EBL) converts the estimate into a local correction budget, so the amount of refinement follows the available observational support.A frozen screen-to-wall surrogate further checks observation consistency during training.Across four screen-content categories and varied observation conditions, EBLTR outperforms 19 reconstruction baselines and recovers screen colour, regional layout, and local structural cues.The reconstructions reduce erroneous edges, textures, interface contours, and text strokes that lack wall-observation support.These results demonstrate the preliminary feasibility of recovering hidden screen content from ordinary-camera recordings of wall illumination, with reconstructed detail guided by the evidence that survives light transport.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Passive Computational Imaging, Wall-Reflected Screen Content Recovery, Light Transport Observability, Evidence-Budgeted Learning, Single-Image Reconstruction, Image Restoration.

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_eJRHOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a method to recover the content from a monitor screen that is reflected onto a wall. The pipeline combines various hand-crafted features and prior biases (such as weighing low-frequency information higher in the prediction) and consists of two main components (LTOR, EBL) and an image-to-image backbone, which is used twice in a single inference. The technical contributions seem quite obscure due to the writing that mostly details low-level implementations with overwhelming notations. The validation is done on four content categories using standard metrics like PSNR, SSIM, LPIPS and compared against baselines designed for general image-to-image tasks. Ablation studies on the two components are provided and experiments on different resampled inputs and masked inputs. The reported scores outperform the baselines; however, the evidence is not convincing.

Critical strengths
  • An interesting and difficult problem that can potentially reveal privacy issues.

  • Extensive effort to include various baselines and numerous ablation studies, such as grid occlusion, replacements, prior variants, etc.

Critical weaknesses
  • The experimental setups and results may only reflect memorization, not that the light transport is effectively learned. The proposed method is never evaluated across categories, and even within the same categories, it is unclear how much content overlaps between the training and testing frames. The results in Fig. 3 show accurate color reproduction and fine details that are extremely unlikely to be recovered from such a diffuse wall input. This seems to suggest that the method simply memorizes category-specific appearances and uses a minimal signal to retrieve those memorized visuals, rather than learning the inverse light transport or the wall-to-screen mapping.
  • The method relies on relatively complex, hand-crafted heuristics, which mostly lack strong justification in the era of modern learning. For example, the two-time inference via the backbone is not well motivated, and the method could incorporate all input signals in one pass and still utilize confidence prediction and refinement without the two passes. The ordered band prior is simply hard-coded and not learned, and the learned correction might make it even irrelevant (Table A4 shows that reversed band prior even yields better PSNRs on all datasets than no prior at all). While handling scales, frequencies differently or utilizing local statistics sounds sensible, manually handling them seems unnecessary, even suboptimal, in the modern learning paradigm. Most critically, the framework does not model any physical relationships, such as the relative positions of the screen, wall, camera, and simply shares the input wall and output screen coordinates, leading the networks to learn a fixed mapping that fails to model generalizable light transport.
Questions for authors
  • Were the baselines’ hyperparameters properly tuned or simply taken from the default values, which were meant for different tasks? Were they trained from scratch or fine-tuned? How were some baselines adapted for the task? The details on the training and evaluation of the baselines are insufficient, making the comparison unconvincing.
  • Have statistical tests been performed? Are the improvements statistically significant, especially for the small gains from key components like LTOR, EBL (Table 2)?
Reviewer_rXkCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Evidence-Budgeted Light Transport Representation Learning (EBLTR), a framework designed to recover hidden screen content from diffuse light reflected onto a wall using an ordinary camera. It utilizes a Light Transport Observability Representation (LTOR) to estimate local observational support by analyzing frequency responses, local wall statistics, and multilevel features. This estimation is fed into Evidence-Budgeted Learning (EBL), which bounds the amount of local correction added to the screen estimate based on the surviving wall evidence. The model also employs a frozen screen-to-wall surrogate during training to ensure observation consistency.

Critical strengths

The model comprehensively outperforms 19 reconstruction baselines across four diverse screen-content categories (Screen, Game, Lock, Home).

By strictly bounding refinement to available evidence, EBLTR successfully reduces erroneous edges, hallucinatory textures, and unsupported text strokes.

The paper provides extensive robustness evaluations, proving the framework maintains strong performance under varying wall brightness, distance, sampling resolution, Gaussian noise, camera jitter, horizontal/vertical viewing angles, and wall textures.

Critical weaknesses

The experimental data relies heavily on tightly controlled indoor geometries, such as a fixed 0.5 m display-to-wall separation and the specific placement of a T-shaped baffle for the occlusion scene.

The training pipeline depends on a phenomenological sensing model and a frozen surrogate that absorbs spatial propagation and diffuse reflection, which may lack flexibility if ambient conditions dynamically change during real-world inference.

Questions for authors

How does the bounded correction mechanism in EBL adapt if the screen-to-wall distance is varied continuously, rather than operating within the controlled 0.5 m setup?

Would it be possible to unfreeze and dynamically update the screen-to-wall surrogate during inference to adapt to unknown room geometries?

The EBL model has been validated through a highly suitable and excellent method as a Proof of Concept, successfully demonstrating that its mathematical constraints work as intended to mitigate hallucinations. However, since the screen-to-wall distance continuously varies and room geometries change unpredictably in real-world environments, there is still a lack of empirical dataset validation to confirm whether the model can flexibly adapt to such dynamic conditions.

Reviewer_NSroOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a system to reconstruct hidden screen content from light reflected onto a wall from RGB inputs. Its main contributions include a framework called the Light Transport Observability Representation to combine frequency bands, local statistics and backbone features to estimate reconstruction reliability, as well as an Evidence-Budgeted Learning strategy that uses that estimate to bound local residual corrections. The authors evaluate four content categories against several baselines. Results reveal the advantage over existing methods.

Critical strengths
  1. The problem of recovering hidden display information from indirect illumination is very meaningful.
  2. The method improves several image quality metrics, so the reported results are promising.
  3. Comparions against existing methods are well controlled.
Critical weaknesses
  1. It is quite unclear whether this method is generalizable to more scenes.
  2. The EBL mechanism is clear, but the mathematics novelty (Line 243-245) is limited.
  3. There are a lot of parameters, some of them were set manually. It seems missing selection rationale and sensitivity analysis.
  4. Some figures seemed to be generated using AI tools, the authors should clarify this clearly.
Questions for authors
  1. Four types of typical scenes are tested. However, there are a lot of cases that can be shown on screen. Is this method generalizable to more scenes?
  2. Some of the parameters were set manually. For example, alpha is set to 0.06. How was the value determined?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.