acceptodds
Under review as a conference paper at ICLR 2027

Can Reflected Light Reveal Screen Content ? Learning Degradation-Routed Evidence Representations for Hidden Screen Content Recovery

Abstract

Can an ordinary camera recover what a hidden screen displays from the light it casts onto a wall? Propagation and diffuse reflection attenuate and spatially mix the screen signal, while surface texture, ambient illumination, and camera imaging further alter the observation. The resulting faint glow contains screen cues entangled with unrelated appearance. To recover screen content from this mixed glow, we propose Reflection-Evidence Routed Representation Learning (RERL). Consensus-Driven Evidence Representation Learning (CDERL) organizes screen-related cues through encoder-decoder agreement and masked prototype transport. Degradation-Aware Routing (DAR) combines complementary expert transformations with spatial guidance to adapt reconstruction to uneven degradation across the reflection. Across four screen-content groups, RERL outperforms 24 image-restoration baselines in PSNR, SSIM, and LPIPS. Component ablations reveal the complementary contributions of evidence organization and adaptive reconstruction, while perturbation tests and visual analyses demonstrate recovery under varied observation conditions. The reconstructions retain screen layouts, colour regions, and major contours, establishing the feasibility of recovering hidden screen content from wall reflections captured by an ordinary camera.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Screen Reconstruction, Deep Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_p4ZTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies recovery of the current front-view RGB screen content from a camera observation of a wall or door illuminated by the hidden display. It focuses on two core challenges: useful screen evidence is compressed and mixed with nuisance appearance. To recover screen content from this mixed glow, the paper proposes Reflection-Evidence Routed Representation Learning (RERL), a recovery path designed to promote evidence alignment and degradation-aware feature. Across four screen-content groups verifies the effectiveness of the proposed RERL.

Critical strengths

1.The writing is good 2.The research task is intriguing, but the methodology is relatively conventional.

Critical weaknesses
  1. Figure 2 exhibits obvious AI-generated artifacts and needs to be redesigned to provide a complete and proper pipeline overview of the proposed method.

  2. The literature review in the related work section is outdated, with references extending only up to 2023. The authors should include recent literature, such as [1–10], and incorporate them into the comparative analysis in Table 1.

  3. The paper lacks the required AI USE STATEMENT.

[1] Tang N, Nie S, Luo X, et al. Degradation-Consistent Test-Time Adaptation for All-in-One Image Restoration[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026: 15476-15485.

[2] Jo S, Ko D, Kang J, et al. Flow Straight to Reality: Perceptually Consistent Flow Matching for Efficient Image Restoration[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 223-239.

[3] Zhou S, Zhang J, Huang W, et al. QuReC: All-in-One Image Restoration with Query-Specific Guidance and Local-Global Response Calibration[J]. arXiv preprint arXiv:2607.15097, 2026.

[4] Lee H, Kang S, Yu J, et al. Tasktok: Delving into task tokens for task-driven image restoration[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 447-464.

[5] Lee H, Son M, Yu J, et al. LogicIR: Logic Gate Networks for Image Restoration[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026: 394-411.

[6] Zheng Y, Li W, Liu Z. Image Restoration via Diffusion Models with Dynamic Resolution[J]. arXiv preprint arXiv:2605.14267, 2026.

[7] Xiao F, Hu P, Xu L, et al. Beyond ground-truth: Leveraging image quality priors for real-world image restoration[J]. arXiv preprint arXiv:2603.29773, 2026.

[8] Chen I, Hadji I, Sanchez E, et al. Restore, Assess, Repeat: A Unified Framework for Iterative Image Restoration[J]. arXiv preprint arXiv:2603.26385, 2026.

[9] Dong G, Li C, Ren C, et al. Learning domain-aware task prompt representations for multi-domain all-in-one image restoration[C]//International Conference on Learning Representations. 2026, 2026: 133754-133777.

[10 ] Zeng X, Yu Y, Xiong S, et al. ShiftLUT: Spatial Shift Enhanced Look-Up Tables for Efficient Image Restoration[J]. arXiv preprint arXiv:2603.00906, 2026.

  1. In the ablation study (Table 3), the authors should provide experimental results where both CDERL and DAR are simultaneously removed.
Questions for authors

Q1: What is the core novelty of the proposed method?

Q2: Compared to the latest literature [1–10], does the proposed method still achieve superior/state-of-the-art performance across Lock, Game, Screen, and Figure?

Q3: What are the experimental results when CDERL and DAR are simultaneously removed?

Please refer to the "Critical Weaknesses" section for further details.

1 comment
AuthorsOpenReview ↗

Response to Reviewer p4ZT

Thank you for recognizing the task. The revised paper has been uploaded, and we answer each point.

Q1. Core novelty.

Restoration networks, including the ten 2026 methods in your list, start from an input that is registered to its target and condition on a degradation type, an image domain, or a quality prior. In a wall reflection, the screen content reaches the wall through an unknown transport that mixes it spatially, and the strength of that mixing changes across the image and from capture to capture. CDERL keeps the feature content on which encoder and decoder agree, and DAR estimates the mixing per image and per region and routes and spatially guides the experts on that estimate.

Controls replace the modules at the same locations with gated cross-attention, a VQ bridge, and a four-expert global mixture. Parameters and FLOPs differ by at most 1.6% from RERL, and all use RGB L1 only, with mean±SD over three seeds.

Variant,RGBL1onlyScreenPSNRGamePSNR
Cross-attention24.638±0.17918.302±0.066
VQbridge24.578±0.21318.248±0.086
4-expertglobalMoE24.600±0.19718.318±0.063
RERLarchitecture25.878±0.15518.607±0.065

On the same test content at two mixing strengths, the margin over a concatenation bridge grows with the mixing.

MixingConcatRERLGain
Weaker25.73626.2990.563
Stronger22.61324.0981.485

Q2. Recent literature and comparative performance.

Table 1 compares with AdaIR (ICLR 2025), DarkIR and HVI-CIDNet (CVPR 2025), DeepSN-Net and DExNet (TPAMI 2025), and DEA-Net (TIP 2024), and Table 3 uses UniRes (ICCV 2025).

We adapted all ten methods to paired reconstruction and trained them on our reflection-target pairs with the training schedule, splits, and full test sets used for Table 1. The three all-in-one methods, [1], [3], and [9], reach the level of the Table 1 baselines.

PSNR/SSIM/LPIPSLockFigureGameScreen
[1]DCTTA15.3/0.548/0.65917.9/0.693/0.60617.5/0.651/0.58619.3/0.718/0.468
[3]QuReC15.3/0.554/0.66718.6/0.691/0.57117.6/0.638/0.59019.6/0.725/0.458
[9]DATPRL-IR15.6/0.576/0.67318.6/0.721/0.58517.6/0.662/0.59020.2/0.748/0.441
RERL17.1/0.620/0.63019.5/0.723/0.54618.7/0.681/0.54226.7/0.868/0.189

The other seven build on generative priors, quality priors, tokenizers, or lookup operators learned on natural images, and their PSNR on this task is lower.

Method,PSNRLockFigureGameScreen
[2]PCFlow12.617.610.515.0
[4]TaskTok12.312.59.89.5
[5]LogicIR10.06.06.09.4
[6]SubDAPS14.617.816.215.1
[7]IQPIR12.513.516.510.0
[8]RAR14.517.615.915.0
[10]ShiftLUT12.614.010.811.4
RERL17.119.518.726.7

RERL leads all ten on every group in PSNR, SSIM, and LPIPS. [1] and [9] are the closest to our setting and are now cited in the revised related work.

Q3. Simultaneously removing CDERL and DAR.

With both modules removed, PSNR falls below the full model by 1.321/0.819/1.447/8.715 dB on Lock/Figure/Game/Screen, and the backbone is below the full model on every group in all three metrics. The row is now the first row of the revised Table 3.

Figure 2 and AI use statement. Figure 2 has been redrawn as a complete pipeline overview in three panels, in which each block carries its equation, Eqs.(2)–(5), or its section of Appendix C. The original Figure 2 was rendered from SVG to PNG directly using Python's CairoSVG, which caused artifacts. This time as well, but we identified and resolved the issue with this conversion. The required AI use statement has been added to the revised paper.

We ask that the rating be reconsidered in light of these additions.

Reviewer_eT4sOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

RERL reconstructs coarse hidden-screen structure from a single RGB image of its diffuse reflection. CDERL uses encoder-decoder feature agreement and masked soft prototypes to refine reconstruction features. DAR combines input-dependent expert weighting with spatial guidance. Experiments cover four content groups and 24 restoration baselines, with component removal, replacement, and controlled perturbations. On Screen, RERL improves PSNR from Restormer's 22.601 dB to 26.699 dB, SSIM from 0.777 to 0.868, and LPIPS from 0.345 to 0.189.

Critical strengths
  1. RERL uses CDERL's encoder-decoder agreement to refine features before DAR, then adds masked soft prototype residuals later in decoding. DAR applies input-dependent expert mixing and spatial correction. The design adapts both the information passed through decoding and its processing. MCNN (Wang et al., 2020) uses multiresolution output supervision, while NLOS-LTM (Zhang et al., 2025) uses transport-condition codes to modulate reconstruction; RERL's local prototype mixtures instead supplement the evolving reconstruction features.

  2. Table 1 reports a 4.098 dB PSNR gain over Restormer on Screen, with improvements in SSIM and LPIPS. In Table 3, removing either CDERL or DAR reduces PSNR on all four groups. On Screen, the full model also exceeds the concatenation and metric-latent bridge replacements by 1.435 and 1.429 dB. Tables 4-8 test internal components and common-backbone alternatives, providing evidence for the representation and routing choices beyond the comparison with restoration baselines.

Critical weaknesses
  1. Equation (2) incorporates the encoder feature into the fused decoder feature before their learned projections are compared in Equation (3). This creates a potential feature-reuse pathway: responses shared by both branches could include nuisance as well as screen structure. Appendix C.3 directly supervises agreement and confidence with target-derived evidence, and the learned projections and gate may suppress nuisance. The shared-input structure therefore does not establish a failure, but leaves a mechanism question about how agreement favors screen evidence over shared interference. The evidence-selection interpretation is less directly supported than the reconstruction improvements.

  2. Table 3 replaces the four-expert DAR with a two-expert global mixture, changing both spatial guidance and expert count. Its 1.563 dB gap on Screen cannot be attributed to spatial guidance alone. Table 5 includes a 'w/o Local Assign.' variant, but its disabled operations are not specified. It is therefore unclear whether this internal ablation provides a matched control that resolves the expert-count confound, limiting attribution of the gain specifically to spatial guidance.

  3. The PSNR gains over the best compared baseline are 4.098 dB on Screen, but 0.330, 0.559, and 0.108 dB on Lock, Game, and Figure. On Game, Table 3's concatenation and global-only replacements are only 0.017 and 0.018 dB below the full model. Under the strongest tested digital one-sided illumination in Table 20, RERL falls to 10.431 dB, below DEA-Net at 11.383 dB. The specific module designs thus provide little additional benefit on some content groups, and RERL loses its advantage over DEA-Net under the strongest tested illumination perturbation.

Questions for authors
  1. How do the learned projections and target-derived supervision in Appendix C.3 address potential shared-nuisance matching after the skip fusion in Equation (2)? Optional diagnostic: compare queries formed before and after the current skip fusion, keeping auxiliary supervision fixed.

  2. Which operations are disabled in Table 5's 'w/o Local Assign.' variant? Are the four expert branches and auxiliary losses otherwise unchanged?

  3. Why are the gains over the simple bridge and routing replacements in Table 3 much larger on Screen than on the other content groups?

  4. Do existing degradation-map or routing analyses explain the loss of PSNR advantage over DEA-Net at the 24 and 30 illumination settings in Table 20?

4 comments
AuthorsOpenReview ↗

Response to Reviewer eT4s

Thank you for the detailed assessment. We answer the four questions in order.

Q1. How agreement favors screen evidence.

Agreement is computed after the skip fusion in Eq.(2). The query and key use separately learned projections and normalization, and their product is transformed and modulated by a learned confidence gate. Appendix C.3 gives a target-based selection criterion in which target evidence combines luminance, Sobel variation, and saturation, a Smooth L1 term matches standardized agreement to this evidence, and its min–max-normalized form supervises the confidence. In this way the projections and gate are optimized for screen structure alongside RGB reconstruction. Section 3.2 of the revised paper now states this.

Additional controls use identical RGB L1 supervision, with parameters and FLOPs within 1.6% of RERL. Values are mean±SD over three seeds.

RGBL1onlyScreenPSNRGamePSNR
Gatedcross-attention24.638±0.17918.302±0.066
VQbridge24.578±0.21318.248±0.086
4-expertglobalMoE24.600±0.19718.318±0.063
RERLarchitecture25.878±0.15518.607±0.065

The architectural advantage holds under common supervision.

Q2. Local assignment and expert-count control.

The 4-expert global MoE row of the controls under Q1 is the matched control for expert count. It has four experts, parameters and FLOPs within 1.6% of RERL, and the same supervision, and RERL stays above it on Screen and Game across three seeds.

In Eq.(4) the spatial guidance is the residual P_l, and the four experts with their input-dependent weights form the image-level mixture. The w/o Local Assign. variant of Table 5 sets P_l to zero and keeps the four experts, the input-dependent global weights, the degradation estimator, and the auxiliary objectives.

Q3. Variation across content groups.

Appendix B.1 describes Screen as interface layouts and Game as richer texture with abrupt colour transitions. The joint ablation added in this revision, with CDERL and DAR both removed, gives the contribution of the two modules on each group. Over this backbone the two modules together add 1.321/0.819/1.447/8.715 dB on Lock/Figure/Game/Screen, so the designs contribute on every group. The gain is largest on Screen, where interface layouts leave the most recoverable structure in the reflection. On Game the matched controls under Q1 stay below RERL across three seeds.

Q4. One-sided illumination at 24 and 30.

Against DEA-Net, RERL leads in PSNR through intensity 18 (15.649 against 14.593 dB) and in LPIPS through intensity 24 (0.609 against 0.649). Under Eqs.(4),(9)–(11), a strong one-sided gradient enters both the pooled routing descriptors and the local degradation cues, which changes cue selectivity at the two strongest settings. Appendix F.5 of the revised paper now states this.

We ask that the rating be reconsidered in light of these answers.

Reviewer_eT4sOpenReview ↗

Post rebuttal

Thank the authors for the clarifications and additional experiments. The local-assignment explanation and four-expert control address my ablation concern. But after reading and considering the comments from the other reviewers and your rebuttal to them, I share their concern about novelty, say the comparisons show that the proposed combination works, but its methodological advance over existing agreement, prototype, and routing mechanisms remains unclear.

AuthorsOpenReview ↗

Reply to Reviewer eT4s(1/2)

Thank you for reading the whole discussion, and for confirming that the local-assignment explanation and the four-expert control settle the ablation concern. The remaining point is how the method advances over existing agreement, prototype, and routing mechanisms. This question runs through all three reviews, so we answer it here for all three reviewers, with the mechanisms themselves and with the evidence that separates each from its existing form. Your review described the design as adapting both the information passed through decoding and its processing, and the three changes below are those adaptations.

The advance is one change, applied to each of the three mechanisms. In their existing forms the variable that selects, an attention weight, a codebook assignment, or a routing gate, receives no signal about which part of the input belongs to the target. It is trained through the output loss and through generic regularizers such as codebook commitment or load balancing, and it takes whatever meaning lowers the pixel error. For a registered input this is enough. We found that it is not enough for a wall reflection, because wall texture and ambient light also lower the pixel error on the flat regions that fill most of a screen. This is what we saw when we trained restoration networks on our pairs. They kept the overall colour of the screen and much less of its layout. In RERL, target-derived evidence directly supervises the agreement, confidence, and prototype masks. Prototype assignments also receive reconstruction and usage regularization, while the spatial degradation map is supervised by per-location reconstruction error and its summary informs high-level routing. Thus, the mechanisms are shaped by explicit evidence and degradation signals, although not every assignment or routing weight is directly supervised by the screen target.

MechanismExisting formForm in RERL
Agreementattention weights or gates computed from the features and trained through the output loss (cross-attention, Non-local, MDTA, SE and CBAM gating)pointwise agreement between the projected screen estimate and the projected observation (Eq.(3)), with a confidence trained toward the luminance, edge, and saturation evidence of the target (Appendix C.3)
Prototypeseach feature replaced by its nearest code (VQ-VAE), or class prototypes used for classification (ProtoNet)a soft mixture of prototypes added as a residual while the feature itself is kept (Eq.(7a)), through a spatial mask trained on the same target evidence
Routinga gate computed from the input features and trained through the output loss, with load balancing in the sparse routers (Adaptive MoE, Switch Transformer, GShard)a gate computed from pooled features and the mean of a spatial degradation map (Eqs.(9),(10)), where the map is trained on the per-location reconstruction error, which marks where the mixed evidence is hardest to recover, and also applies a spatial correction (Eq.(11))

Four checks separate this from a combination that happens to work.

Each change is tested by a one-at-a-time replacement within the full RERL design. One mechanism at a time is replaced by its existing form at matched parameters and FLOPs, with the RGB loss alone for every row. The other RERL components remain in place, so these controls measure each replacement within the full design, not the independent effect of an isolated component. Gated cross-attention, a VQ bridge, and a four-expert global mixture each fall 1.240 to 1.300 dB below the RERL structure on Screen, more than five times the three-seed standard deviation (Appendix J.1).

Each change holds when it is the only difference in a common framework. Tables 6 to 8 place one mechanism at a time in a common U-Net-style recovery framework and compare it with existing forms in the same position. CDERL leads the plain skip and five attention or gating bridges by 3.234 dB on Screen and by 1.808 dB on Figure. Consensus-driven learning, which adds the auxiliary supervision to L1, leads six existing objectives by 2.582 dB on Figure. DAR is the best PSNR row against six mixture-of-experts routers on Screen, Figure, and Game.

AuthorsOpenReview ↗

Reply to Reviewer eT4s(2/2)

The most recent existing mechanisms do not reach it. The 24 baselines of Tables 1 and 2 and the ten 2026 methods proposed by Reviewer p4ZT cover the current forms of attention, prompting, task tokens, and query-specific guidance in restoration. Our existing experimental results show that RERL outperforms all 34 methods on every content group based on the metrics reported for each comparison. The best of the ten reaches 20.2 dB on Screen against 26.7 dB.

The improvement is also observed in layout recovery. Table 24 evaluates 100 pairs of screens that have identical colour histograms and differ only in spatial layout. Restormer, the best-performing baseline on Screen, identifies the correct pairing in 63 of the 100 cases and RERL in 91, against a chance level of 50. This result suggests that the improvement of RERL is associated with the recovery of spatial layout, which colour statistics alone do not determine.

Figures 25 and 26 in Appendix G.2 show the same difference in the images. For the same observations and the same framework, the reconstructions of the skip and attention bridges keep the shape of the reflection spot, while the reconstruction of CDERL shows the layout of the screen.

Reviewer_XLQ4OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies reconstructing a hidden screen’s front-view RGB image from its diffuse reflection captured by an ordinary camera. Unlike conventional restoration, the observation undergoes unknown spatial mixing and contains substantial surface and illumination interference. The proposed Reflection-Evidence Routed Representation Learning (RERL) learns a supervised reflection-to-screen mapping, primarily recovering coarse layouts, color regions, and major contours rather than demonstrating reliable fine-text recovery.

Critical strengths
  1. Recovering screen content from diffuse reflections using an ordinary RGB camera is an important problem for both passive imaging and privacy.

  2. The reported results outperform 24 restoration baselines across four content groups.

  3. The paper is easy to follow, with a clear explanation of the task, the challenges, and the motivation.

Critical weaknesses
  1. The imaging problem is not defined precisely enough. Although the paper provides a general observation model, it does not connect the proposed degradation factors to the captured examples. There is a lack of analysis regarding the impact of the wall surface on projected images and the factors that ultimately influence the final image. Thus, it is difficult to understand what screen information survives in the reflection

  2. The data acquisition and alignment procedures are unclear. The paper describes the acquisition configurations, but does not adequately explain how reflection–target pairs are prepared or whether they require approximate spatial alignment.

  3. The methodological novelty is limited. The method appears to be a combination of existing well-studied technologies, including feature agreement, confidence gating, learned prototypes, and mixture-of-experts routing.

Questions for authors
  1. What causes the blurred dark rectangle in the first reflected observation in Figure 3? Which part of the ground-truth screen produces it, and through what physical process?

  2. In Figure 3, the first and third reflected observations appear to have similar spatial layouts, yet their targets differ substantially. What cues distinguish these targets, and how are these correspondences established during collection and preprocessing?

2 comments
AuthorsOpenReview ↗

Response to Reviewer XLQ4 (1/2)

Thank you for recognizing the importance of recovering screen content from diffuse reflections with an ordinary camera.

We wanted to find out whether the faint glow that a screen leaves on a wall still tells what the screen shows. We placed a 60-inch LED display outside the camera's direct view and photographed its reflection with a Sony A7S II on a tripod. A script played the screens one after another and the camera took one photograph of each, so every reflection is stored with the exact frame that produced it.

We built two separate scenes, the two acquisition configurations of Figure 7, and collected four datasets in them. In the first scene a T-shaped board stands between the display and the wall and keeps the display out of the camera's direct line of sight. The Screen dataset, 2,496 pairs, was captured there. In the second scene the camera stands behind the display, and the display faces the wall and lights it directly. The Lock, Game, and Figure datasets, 3,899 pairs, were captured there.

With the 6,395 pairs collected, we still had to find a way from the wall back to the screen. We trained 24 restoration networks on them and compared their outputs with the displayed frames. The comparison made the difficulty concrete. Most of what the camera records is wall and ambient light, and the screen appears as a weak, spatially mixed change of colour and brightness. A network that treats every feature of the photograph as content of the image to be restored keeps the overall colour of the screen and much less of its layout. This led us to RERL, which first decides which part of the observation is evidence of the screen and then chooses how to transform it.

Your two questions on Figure 3 ask what the reflection really contains. We answer them with the capture itself and with measurements of the reflected signal, which are now in Section 3.1, Section 4.1, and Appendices I to K of the revised paper.

1. The dark rectangle in Figure 3.

The dark rectangle is the T-shaped board itself, the intervening partition of the first acquisition configuration in Figure 7. The camera looks toward the wall, and in this scene the board stands in front of the wall inside the camera's view, so every Screen photograph contains the lit wall and the board. The board is a fixed object of that scene and appears at the same place under every displayed frame, independent of the screen content. The first three examples of Figure 3 are Screen samples, which is why they show the rectangle. The fourth is a Figure sample, captured in the second scene. RERL leads all 24 baselines on each of the four datasets in Table 1, in the scene with the board and in the scene without it.

2. What screen information survives in the reflection.

We then measured what the wall keeps of the screen. In Eqs.(1),(6) geometry and visibility fix which screen regions reach each wall location, surface reflectance and diffuse transport redistribute intensity and colour, and ambient light and camera response set contrast and noise. To link this model to the captures we use an independent forward-calibration diagnostic. Within the diagnostic, background subtraction and per-channel mean removal isolate the spatial residual, and predicted and measured residuals correlate on 120 held-out procedural screens.

ContentMediancorrelationShuffledtargets
40rectanglelayouts0.5580.022
40barcharts0.5110.008
40same-histogramlayouts0.5370.016

These controls show that spatial screen information survives beyond global colour. Grating measurements quantify how much modulation survives (H/V horizontal/vertical, ratios of recovered to displayed modulation).

Cycles/screenObservedA/σ,H/VRERLratio,H/VRestormerratio,H/V
129.56/25.700.926/0.9110.822/0.787
221.68/18.350.854/0.8170.606/0.567
410.60/8.770.689/0.6540.349/0.307
61.54/1.230.083/0.0690.052/0.043
80.53/0.500.032/0.0330.018/0.021

In these captures the modulation is measurable through four cycles and approaches the noise floor beyond, and RERL preserves more of it in both orientations, which grounds our recovery of layouts, colour regions, and major contours in the observed signal. The effect of wall surface, distance, viewpoint, and ambient light is quantified in Tables 16 to 19.

AuthorsOpenReview ↗

Response to Reviewer XLQ4 (2/2)

3. How pairs are formed, and what separates the first and third observations.

Each target is the displayed frame, paired to its capture via the display log, so the correspondence is exact at frame level and the reflection is used in its own wall coordinates. A script plays the screens in a fixed order and holds each for 3 s, and a tripod-mounted Sony A7S II with manual exposure, fixed ISO, and fixed white balance takes one photograph about 1.5 s after each screen appears. Photographs within 0.5 s of a switch are discarded. Inputs and targets are resized with Lanczos interpolation at their aspect ratio, centre-padded to 384×512, and scaled to [0,1], and the padded area is excluded from loss and evaluation. The same displayed content appears in only one of the training, validation, and test sets.

The first and third observations in Figure 3 share the board and the wall, which is why they look alike, and they differ in the light that the screen adds. The first target is a blue sky over a green field and its reflection is cyan-green, while the third is a white window on a dark navy desktop and its reflection is blue-grey. Once the fixed background is removed, the remaining light also falls differently across the wall. The revised Figure 3 caption states this.

To isolate spatial layout we used 100 pairs of screens with identical colour histograms and different arrangements.

Model/inputCorrectpairs
RERL/correctreflection91/100
Restormer/correctreflection63/100
RERL/pairwise-shuffledinput50/100
RERL/mean-colourinput54/100

RERL's 95% Wilson interval is [83.8%,95.2%], so spatial observations distinguish the targets well beyond global colour or chance pairing. On 40 unseen rectangle layouts RERL reaches IoU 0.840 and ΔE00 6.098, versus 0.567 and 14.801 for Restormer.

4. Methodological novelty.

The design came out of the difficulty described above. RERL changes two things that restoration networks keep fixed, the representation that the decoder receives and the signal from which that representation is learned. The four techniques you name implement these two changes.

The representation. In a restoration network the decoder receives the encoder features of the input, through a skip connection or an attention bridge. This representation rests on the premise that input and target are registered, so every encoder feature describes the image being restored. A wall reflection removes the premise. The encoder sees wall texture, ambient light, and a spatially mixed trace of the screen, all in wall coordinates, and most of what it extracts describes the wall. RERL therefore replaces the representation. The decoder receives reflection evidence, the part of an encoder feature that agrees with the decoder's current estimate of the screen (Eq.(3)), and recurring evidence configurations are carried between decoder stages as masked prototypes.

The learning. Restoration learning supervises the output pixels alone, with L1, Charbonnier, SSIM, perceptual, frequency, or adversarial terms, and leaves the network to find which features are signal. Under a pixel loss, wall features and screen features both lower the error on flat regions, so the loss treats them alike. Consensus-driven learning supervises the evidence itself. The confidence of the agreement is trained toward the luminance, edge, and saturation structure of the target, so the representation is learned as screen evidence at the stage where it is formed.

The transformation. Mixture-of-experts routing in restoration selects experts by task or by degradation type. DAR selects them by the estimated mixing state of the evidence and adds a per-region guidance residual (Eq.(4)), because the mixing varies across the wall.

Each change is isolated in the paper within one common recovery backbone, where only the named part differs between rows.

Whatischanged,allelsefixedStrongeststandardalternativeRERLchoiceGain(dB)
Representation,Screen(Table6)Directskip,10.688CDERL,13.922+3.234
Representation,Figure(Table6)MDTAbridge,15.239CDERL,17.047+1.808
Learning,Figure(Table7)L1+SSIM,14.170Consensus-driven,16.752+2.582
Routing,Screen(Table8)AdaptiveMoE,13.153DAR,13.982+0.829

The alternatives are CBAM, Non-local, Swin window-attention, MDTA, and NAFBlock bridges, six pixel, perceptual, and adversarial objectives, and six mixture-of-experts routers, and the RERL choice is the best PSNR row in all three content groups of each table.

On the quantities tied to the physics of the reflection, RERL returns about twice the modulation of Restormer at four cycles, the finest scale that reaches the wall, and separates layouts that share one colour histogram, which is the content that the evidence representation is meant to keep.

The capture protocol is now in Section 4.1, the grating and forward measurements are summarized in Section 3.1 with their full tables in Appendix I, and we ask that the rating be reconsidered.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.