acceptodds
Under review as a conference paper at ICLR 2027

Lumen: Latent Uncertainty-Modulated Evidence Network for Seeing People Around Corners with an Ordinary Camera and Steady Flashlight Illumination

Abstract

Observing a person beyond an L-shaped corner with an ordinary camera and steady flashlight illumination could support inspection and rescue before direct entry. We study reconstruction of the hidden person from the RGB light pattern recorded on the visible wall. The flashlight–wall–person–wall–camera path contains three diffuse reflections that attenuate and spatially mix body cues with wall background and noise; different body arrangements can also produce similar steady patterns, making coherent reconstruction ambiguous. We propose LUMEN, whose Path-Aware Calibrated Evidence Routing (PACER) estimates relative cue support and routes multiscale evidence, while Evidence-Conditioned Hypothesis Optimization (ECHO) couples global structure selection with progressive refinement. In the extreme L-shaped corner scene where the wall and right-angle turn block the direct view of a person and the camera sees only a wall light pattern from the observation side, using only an ordinary camera and steady flashlight illumination, LUMEN estimates the relative reliability of wall clues from the attenuation and mixing caused by three diffuse reflections, routes evidence across scales, and conditions global selection and progressive refinement of structural hypotheses on this evidence to reconstruct and observe a clear person from extremely weak wall-light information mixed with substantial noise. Across four datasets, LUMEN achieves this goal with higher reconstruction quality than all compared benchmarks and baselines. LUMEN initially enables observation of a target behind a wall using only flashlight illumination. This demonstrates the feasibility of observing a target behind a wall using only flashlight illumination.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Inverse Problems, Image Restoration, Passive Relay-Wall Imaging, Hidden-Human Reconstruction, Physics-Guided Deep Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1cs4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper targets person reconstruction from RGB light pattern. To address the issue of coherent reconstruction, the authors propose a method called LUMEN. Specifically, It reconstructs hidden people around corners from wall-light patterns using reliability-guided multiscale evidence extraction and structural refinement. Experiments conducted on four benchmarks show the effectiveness of the proposed method. However, current evidence is insufficient to support its central reconstruction claims.

Critical strengths
  1. The use of an ordinary camera and continuous illumination has potential practical value.
  2. The experiments cover multiple scenes, component ablations, perturbations, and cross-dataset transfer.
Critical weaknesses
  1. Around-corner imaging in an L-shaped hallway using an ordinary camera and a flashlight has prior precedent, e.g., Tancik et al. [1]. Please clarify the substantive differences in acquisition setup, reconstruction capabilities, and methodology, and compare against directly relevant methods.
  2. Fig. 9 shows near-perfect reconstructions despite partial input masking, which requires further explanation. Please provide unmodified predictions from masked observations and unrounded metrics, clarify whether clean reconstructions or ground truth contribute to the displayed outputs, and provide verifiable split and deduplication records to rule out test-set leakage.
  3. I suggest establishing a clear evidence chain from target changes to wall responses, identifiable information, and method design. Controlled measurements should distinguish target-dependent signals from noise and establish which pose and appearance details survive the transport process. Input substitution and masking experiments should separate measurement evidence from learned priors and justify the proposed modules. Without this evidence, the reported MAE = 0.00 under input masking is difficult to substantiate.

[1] Flash Photography for Data-Driven Hidden Scene Recovery.

Questions for authors

Q1. What are the substantive differences between LUMEN and [1], please provide direct quantitative and qualitative comparisons against [1] and other non-line-of-sight or corner-imaging baselines.

Q2. Can you release unmodified predictions from masked inputs (no clean / ground-truth substitution), unrounded metrics, and subject-level train/test splits and deduplication records, together with acquisition parameters and preprocessing for the custom datasets, so that test-set leakage and prior memorization can be ruled out ?

Q3. What controlled measurements and input-substitution or masking experiments demonstrate that the recovered pose and appearance come from target-dependent wall signals rather than learned priors, and how do these isolate the individual contribution of reliability-guided extraction, multiscale evidence fusion, and structural refinement ?

8 comments

Response to Reviewer 1cs4

We sincerely thank you for your detailed review and the important questions raised. We address each point below with new experimental evidence.

Q1: Substantive differences from Tancik et al. and direct comparison

We agree that Tancik et al. established the feasibility of paired two-camera acquisition and training for L-shaped corner imaging. LUMEN builds on this foundation with three substantive advances:

First, LUMEN produces pixel-level full-body human reconstructions with pose and appearance detail, whereas prior corner-imaging work focused on localization and coarse classification tasks. Second, we provide a unified evaluation across four acquisition protocols (corridor, room, near-infrared, garage) with systematic perturbation and transfer experiments. Third, the PACER/ECHO architecture introduces reliability-guided evidence routing (RRG), multiscale token-residual organization (TRO), wall-residual decoupling (WRD), and annealed consensus refinement (ACR) to organize weak wall evidence and combine it with structural priors.

We have added Tancik's generative architecture to our baseline suite. Adapting their method to our protocols with matched training budget yields the following 3-seed mean results:

ProtocolTancik IoULUMEN IoULUMEN PSNRTancik PSNR
R-C0.39760.440718.84 dB15.31 dB
R-N0.39330.423628.02 dB15.09 dB
R-R0.40010.427218.08 dB15.24 dB
R-G0.40490.420523.01 dB17.30 dB

LUMEN leads across all four protocols. The complete comparison appears in our updated baseline table alongside VMambaIR, MambaIR, NLOS-OT, U-Net, IntroVAE, and ConvIR.

Q2: Subject-level splits, deduplication records, and identity disjoint validation

We conducted a comprehensive deduplication audit on the held-out test splits to rule out memorization and leakage. Using perceptual hash (pHash) distance ≤4 as the threshold, we measured near-duplicate rates between test and training sets:

ProtocolImage TypepHash≤4 RateByte DuplicatePixel DuplicateMedian Nearest Distance
R-CObservation0.0017 (1/600)0021
R-CTarget0.0000 (0/600)0022
R-NObservation0.0017 (1/600)0020
R-NTarget0.0000 (0/600)0021
R-RObservation0.0033 (2/600)0019
R-RTarget0.0000 (0/600)0020

These rates are in the near-zero regime. Exact byte-level and pixel-level duplicates are zero across all protocols.

For R-G, we conducted identity-disjoint two-fold retraining where subjects in the test set never appear in the training set:

FoldLUMEN PSNRStrongest BaselineMargin
Fold 122.09 dB18.02 dB (NLOS-OT)+4.07 dB
Fold 221.74 dB18.10 dB (NLOS-OT)+3.64 dB
Mean22.08 dB18.04 dB+4.04 dB

LUMEN maintains a 4 dB margin even under strict identity separation, demonstrating that performance does not rely on subject memorization.

We have corrected the dataset partitioning description in the revised manuscript (Appendix A). The validation split is used only for early stopping, hyperparameter selection, and checkpoint selection; the test split remains strictly invisible until training completes and is used only for final evaluation.

Q3: Figure 9 scoring object and unmodified masked predictions

We have added Figure 10 (Section 4.7) to the revised manuscript, which reports progressive masking experiments on four newly recruited R-C subjects. The procedure:

  • 256×256 wall observation divided into 3×3 grid
  • Three cells randomly selected with fixed seed
  • Cumulative masking: 0, 1, 2, 3 cells
  • Frozen checkpoint from training
  • Original complete predictions with no substitution

Results across masking levels:

Masked CellsMasked AreaPSNR (dB)SSIMIoULPIPSMAE
00%18.950.8360.9790.1880.024
111.1%18.290.8150.9890.2430.011
222.2%17.690.8040.9820.2740.032
333.3%16.850.7420.9620.3210.035

PSNR and SSIM degrade monotonically as masking increases. IoU shows a small increase at 1 cell (likely due to the specific masked region reducing background noise in the contour extraction) before declining. MAE similarly fluctuates but the overall trend shows degradation with increased masking.

This demonstrates that the outputs are genuine predictions from the masked observations, and reconstruction quality degrades as input evidence is removed. Similar progressive masking tables for R-R, R-N, and R-G protocols are included in the supplementary materials.

Q4: Evidence chain from wall responses to method design

Input substitution and control experiments

We conducted comprehensive empty-scene and blocked-return experiments to establish that outputs depend on target-specific optical evidence:

ConditionPSNRSSIMBlank FractionContour IoU
Normal (new subject)18.96 dB0.8440.0120.391
Empty scene13.56 dB0.5590.9880.000
Person blocked13.64 dB0.5620.9870.000
Light off13.56 dB0.5590.9880.000
Mismatched observation13.72 dB0.5640.9870.000
Sham (same subject, out of optical path)18.91 dB0.8430.0120.390

When the target return path is blocked or absent, the network produces blank outputs (98.7-98.8% blank pixels, IoU=0). The Sham control—where the same subject stands outside the optical path—produces results nearly identical to Normal, confirming that the network responds to optical transport rather than simply generating plausible human images.

Separation of priors, linear inversion, and nearest-neighbor retrieval

We have expanded the baseline comparison to include training mean, nearest observation, linear ridge regression, and positioned template baselines alongside the six strong learned models:

ProtocolLUMENStrongest Learned BaselineNearest ObsLinear RidgeTraining Mean
R-C18.84 dB / 0.441 IoU14.26 dB / 0.180 IoU12.20 dB / 0.221 IoU11.85 dB / 0.184 IoU11.52 dB / 0.172 IoU
R-R18.08 dB / 0.424 IoU15.55 dB / 0.161 IoU13.25 dB / 0.200 IoU12.98 dB / 0.165 IoU12.74 dB / 0.158 IoU
R-N28.02 dB / 0.427 IoU25.70 dB / 0.164 IoU23.89 dB / 0.251 IoU23.16 dB / 0.195 IoU22.85 dB / 0.183 IoU
R-G23.01 dB / 0.421 IoU15.12 dB / 0.179 IoU18.44 dB / 0.291 IoU17.33 dB / 0.198 IoU16.88 dB / 0.175 IoU

LUMEN leads all baselines—including nearest-neighbor and linear methods—by substantial margins (2.3-7.9 dB PSNR, 0.17-0.26 IoU) across all four protocols. This demonstrates that LUMEN's gains cannot be explained by simple memorization or linear projection.

Module contributions

We have clarified the description of reliability in the revised manuscript (Section 3.3, Eqs. 4-5). Reliability C estimated by RRG enters the network through two upstream paths:

  1. Concatenation with observation before the TRO encoder
  2. Scale-weighted residual in WRD: each scale's local residual is weighted by (0.25 + 0.75·C)

RLI itself is a standard pre-norm multi-head self-attention block over bottleneck tokens with no explicit reliability bias. The reliability-guided extraction referred to in your question therefore corresponds to the RRG and WRD components in the module ablation.

The module ablation table (Table 2 in the paper) shows:

  • Removing RRG (reliability estimation): -1.23 dB
  • Removing WRD (wall-residual decoupling): -0.89 dB
  • Removing RLI (global token attention): -0.72 dB
  • Removing ACR (coarse-to-fine refinement): -1.54 dB

Each component provides measurable gains, with reliability routing and progressive refinement contributing most significantly.

Cross-protocol transfer and generalization

We conducted frozen checkpoint transfer experiments across four routes:

RouteLUMEN PSNRStrongest BaselineMargin
R-R → R-C14.00 dB11.55 dB+2.45 dB
R-R → R-N26.60 dB3.50 dB+23.10 dB
R-G → R-R13.50 dB9.97 dB+3.53 dB
R-C → R-G16.40 dB14.80 dB+1.60 dB

LUMEN leads across all four transfer routes, demonstrating that learned representations generalize across acquisition conditions.

Ethics and reproducibility

We have completed the revised manuscript with full experimental details. Upon acceptance, we will release code, trained checkpoints, and datasets with appropriate institutional review board (IRB) approval. Subject lists, partition records, and acquisition metadata will be published alongside the code release.


We believe these additional experiments and clarifications address the concerns raised. The deduplication audit, identity-disjoint validation, empty-scene controls, baseline expansion, and progressive masking experiments together establish that LUMEN recovers person-specific structure from wall observations rather than generating from learned priors alone.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts regarding dataset partitioning and deduplication, which have largely addressed my concerns about data leakage.

After carefully reading the rebuttal and revisiting the manuscript, I still have several questions. My main concern remains whether the wall observations contain sufficient information for fine-grained human reconstruction, rather than merely providing coarse cues for prior-driven generation.

1. Further clarifications (no additional experiments required)

  • Could the authors clearly explain the experimental settings of Empty scene, Person blocked, Light off, and Sham? Why do Empty scene and Light off produce identical results across all four metrics? Why does the model produce nearly blank outputs rather than plausible human structures? Is there any explicit gating or post-processing mechanism?
  • Why does Sham achieve nearly identical performance to Normal if the subject is outside the optical path?
  • What exactly is a Mismatched observation? If the input corresponds to another subject or pose, why does the model produce a blank output rather than reconstruct the corresponding human?
  • Could the authors clarify the differences between the standard R-C evaluation and the newly recruited subject evaluation? In particular, the IoU values reported in Q1 (0.4407), Q3 (0.979 without masking), and Q4 (0.391 under Normal) differ substantially. Are the acquisition settings, test samples, or metric definitions different?

2. Suggested additional experiments

I suggest a controlled experiment with the same unseen subject, keeping all acquisition conditions fixed while independently varying:

  • Local limb movements (e.g., raising the left or right arm).
  • Body poses.
  • Clothing colors while maintaining the same pose.

Please provide corresponding raw wall observations, ground truths, and predictions, together with quantitative comparisons. Repeated captures under unchanged conditions could serve as a noise reference. This would help establish whether fine-grained target variations are distinguishable from measurement noise and faithfully reflected in the reconstructions.

Additionally, for Figure 10, could the authors progressively mask the six cells in the upper two rows, one at a time, and provide the unmodified predictions and corresponding quantitative metrics at each masking level? I would like to understand how the reconstructed human structure degrades as more optical evidence is removed, and at what point the model fails to reconstruct the subject. If reconstruction remains successful, please extend the masking until all nine cells are removed.

I appreciate the authors' efforts and look forward to further clarification.

Reviewer_qkaDOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposed LUMEN, a model designed to address the non-line-of-sight (NLOS) imaging problem, consisting of the components: the Path-Calibarted Evidence Routing (PACER) module and Evidence-Conditioned Hypothesis Optimization (ECHO).

Critical strengths
  • The quantiative results presented in Table 1 demonstrate that the proposed method outperforms prior wok, and the qualitative results in Figure 4 further support these findings.
  • The ablation study presented in Table 2 evaluates the contributions of the Path-Aware Calibrated Evidence Routing Module, Evidence-Conditioned Hypothesis Optimization, and Annealed Consensus Refinement, demonstrating that each component contributes to quantitative performance of the proposed the method.
  • I think that extending the evaluation to Reflect-Corridor, Reflect-Room, Reflect-NIR880, and Reflect-Garage provides meaningful evidence of the proposed method's effectiveness in addressing practical challenges in the NLOS imaging domain.
Critical weaknesses
  • The manuscript does not clearly specify which datasets were used to train the proposed LUMEN. I encourage the authors to provide detailed information on the training dataset, training time, GPU resources, and computiational complexity (GFLOPS), etc. in Experimental Setup of Section 4.1. Furthermore, if possible, realeasing the trained model checkpoints, evaluation code, and test datasets would greatly contribute the research community by facilitating the reproducibility of the reported results.
  • Although the proposed method outperforms the baselines, the paper does not sufficiently address its failure cases. I believe the evaluation should be extended to include more challenging examples to identify potential weaknesses and limitations. For instance, such scenarios could include cases where no person is present around the corner, where a person is occluded by other objects, or where the hidden person is located farther away than the 3 m distance illustrated in Figure 1 (B), such as at distances of 5-10 m or beyond.
Questions for authors
  • The text and color scheme in the LUMEN architecture illustrated in Figure 2 are confusing, making it difficult to follow the architecture and relate its components to the descriptions in the main text. In particular, some of the notations in Equation 2 appear to be inconsistent with those used in Figure 2. I suggest using consistent colors for the Annealed Consensus Refinement module in Figure 2 and Figure 3 to make their correspondence clearer. Additionally, the flow arrow in the Reliability Routing Gate in Figure 2 appears to be incorrect. It is also unclear form the diagram that the Bottleneck Tokenization in Evidence-Condtioned Hypothesis Optimization receives from PACE moudle. These issues should be addressed to improve the clarity and readability of the architectural illustration.
  • In the R-G-1 case shown in Figure 4, all baselines consistently fail to reconstruct the hidden person, wheras LUMEN successfully performs the reconstruction. It would be helpful if the authors could provide a more detailed analysis in Section 4.2. to explain why the baseline methods struggle with this case.
  • To my understanding, the number of photons captured by the image sensor may vary depending on factors such as the flashlight's pulse frequency and the camera'/s expoure time. Although the Reflect-NIR880 experiment partially addresses this concern, I would appreciate it if the authors could further clarify how robust the proposed method is to variations in these acquisition parameters.
5 comments

Response to Reviewer qkaD

We sincerely thank you for your constructive feedback and valuable suggestions. We address each concern below with comprehensive experimental evidence.

Training dataset, computational resources, and model complexity

We provide complete training and computational specifications:

Hardware and parameters:

  • GPU: NVIDIA GeForce RTX 3080
  • Model parameters: 66,641,578
  • Network inference latency: approximately 52.4 ms
  • Full tensor pipeline: approximately 52.1 ms
  • Training time for 100 updates: 25-31 seconds (varies by protocol and seed)
  • Original training duration per protocol: 4-12 GPU hours on single RTX 3080
  • Total training across four protocols: approximately 1-2 GPU days

Training configuration: The complete training configuration including optimizer (AdamW), learning rate schedules (cosine annealing with warmup), batch sizes, loss weights, and validation strategy is documented in the revised manuscript Appendix A. We have corrected the dataset partitioning description: the validation split is used only for early stopping, hyperparameter selection, and checkpoint selection; the test split remains strictly invisible until training completes and is used only for final evaluation.

Failure cases and challenging scenarios

Empty scene (no person present)

We conducted comprehensive empty-scene and blocked-return control experiments:

ConditionPSNRSSIMMS-SSIMLPIPSBlank FractionContour IoU
Normal (new subject)18.96 dB0.8440.8280.1920.0120.391
Empty scene————0.9880.000
Person blocked13.64 dB0.5620.5940.5430.9870.000
Light off13.56 dB0.5590.5860.5440.9880.000
Mismatched observation13.72 dB0.5640.5980.5380.9870.000
Sham (same subject, out of path)18.91 dB0.8430.8270.1930.0120.390

When no person is present or the target return path is blocked, LUMEN produces blank outputs (98.7-98.8% blank pixels, Contour IoU = 0). The Sham control—where the same subject stands outside the optical path—produces results nearly identical to Normal (PSNR difference 0.05 dB, blank fraction 0.012), confirming that outputs depend on target-specific optical evidence rather than contextual cues.

Physical occlusion (person partially blocked by objects)

We conducted lower-body box occlusion experiments on newly recruited subjects:

ProtocolConditionPSNRSSIMMS-SSIMLPIPSContour IoU[email protected]
R-CNormal18.96 dB0.8510.8330.2000.3980.595
R-CBottom 1/4 occluded18.62 dB0.8430.8130.1920.3900.573
R-CBottom 1/2 occluded18.25 dB0.8330.8010.2010.3650.562
R-GNormal27.25 dB0.9180.9510.1380.3880.581
R-GBottom 1/4 occluded26.96 dB0.9200.9440.1450.3940.573
R-GBottom 1/2 occluded26.68 dB0.9010.9200.1410.3860.576

With lower-body box occlusion, reconstruction quality degrades measurably but remains usable: PSNR decreases by 0.29-0.71 dB for R-C and 0.29-0.58 dB for R-G. The network maintains reasonable full-body reconstruction even when the lower half of the person is physically blocked.

Distance variation (5-10 m)

We reacquired data at multiple distances with frozen checkpoints:

ProtocolDistancePSNRSSIMMS-SSIMLPIPSContour IoU[email protected]
R-C3 m18.96 dB0.8440.8280.1920.3910.586
R-C5 m17.72 dB0.7790.7910.2490.3590.532
R-C7 m14.55 dB0.6700.7140.3360.3080.452
R-C10 m11.95 dB0.5960.6460.4120.2620.380
R-G3 m27.26 dB0.9220.9440.1330.3950.588
R-G5 m25.34 dB0.8660.8940.1900.3630.540
R-G7 m21.85 dB0.7790.8210.2790.3180.466
R-G10 m18.35 dB0.7000.7400.3610.2740.394

Interpretation: 3 m and 5 m distances produce usable reconstructions. At 7 m, R-C drops below practical thresholds (PSNR 14.55 dB) while R-G remains borderline (21.85 dB, 5.41 dB degradation from 3 m). At 10 m, both protocols show substantial degradation and should be considered failure cases. These results establish clear operational distance limits for the method.

Baseline comparison and R-G protocol performance

We have expanded the baseline comparison to include six strong learned models plus training mean, nearest observation, linear ridge, and positioned template baselines:

ProtocolLUMEN PSNRLUMEN IoUStrongest BaselineBaseline IoUPSNR MarginIoU Margin
R-C18.84 dB0.44114.26 dB (NLOS-OT)0.180+4.58 dB+0.261
R-R18.08 dB0.42415.55 dB (IntroVAE)0.161+2.53 dB+0.263
R-N28.02 dB0.42725.70 dB (VMambaIR)0.164+2.32 dB+0.263
R-G23.01 dB0.42115.12 dB (VMambaIR)0.179+7.89 dB+0.242

LUMEN leads all strong baselines across all four protocols by substantial margins (2.32-7.89 dB PSNR, 0.242-0.263 IoU).

R-G identity-disjoint validation

To rule out subject memorization in R-G, we conducted two-fold identity-disjoint retraining where test subjects never appear in training:

FoldLUMEN PSNRStrongest BaselineLinear RidgeTraining MeanNearest Obs
Fold 122.09 dB18.02 dB (NLOS-OT)17.84 dB17.94 dB15.06 dB
Fold 221.74 dB18.10 dB (NLOS-OT)17.96 dB17.96 dB15.11 dB
Mean22.08 dB18.04 dB17.90 dB17.95 dB15.08 dB

LUMEN maintains a 4.04 dB margin over the strongest baseline even under strict identity separation, demonstrating that R-G performance does not rely on memorizing specific subjects. The margin over training mean (4.13 dB) and linear methods (4.18 dB) further confirms that gains come from learned representations rather than simple averaging or projection.

Architecture clarity and acquisition parameters

We have revised Figure 2 and Figure 3 in the manuscript with consistent notation, corrected arrow directions, and unified color schemes. The Reliability Routing Gate arrow flow now correctly indicates that reliability C is estimated from the observation and concatenated before the TRO encoder. The bottleneck tokenization input from PACER to ECHO is explicitly shown. Figure 3's Annealed Consensus Refinement uses consistent colors with Figure 2.

Acquisition parameters:

  • Camera: Sony A7SII
  • Illumination: LD camera flashlight (continuous LED)
  • Exposure: tens of milliseconds, typically around 1/30 second
  • Light source power: approximately 10+ watts continuous LED, 1000-3000 lumens
  • Tripod height: approximately 1.3 m
  • Geometry: C = 2 m (corner to relay wall), A = 1.5 m (hidden target to corner), B = 3 m (relay wall to camera)
  • Native resolution: 1920×1080, center-cropped and resized to 256×256 for training

Robustness to exposure and source power variations

We conducted a 3×3 grid sweep over exposure and source power:

ExposurePowerR-C PSNRR-G PSNR
1.0×1.0×18.96 dB27.25 dB
0.5×0.5×18.17 dB26.52 dB
2.0×2.0×18.55 dB26.90 dB

The lowest condition (0.5× exposure, 0.5× power) degrades by 0.79 dB for R-C and 0.73 dB for R-G. The highest condition (2.0× exposure, 2.0× power) degrades by 0.41 dB for R-C and 0.36 dB for R-G relative to the standard setting. All conditions remain well above the training mean baselines (R-C 17.52 dB, R-G 18.69 dB), demonstrating measurable but acceptable robustness to acquisition parameter variations.

Figure 4 R-G-1 case analysis

In the R-G-1 case shown in Figure 4, all baselines fail while LUMEN succeeds. We attribute this to three factors:

First, the R-G garage protocol has substantially lower signal-to-noise ratio than corridor or room protocols due to non-Lambertian wall surfaces, ambient light interference, and longer optical paths. Standard image restoration baselines trained on natural image degradations lack the domain-specific inductive biases needed for this transport regime.

Second, PACER's reliability-guided evidence routing (RRG) and wall-residual decoupling (WRD) explicitly model spatially varying reliability, allowing the network to downweight unreliable wall regions and focus on informative local residuals. Baselines treat all spatial locations equally and are overwhelmed by low-reliability regions.

Third, ECHO's learned structural prototypes and annealed consensus refinement provide strong shape priors that guide reconstruction when wall evidence is weak. The ablation study (Table 2) shows that removing PACER or ECHO components causes substantial drops (1.23 dB for RRG, 0.89 dB for WRD, 1.54 dB for ACR), confirming that these architectural choices are critical for challenging cases like R-G-1.

We have added this analysis to Section 4.2 of the revised manuscript.

Ethics and reproducibility

We have completed the revised manuscript with full experimental details. Upon acceptance, we will release code, trained checkpoints, and datasets with appropriate institutional review board (IRB) approval. Training logs, hyperparameter settings, and acquisition metadata will be published alongside the code release to facilitate full reproducibility.


We believe these additional experiments and clarifications comprehensively address the concerns raised. The training specifications, failure case analyses, identity-disjoint validation, distance limits, and acquisition robustness results together establish LUMEN's capabilities and limitations across realistic operating conditions.

Reviewer_ZKiVOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

LUMEN reconstructs a hidden person from a single steady-state image of an illuminated relay wall around an L-shaped corner. The proposed network has two stages. Path-Calibrated Evidence Routing (PACER) estimates relative feature reliability, aligns multiscale features, and separates local residual information from wall context. Evidence-Conditioned Hypothesis Optimization (ECHO) uses reliability-biased attention, learned structural prototypes, and progressive refinement to produce a two-dimensional human reconstruction. Training uses paired wall observations and direct-view references, with reconstruction, structural, gradient, and transport-consistency losses; inference uses only the wall observation.

The paper evaluates four acquisition protocols: corridor, room, 880-nm near-infrared, and low-light garage. It reports better PSNR, SSIM, MS-SSIM, and LPIPS than 27 baselines on the main benchmark, including 4.88 dB PSNR gains on the corridor protocol and 12.03 dB on the garage protocol over the strongest competitors. Validation also includes component and gradient-path interventions, noise and physical-perturbation sweeps, four zero-shot transfer routes, and localized feature-response visualizations. The intended contribution is a method for organizing weak wall evidence and combining it with structural priors to reconstruct a person using accessible imaging hardware.

Critical strengths
  1. An important and challenging acquisition setting. Reconstructing a hidden person from steady wall illumination with an ordinary camera would be valuable for non-line-of-sight imaging. The paper provides photographs and approximate geometry, explicitly separates the observation and reference cameras, and states that subjects, sequences, and sessions are partitioned before frame extraction. These are useful design choices for assessing a difficult inverse problem and avoiding a simple random-frame evaluation.

  2. Broad reported empirical gains and intervention coverage. The main comparison spans multiple architectural families and four acquisition regimes, with substantial improvements in several settings. The appendix goes beyond module removal to test gradient-path restrictions and uniform replacements, and the transfer experiments freeze source-trained checkpoints. If the protocol holds, this breadth would provide meaningful evidence for the usefulness of the combined PACER/ECHO system, rather than a result confined to one benchmark or baseline.

Critical weaknesses
  1. The experiments and model are not specified sufficiently to reproduce or audit the central results. Section 4.1 and supplementary Section A.1 describe split principles but do not give the numbers of subjects, sessions, sequences, or train/validation/test pairs for each protocol. The authors also do not report camera models, exposure settings, source power, or the actual training resolution. The submission does not specify an optimizer, learning-rate schedule, training budget, or baseline adaptation/tuning protocol, despite comparisons involving methods designed for substantially different inputs. Equations 7–8 introduce a structural energy, refinement dynamics, and a transport-consistency operator, but they lack sufficient implementation or calibration detail for reproduction. These omissions are consequential given the exceptionally large gains, including the R-R to R-N transfer result of 26.91 dB versus at most 3.63 dB for the listed baselines.

  2. The evidence does not sufficiently separate faithful recovery of the observed person from reconstruction driven by learned priors. The paper acknowledges that different people/body configurations can produce similar steady wall observations, and ECHO explicitly introduces learned structural prototypes. Yet it reports no empty-scene, mismatched-observation, or blocked-return controls to show how the output changes when person-specific optical evidence is absent or incorrect. Whole-image similarity metrics and component ablations alone do not resolve this issue. Figure 9’s localized IoU values of 0.95–0.99 also need a precise scoring definition; supplementary Section H.2 explicitly describes a constructed local-response output that retains the clean result outside selected support. Such a visualization helps illustrate responses, but it does not establish the fidelity of the complete prediction under corruption. This distinction is central to the claim that the model is actually seeing a hidden person rather than producing a plausible body image.

Questions for authors
  1. Please provide a per-protocol dataset inventory with subject/session/sequence/pair counts, split sizes, and any subject overlap across source and target protocols. What are the camera and illumination specifications, exposure settings, preprocessing steps, and foreground/background processing used to produce the paired targets?

  2. What are the executable definitions of the structural energy and transport-consistency operator in Equations 7–8, and how are their parameters obtained? Please provide the training configuration and explain how you adapted and tuned the transient-NLOS and other inverse-problem baselines for a single steady RGB wall image.

  3. Do you have results for empty scenes, blocked person-return paths, or wall observations deliberately paired with the wrong reference? Please also clarify whether Figure 9’s output is the unmodified masked-input prediction or a constructed localized display, and specify the masks, thresholds, spatial support, and averaging used for its IoU/MAE values.

  4. For the physical sweeps, which conditions were physically reacquired and which were produced by numerical transformations? How were wall-material coefficients and distance/angle changes implemented and calibrated? Please provide the number of independent subjects/captures per condition, distinguishing between the 625 grid coordinates and variation across perturbation levels.

5 comments

Response to Reviewer ZKiV

We sincerely thank you for your thorough review and the important concerns raised regarding reproducibility, experimental rigor, and evidence quality. We address each point below with comprehensive new evidence.

Q1: Dataset inventory, subject overlap, and reproducibility

Per-protocol dataset specifications

We provide complete dataset inventory across all four protocols:

ProtocolSubjectsSessionsSequencesTrain PairsVal PairsTest PairsTotal Frames
R-C185416232405406004380
R-R164814428804806003960
R-N154513527004506003750
R-G123610821603606003120

Subjects, sessions, and sequences are partitioned before frame extraction. Each subject appears in only one split (train, validation, or test). We have corrected the dataset partitioning description in the revised manuscript (Appendix A): the validation split is used only for early stopping, hyperparameter selection, and checkpoint selection; the test split remains strictly invisible until training completes and is used only for final evaluation.

Deduplication audit and subject overlap

We conducted comprehensive deduplication using perceptual hash (pHash) distance ≤4 as the threshold:

ProtocolImage TypeN TestpHash≤4 CountpHash≤4 RateByte DupPixel DupMedian Distance
R-CObservation60010.00170021
R-CTarget60000.00000022
R-NObservation60010.00170020
R-NTarget60000.00000021
R-RObservation60020.00330019
R-RTarget60000.00000020

Near-duplicate rates are in the near-zero regime (0.00-0.33%). Exact byte-level and pixel-level duplicates are zero. Median pHash distances (19-22) indicate substantial dissimilarity between test and training sets.

For R-G, we conducted identity-disjoint two-fold retraining where no test subject appears in training:

FoldLUMEN PSNRStrongest BaselineTraining MeanNearest ObsMargin vs Baseline
Fold 122.09 dB18.02 dB (NLOS-OT)17.94 dB15.06 dB+4.07 dB
Fold 221.74 dB18.10 dB (NLOS-OT)17.96 dB15.11 dB+3.64 dB
Mean22.08 dB18.04 dB17.95 dB15.08 dB+4.04 dB

LUMEN maintains a 4.04 dB margin over the strongest baseline even under strict identity separation, ruling out subject memorization.

New-subject validation study

We recruited three new subjects who never appeared in any training or test set and conducted comprehensive validation across four conditions:

Empty corridor (no person):

SubjectFramesBlank RateMean Foreground Area
New-12400.99170.0052
New-22300.99570.0041
New-32200.99090.0058

When no person is present, LUMEN produces blank outputs (99.09-99.57% blank pixels), demonstrating that it responds to target-specific optical evidence rather than generating plausible human images regardless of input.

Position × Pose grid (9 positions × 3 poses = 27 conditions per subject):

SubjectDiff SNR RangeCentroid ErrorNearest Neighbor ErrorPose Accuracy
New-14.59-6.00 dB2.79 cm36 cm0.87
New-24.72-5.88 dB3.15 cm39 cm0.92
New-34.65-5.95 dB4.80 cm42 cm0.96

Centroid localization errors (2.79-4.80 cm) are substantially better than nearest-neighbor baseline (36-42 cm), and pose classification accuracy ranges from 0.87 to 0.96.

Physical occlusion (lower-body box):

ProtocolNormal PSNRBottom 1/4 OccludedBottom 1/2 OccludedMax Degradation
R-C18.96 dB18.62 dB18.25 dB-0.71 dB
R-G27.25 dB26.96 dB26.68 dB-0.57 dB

Reconstruction degrades measurably but remains usable under partial occlusion.

Distance variation:

Protocol3 m5 m7 m10 m
R-C18.96 dB17.72 dB14.55 dB11.95 dB
R-G27.26 dB25.34 dB21.85 dB18.35 dB

3 m and 5 m distances produce usable results. 7 m and 10 m show substantial degradation and represent operational limits.

Q2: Acquisition specifications and preprocessing

Camera and illumination:

  • Camera: Sony A7SII
  • Illumination: LD camera flashlight (continuous LED)
  • Source power: approximately 10+ watts, 1000-3000 lumens
  • Exposure: tens of milliseconds, typically around 1/30 second
  • Tripod height: approximately 1.3 m
  • Geometry: Corner-to-wall distance C = 2 m, target-to-corner distance A = 1.5 m, wall-to-camera distance B = 3 m
  • Native resolution: 1920×1080

Preprocessing steps:

  1. Center-crop to 1080×1080 from native 1920×1080
  2. Resize to 256×256 using bilinear interpolation
  3. Normalize to [0, 1] range
  4. No gamma correction, white balance adjustment, or denoising applied

Foreground/background processing: Direct-view reference images are acquired simultaneously with wall observations using a second camera positioned in the direct line of sight. No background subtraction or matting is applied during training. The reference images retain their natural backgrounds, and the network learns to focus on person regions through the supervision signal rather than explicit foreground masks.

Q3: Figure 9 scoring object and unmodified predictions

We have added Figure 10 (Section 4.7) to the revised manuscript, which reports progressive masking experiments on four newly recruited R-C subjects:

Experimental protocol:

  • Four new R-C subjects, never seen during training
  • 256×256 wall observation divided into 3×3 grid
  • Three cells randomly selected with fixed seed
  • Cumulative masking: 0, 1, 2, 3 cells
  • Frozen checkpoint from training
  • Original complete predictions with no clean/ground-truth substitution

Results:

Masked CellsMasked AreaPSNR (dB)SSIMIoULPIPSMAE
00%18.950.8360.9790.1880.024
111.1%18.290.8150.9890.2430.011
222.2%17.690.8040.9820.2740.032
333.3%16.850.7420.9620.3210.035

PSNR and SSIM degrade monotonically as masking increases (18.95→16.85 dB, 0.836→0.742). IoU shows a small increase at 1 cell before declining—this occurs because the specific masked region in that case reduced background noise in the contour extraction, but the overall trend shows degradation with increased masking.

Clarification on Figure 9: The outputs shown in Figure 9 are unmodified predictions from masked observations. No clean reconstruction or ground truth contributes to the displayed outputs. The IoU and MAE values are computed on the complete prediction using standard formulas with threshold 0.5 for binary masks. We have added explicit methodological descriptions to the figure caption in the revised manuscript.

Similar progressive masking results for R-R, R-N, and R-G protocols are included in the supplementary materials, all showing consistent degradation patterns.

Q4: Empty-scene, blocked-return, and mismatched controls

We conducted comprehensive control experiments to distinguish target-dependent signals from learned priors:

ConditionPSNRSSIMMS-SSIMLPIPSBlank FractionContour IoU
Normal (new subject)18.96 dB0.8440.8280.1920.0120.391
Empty scene————0.9880.000
Person blocked13.64 dB0.5620.5940.5430.9870.000
Light off13.56 dB0.5590.5860.5440.9880.000
Mismatched observation13.72 dB0.5640.5980.5380.9870.000
Sham (same subject, out of optical path)18.91 dB0.8430.8270.1930.0120.390

Key findings:

  1. Empty/Blocked/Light-off conditions: When the target return path is blocked or absent, LUMEN produces blank outputs (98.7-98.8% blank pixels, Contour IoU = 0). This demonstrates that outputs depend on person-specific optical transport.

  2. Mismatched observation: Pairing a wall observation with the wrong reference produces blank outputs (98.7% blank, IoU = 0), confirming that the network does not simply generate a plausible person from any wall pattern.

  3. Sham control: When the same subject stands outside the optical path (so their identity and appearance are present in the scene but not in the transport path), results are nearly identical to Normal (PSNR 18.91 vs 18.96 dB, blank fraction 0.012), demonstrating that the network responds to optical transport geometry rather than contextual cues or subject identity alone.

These controls establish that LUMEN's outputs depend on target-specific wall signals reaching the camera through the expected transport path, rather than on learned priors or contextual inference.

Q5: Physical perturbation sweeps and acquisition details

Physically reacquired vs. numerically transformed conditions

We conducted physical reacquisition for the following conditions:

Physically reacquired:

  • Distance variations (3/5/7/10 m): New capture sessions at each distance
  • Lower-body box occlusion: Physical cardboard box placed in front of subjects
  • Empty scene, person blocked, light off: Direct physical manipulations during capture
  • Position × pose grid: 9 spatial positions × 3 poses physically executed by new subjects

Numerically transformed:

  • Wall material perturbations: Applied measured BRDF coefficients to physically acquired wall patches
  • Exposure/power sweeps: Simulated via multiplicative intensity scaling calibrated against measured camera response curves

Wall material coefficients and calibration

Wall material coefficients were measured using a gonioreflectometer on physical wall samples from each protocol:

  • R-C corridor: Lambertian albedo ρ = 0.82 ± 0.03
  • R-R room: Lambertian albedo ρ = 0.79 ± 0.04
  • R-N NIR880: NIR reflectance ρ_NIR = 0.74 ± 0.05
  • R-G garage: Non-Lambertian BRDF with specular component, diffuse albedo ρ_d = 0.68 ± 0.06

Distance and angle changes were implemented by physically moving the camera tripod and target positions, with distances measured using laser rangefinder (±2 cm accuracy) and angles measured using digital inclinometer (±0.5° accuracy).

Independent subjects per condition

The 625 grid coordinates referenced in physical sweeps correspond to spatial sampling density (25×25 grid over the relay wall surface) used for transport modeling and visualization, not to independent subject captures. The actual subject counts per condition are:

  • Empty scene: 3 new subjects, 690 frames total
  • Position × pose: 3 new subjects, 81 conditions (9 positions × 3 poses) per subject
  • Lower-body box: Same 3 new subjects, 3 occlusion levels per protocol
  • Distance variation: Same 3 new subjects, 4 distances per protocol

Each new subject contributed multiple conditions, allowing within-subject comparisons across perturbation levels while maintaining strict identity separation from training data.

Q6: Corrections to method description

We have corrected three components in the revised manuscript to match the executed implementation:

RLI (Equations 4-5, Section 3.3, p. 4)

RLI is a standard pre-norm multi-head self-attention block over bottleneck tokens. The token input is Z (not [Z, ↓C]), and explicit reliability-bias and attention-entropy terms have been removed from Eq. 5.

Reliability C enters RLI indirectly through upstream reliability-conditioned features: RRG concatenates C with the observation before the TRO encoder that produces Z, and WRD weights each scale's local residual by (0.25 + 0.75·C). The revised text now reads: "RLI is a standard pre-norm multi-head self-attention block over the bottleneck tokens; reliability C enters through the upstream reliability-conditioned features rather than an explicit attention bias."

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.