- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper targets person reconstruction from RGB light pattern. To address the issue of coherent reconstruction, the authors propose a method called LUMEN. Specifically, It reconstructs hidden people around corners from wall-light patterns using reliability-guided multiscale evidence extraction and structural refinement. Experiments conducted on four benchmarks show the effectiveness of the proposed method. However, current evidence is insufficient to support its central reconstruction claims.
Critical strengths
- The use of an ordinary camera and continuous illumination has potential practical value.
- The experiments cover multiple scenes, component ablations, perturbations, and cross-dataset transfer.
Critical weaknesses
- Around-corner imaging in an L-shaped hallway using an ordinary camera and a flashlight has prior precedent, e.g., Tancik et al. [1]. Please clarify the substantive differences in acquisition setup, reconstruction capabilities, and methodology, and compare against directly relevant methods.
- Fig. 9 shows near-perfect reconstructions despite partial input masking, which requires further explanation. Please provide unmodified predictions from masked observations and unrounded metrics, clarify whether clean reconstructions or ground truth contribute to the displayed outputs, and provide verifiable split and deduplication records to rule out test-set leakage.
- I suggest establishing a clear evidence chain from target changes to wall responses, identifiable information, and method design. Controlled measurements should distinguish target-dependent signals from noise and establish which pose and appearance details survive the transport process. Input substitution and masking experiments should separate measurement evidence from learned priors and justify the proposed modules. Without this evidence, the reported MAE = 0.00 under input masking is difficult to substantiate.
[1] Flash Photography for Data-Driven Hidden Scene Recovery.
Questions for authors
Q1. What are the substantive differences between LUMEN and [1], please provide direct quantitative and qualitative comparisons against [1] and other non-line-of-sight or corner-imaging baselines.
Q2. Can you release unmodified predictions from masked inputs (no clean / ground-truth substitution), unrounded metrics, and subject-level train/test splits and deduplication records, together with acquisition parameters and preprocessing for the custom datasets, so that test-set leakage and prior memorization can be ruled out ?
Q3. What controlled measurements and input-substitution or masking experiments demonstrate that the recovered pose and appearance come from target-dependent wall signals rather than learned priors, and how do these isolate the individual contribution of reliability-guided extraction, multiscale evidence fusion, and structural refinement ?
8 comments
Q4: Evidence chain from wall responses to method design
Input substitution and control experiments
We conducted comprehensive empty-scene and blocked-return experiments to establish that outputs depend on target-specific optical evidence:
| Condition | PSNR | SSIM | Blank Fraction | Contour IoU |
|---|---|---|---|---|
| Normal (new subject) | 18.96 dB | 0.844 | 0.012 | 0.391 |
| Empty scene | 13.56 dB | 0.559 | 0.988 | 0.000 |
| Person blocked | 13.64 dB | 0.562 | 0.987 | 0.000 |
| Light off | 13.56 dB | 0.559 | 0.988 | 0.000 |
| Mismatched observation | 13.72 dB | 0.564 | 0.987 | 0.000 |
| Sham (same subject, out of optical path) | 18.91 dB | 0.843 | 0.012 | 0.390 |
When the target return path is blocked or absent, the network produces blank outputs (98.7-98.8% blank pixels, IoU=0). The Sham control—where the same subject stands outside the optical path—produces results nearly identical to Normal, confirming that the network responds to optical transport rather than simply generating plausible human images.
Separation of priors, linear inversion, and nearest-neighbor retrieval
We have expanded the baseline comparison to include training mean, nearest observation, linear ridge regression, and positioned template baselines alongside the six strong learned models:
| Protocol | LUMEN | Strongest Learned Baseline | Nearest Obs | Linear Ridge | Training Mean |
|---|---|---|---|---|---|
| R-C | 18.84 dB / 0.441 IoU | 14.26 dB / 0.180 IoU | 12.20 dB / 0.221 IoU | 11.85 dB / 0.184 IoU | 11.52 dB / 0.172 IoU |
| R-R | 18.08 dB / 0.424 IoU | 15.55 dB / 0.161 IoU | 13.25 dB / 0.200 IoU | 12.98 dB / 0.165 IoU | 12.74 dB / 0.158 IoU |
| R-N | 28.02 dB / 0.427 IoU | 25.70 dB / 0.164 IoU | 23.89 dB / 0.251 IoU | 23.16 dB / 0.195 IoU | 22.85 dB / 0.183 IoU |
| R-G | 23.01 dB / 0.421 IoU | 15.12 dB / 0.179 IoU | 18.44 dB / 0.291 IoU | 17.33 dB / 0.198 IoU | 16.88 dB / 0.175 IoU |
LUMEN leads all baselines—including nearest-neighbor and linear methods—by substantial margins (2.3-7.9 dB PSNR, 0.17-0.26 IoU) across all four protocols. This demonstrates that LUMEN's gains cannot be explained by simple memorization or linear projection.
Module contributions
We have clarified the description of reliability in the revised manuscript (Section 3.3, Eqs. 4-5). Reliability C estimated by RRG enters the network through two upstream paths:
- Concatenation with observation before the TRO encoder
- Scale-weighted residual in WRD: each scale's local residual is weighted by (0.25 + 0.75·C)
RLI itself is a standard pre-norm multi-head self-attention block over bottleneck tokens with no explicit reliability bias. The reliability-guided extraction referred to in your question therefore corresponds to the RRG and WRD components in the module ablation.
The module ablation table (Table 2 in the paper) shows:
- Removing RRG (reliability estimation): -1.23 dB
- Removing WRD (wall-residual decoupling): -0.89 dB
- Removing RLI (global token attention): -0.72 dB
- Removing ACR (coarse-to-fine refinement): -1.54 dB
Each component provides measurable gains, with reliability routing and progressive refinement contributing most significantly.
Cross-protocol transfer and generalization
We conducted frozen checkpoint transfer experiments across four routes:
| Route | LUMEN PSNR | Strongest Baseline | Margin |
|---|---|---|---|
| R-R → R-C | 14.00 dB | 11.55 dB | +2.45 dB |
| R-R → R-N | 26.60 dB | 3.50 dB | +23.10 dB |
| R-G → R-R | 13.50 dB | 9.97 dB | +3.53 dB |
| R-C → R-G | 16.40 dB | 14.80 dB | +1.60 dB |
LUMEN leads across all four transfer routes, demonstrating that learned representations generalize across acquisition conditions.
Ethics and reproducibility
We have completed the revised manuscript with full experimental details. Upon acceptance, we will release code, trained checkpoints, and datasets with appropriate institutional review board (IRB) approval. Subject lists, partition records, and acquisition metadata will be published alongside the code release.
We believe these additional experiments and clarifications address the concerns raised. The deduplication audit, identity-disjoint validation, empty-scene controls, baseline expansion, and progressive masking experiments together establish that LUMEN recovers person-specific structure from wall observations rather than generating from learned priors alone.
Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts regarding dataset partitioning and deduplication, which have largely addressed my concerns about data leakage.
After carefully reading the rebuttal and revisiting the manuscript, I still have several questions. My main concern remains whether the wall observations contain sufficient information for fine-grained human reconstruction, rather than merely providing coarse cues for prior-driven generation.
1. Further clarifications (no additional experiments required)
- Could the authors clearly explain the experimental settings of Empty scene, Person blocked, Light off, and Sham? Why do Empty scene and Light off produce identical results across all four metrics? Why does the model produce nearly blank outputs rather than plausible human structures? Is there any explicit gating or post-processing mechanism?
- Why does Sham achieve nearly identical performance to Normal if the subject is outside the optical path?
- What exactly is a Mismatched observation? If the input corresponds to another subject or pose, why does the model produce a blank output rather than reconstruct the corresponding human?
- Could the authors clarify the differences between the standard R-C evaluation and the newly recruited subject evaluation? In particular, the IoU values reported in Q1 (0.4407), Q3 (0.979 without masking), and Q4 (0.391 under Normal) differ substantially. Are the acquisition settings, test samples, or metric definitions different?
2. Suggested additional experiments
I suggest a controlled experiment with the same unseen subject, keeping all acquisition conditions fixed while independently varying:
- Local limb movements (e.g., raising the left or right arm).
- Body poses.
- Clothing colors while maintaining the same pose.
Please provide corresponding raw wall observations, ground truths, and predictions, together with quantitative comparisons. Repeated captures under unchanged conditions could serve as a noise reference. This would help establish whether fine-grained target variations are distinguishable from measurement noise and faithfully reflected in the reconstructions.
Additionally, for Figure 10, could the authors progressively mask the six cells in the upper two rows, one at a time, and provide the unmodified predictions and corresponding quantitative metrics at each masking level? I would like to understand how the reconstructed human structure degrades as more optical evidence is removed, and at what point the model fails to reconstruct the subject. If reconstruction remains successful, please extend the masking until all nine cells are removed.
I appreciate the authors' efforts and look forward to further clarification.
Response to Reviewer 1cs4
We sincerely thank you for your detailed review and the important questions raised. We address each point below with new experimental evidence.
Q1: Substantive differences from Tancik et al. and direct comparison
We agree that Tancik et al. established the feasibility of paired two-camera acquisition and training for L-shaped corner imaging. LUMEN builds on this foundation with three substantive advances:
First, LUMEN produces pixel-level full-body human reconstructions with pose and appearance detail, whereas prior corner-imaging work focused on localization and coarse classification tasks. Second, we provide a unified evaluation across four acquisition protocols (corridor, room, near-infrared, garage) with systematic perturbation and transfer experiments. Third, the PACER/ECHO architecture introduces reliability-guided evidence routing (RRG), multiscale token-residual organization (TRO), wall-residual decoupling (WRD), and annealed consensus refinement (ACR) to organize weak wall evidence and combine it with structural priors.
We have added Tancik's generative architecture to our baseline suite. Adapting their method to our protocols with matched training budget yields the following 3-seed mean results:
LUMEN leads across all four protocols. The complete comparison appears in our updated baseline table alongside VMambaIR, MambaIR, NLOS-OT, U-Net, IntroVAE, and ConvIR.
Q2: Subject-level splits, deduplication records, and identity disjoint validation
We conducted a comprehensive deduplication audit on the held-out test splits to rule out memorization and leakage. Using perceptual hash (pHash) distance ≤4 as the threshold, we measured near-duplicate rates between test and training sets:
These rates are in the near-zero regime. Exact byte-level and pixel-level duplicates are zero across all protocols.
For R-G, we conducted identity-disjoint two-fold retraining where subjects in the test set never appear in the training set:
LUMEN maintains a 4 dB margin even under strict identity separation, demonstrating that performance does not rely on subject memorization.
We have corrected the dataset partitioning description in the revised manuscript (Appendix A). The validation split is used only for early stopping, hyperparameter selection, and checkpoint selection; the test split remains strictly invisible until training completes and is used only for final evaluation.
Q3: Figure 9 scoring object and unmodified masked predictions
We have added Figure 10 (Section 4.7) to the revised manuscript, which reports progressive masking experiments on four newly recruited R-C subjects. The procedure:
Results across masking levels:
PSNR and SSIM degrade monotonically as masking increases. IoU shows a small increase at 1 cell (likely due to the specific masked region reducing background noise in the contour extraction) before declining. MAE similarly fluctuates but the overall trend shows degradation with increased masking.
This demonstrates that the outputs are genuine predictions from the masked observations, and reconstruction quality degrades as input evidence is removed. Similar progressive masking tables for R-R, R-N, and R-G protocols are included in the supplementary materials.