acceptodds
Withdrawn from ICLR 2027

A Physical Measurement Benchmark for Passive Non-Line-of-Sight Imaging under Indoor Illumination and Environmental Reflections

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recovering hidden images from indirect light is a challenging problem in physical imaging. In illuminated indoor environments, target-dependent signals are mixed with ambient illumination and reflections from walls, furniture, and other surfaces, making it difficult to determine whether generative models preserve target-specific information or merely produce visually plausible outputs. We introduce a physical benchmark for passive non-line-of-sight image reconstruction under indoor illumination and environmental reflections. The benchmark contains 13,000 physically acquired measurement–target pairs spanning ten STL-10 categories, together with background references, fixed split manifests, and provenance information. The target content consists of 2D STL-10 images projected onto a hidden screen, while the physical contribution lies in the paired measurement acquisition and its evaluation protocol. We support deterministic measurement-to-image reconstruction and coarse-conditioned latent-diffusion refinement, and separately evaluate measurement-only inputs and additional class-conditioned settings. The benchmark provides unified pixel, structural, perceptual, and semantic metrics, together with matched evaluations of deterministic reconstruction, a pretrained latent diffusion model, and a full-UNet fine-tuned variant. Across the matched diagnostic comparisons, domain fine-tuning produces a pronounced perceptual shift at high denoising strength, substantially reducing LPIPS without improving pixel or structural metrics. These results reveal a consistent gap between perceptual plausibility and target-specific reconstruction fidelity rather than a uniform improvement across metrics.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Passive NLOS imaging, Hidden-target image reconstruction, Latent diffusion models, Inverse imaging, Computational imaging

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_to8ZOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

To address the lack of standardized, physically acquired datasets for passive non-line-of-sight (NLOS) imaging in complex indoor environments—characterized by ambient light and reflections from furniture and walls—this paper introduces a novel physical benchmark dataset and evaluation protocol for passive NLOS imaging.

  1. The benchmark comprises 13,000 "measurement-target" pairs physically captured within a controlled indoor conference room setting (covering 10 categories from the STL-10 dataset). It includes background references, a fixed data split (12,000 for training, 1,000 for testing), and comprehensive provenance information. The task is divided into two complementary reconstruction tracks: (1) A "measurement-only" track, which evaluates the ability of deterministic models (e.g., NLOSNet-T, U-Net, NAFNet, GAN, NLOS-OT) to recover hidden targets directly from weak, entangled passive observations; and (2) A "known-class-conditioned refinement" track, which utilizes a coarse initial reconstruction combined with external semantic class labels to perform generative refinement via a pre-trained latent diffusion model (Base LDM) or full U-Net supervised fine-tuning (Full-UNet SFT).

  2. Experiments reveal a significant trade-off: while deterministic models excel in pixel-level and local structural fidelity (as measured by PSNR/SSIM), generative diffusion-based refinement—though enhancing perceptual naturalness and distribution-level metrics (such as LPIPS/FID)—does not guarantee improved reconstruction fidelity for specific target instances. In some cases, the introduction of class priors may even cause the reconstruction to deviate from the actual ground-truth target.

Critical strengths
  1. Unlike previous benchmarks that relied heavily on purely synthetic simulations or active, time-resolved (laser-based) data, this work utilizes a dataset of 13,000 real-world paired samples collected in a physical environment featuring authentic indoor lighting (~300 lx) and complex multi-path reflections. This contribution is highly significant for advancing passive non-line-of-sight (NLOS) technologies toward practical applications, such as robotic search-and-rescue operations and blind-spot perception for autonomous vehicles.

  2. By rigorously controlling experimental variables, the authors clearly demonstrate that "perceptual quality" (as measured by LPIPS/FID) and "pixel-level fidelity" (as measured by PSNR/SSIM) can diverge in underdetermined passive NLOS tasks. This sober scientific assessment highlights the risk of "hallucinations"—where diffusion models produce results that are visually realistic yet factually inconsistent with the actual scene—serving as an important cautionary note for the broader Scientific Machine Learning (SciML) community.

Critical weaknesses
  1. In the main experiments presented in Table 1, deterministic models for direct reconstruction (such as NLOSNet-T and NAFNet) take only the processed passive measurement map Y~\tilde{Y} as input. In contrast, the "Base LDM" and "Full-UNet SFT" models—used in the refinement track—leverage an additional external input: the ground-truth class label CC (class condition). This architectural information asymmetry gives the diffusion models an advantage in terms of LPIPS and FID scores (Table 1). This performance gain is largely attributable to the strong semantic priors provided by the class labels, rather than being an inherent benefit of the generative diffusion architecture itself. To ensure a truly fair architectural comparison, the authors should include an "unconditional LDM" in the refinement track, or alternatively, evaluate the deterministic models under the same conditional settings (e.g., by providing class labels to a U-Net).

  2. As described in Section 3.2.1, because the projector casts images directly onto a hidden screen, the authors employ a "fixed image-space black mask" during preprocessing to manually suppress the direct specular highlights (hotspots) produced by the projector. In real-world passive non-line-of-sight (NLOS) scenarios—such as around a vehicle corner or within a completely occluded zone—the probe or target light sources are typically unknown and uncontrollable; therefore, such an ideal "prior mask" for direct hotspots does not exist. This preprocessing step implicitly leaks information about the projection geometry, thereby undermining the benchmark’s validity for evaluating scenarios involving unknown light sources in truly occluded environments.

  3. While the paper highlights challenges posed by furniture, walls, and ambient reflections in indoor settings, the dataset description lacks ablation studies regarding factors such as object reflectivity, surface roughness, or geometric variations within the scenes. The fundamental ill-posed nature of passive NLOS imaging stems from severe information ambiguity caused by multipath interference. Since the benchmark is based exclusively on data collected in a static conference room, it lacks testing for generalization across different room layouts or dynamic background occlusions, making it difficult to substantiate the robustness of the proposed methods in general indoor environments.

  4. In Section 3.3 and Appendix B, the authors state that the FID score is calculated using a full test set consisting of 1,000 predicted images and 1,000 target images. However, as a metric for measuring the distributional discrepancy between sets of images, FID typically requires a sufficiently large sample size—often on the order of tens of thousands—to ensure stable estimation of feature covariance. Given the small test set of only 1,000 samples, combined with an uneven distribution across 10 classes, calculating the FID on the entire set introduces significant statistical variance and bias (i.e., the "sample-size bias" of FID). Furthermore, the analysis lacks stratified statistics for individual classes; while the appendix notes that class-level FID was not computed, the overall reliability of FID results derived from such a small sample size remains questionable.

Questions for authors
  1. To effectively decouple the respective contributions of the "diffusion generative prior" and the "known class condition," have the authors evaluated the performance of the Base LDM or SFT models—when the class label is omitted during inference (i.e., the CnullC_{null} setting)—as presented in Table 1? Specifically, if the class label is excluded and the model relies solely on the diffusion prior to refine the output of NLOSNet-T, do the LPIPS and FID scores still outperform those of NAFNet or the original NLOSNet-T?

  2. To what extent does the fixed "black mask" simplify the mapping from measurements to the target scene? How significantly would the performance of NLOSNet-T and SFT degrade if this mask were removed during testing, or if the projector’s illumination position on the hidden screen were varied randomly? Does this imply that the current benchmark relies, to some degree, on the constraints of a fixed acquisition geometry?

  3. Given that the diagnostic test set contains only 1,000 images (averaging just 100 images per class), how have the authors verified that the FID scores calculated at this scale (e.g., 85.49 vs. 53.05 in Table 1) are statistically significant rather than merely the result of random noise? Could the authors provide additional stability verification, such as through bootstrapping or multi-scale structural similarity (MS-SSIM) analysis?

Reviewer_ZMMzOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper introduces a benchmark for passive non-line-of-sight image reconstruction under indoor illumination and environmental reflections. The benchmark contains 13,000 physically acquired measurement–target pairs acquired with a camera monitor pair. The authors evaluate several recent NLOS methods on this data but find no conclusive results. Unfortunately, the monitor/relaywall setup is inadequate to represent real scenes and, as such the benchmark has fundamental technical flaws.

Critical strengths

The paper addresses an important problem, namely benchmarking of NLOS methods. As an underexplored area, a benchmark with representative physical measurements for NLOS evaluation could fill a big gap in the research of indirect imaging methods. To this end, the benchmark must contain diverse and realistic scenes.

Critical weaknesses

Unfortunately, the acquisition protocol described in the paper does not allow for realistic and representative NLOS measurements. Specifically, the protocol is:

"To obtain paired measurement–target samples, we project two-dimensional RGB images onto the hidden screen and capture the corresponding passive camera observations. We use STL-10(Coates et al., 2011) as the source of hidden target images, covering ten object categories. For each target image Ih,i, the projected image is treated as the ground-truth hidden target, while the corresponding passive camera observation is recorded as Y raw i . The category label Ci is retained as optional semantic information and is used only in the known-class-conditioned refinement track defined in Section 3.1."

As such, the authors use a hidden "screen". Due to occlusion, multi-path and intensity falloff, this does not represent realistic NLOS scenarios and, as such, renders the benchmark unusable. Note that a planar scene image is dramatically simpler than a full 3D scene:

Chen, Wenzheng, et al. "Steady-state non-line-of-sight imaging." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2019.

Zhou, Y., Li, W., Li, W., Dai, C., Zeng, J. W., & Xu, F. (2025). Passive non-line-of-sight imaging at 10 meters. Optics Letters, 50(20), 6333-6336.

Overall, the benchmark has substantial flaws, and as such, is not adequate for evaluating NLOS methods.

Questions for authors

How would you extend your capture setup to realistic diverse 3D scenes.

Reviewer_sUwgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a controlled physical benchmark for passive non-line-of-sight image reconstruction under indoor reflected-light conditions, with 13,000 paired measurement–target records, hidden-screen targets from ten classes, and a fixed 12,000/1,000 training/diagnostic-test split. Its contributions are the benchmark, an auditable evaluation protocol with standardized preprocessing, checkpoint logging, reproducibility checks, and paired plus distributional metrics, and a controlled comparison between deterministic measurement-only reconstruction and known-class-conditioned latent-diffusion refinement; NLOSNet-T is provided as a coarse initializer, while DPO and GRPO are exploratory analyses. Validation uses the same measurement–target pairs and reports MSE, PSNR, SSIM, LPIPS, and FID across measurement-only and refinement tracks, plus a true-versus-null class-condition ablation and exploratory preference-optimization comparison. Results show deterministic models generally retain stronger paired fidelity, whereas diffusion refinement can improve perceptual and distributional scores without guaranteeing closer correspondence to a particular target; DPO and GRPO show no material improvement.

Critical strengths

Its main defensible strength is the controlled paired benchmark: 13,000 measurement–target records, ten target classes, a fixed 12,000/1,000 split, standardized preprocessing, and documented evaluation procedures. This matters because it enables instance-level and reproducible comparison of passive NLOS reconstruction methods under the same paired measurements, rather than relying on loosely controlled or non-paired evaluations.

The second strength I am willing to credit is the separation of measurement-only deterministic reconstruction from known-class-conditioned diffusion refinement, with paired fidelity metrics and a distribution-level diagnostic reported for each. This is consequential because it permits a controlled examination of the fidelity–perception trade-off and avoids treating methods with different inference-time information as a single unconditional ranking.

Critical weaknesses

The main limitation is the narrow physical instantiation of the benchmark: one illuminated indoor conference room, a 100-inch hidden screen at approximately 3 m, a fixed 640×480 camera, and projected STL-10 two-dimensional RGB targets from ten classes. This is presented as a repeatable physical proxy rather than a reproduction of every real deployment, which is a reasonable scope choice, but the evidence for broader passive NLOS conditions involving ambient light, multiple reflective objects, occlusions, and uncontrollable background interference remains limited. The contribution is therefore best read as a controlled benchmark rather than a general real-world validation.

A further limitation is that the measurement-only and known-class-conditioned refinement tracks use different inference-time information and reconstruction objectives, and the paper itself states that comparisons between them are not interpreted as direct rankings of model architectures. This limits what the cross-track results can establish about architecture-level superiority; they support a controlled analysis of the fidelity–perception trade-off, but the benchmark alone does not settle broader claims about which reconstruction paradigm is generally preferable.

Questions for authors

Will the release include the artifacts listed in Appendix C—specifically the 12,000/1,000 split manifest, raw-to-processed measurement mapping, background and mask artifacts, checkpoint hashes, per-sample predictions, and metric files—and will these be sufficient to reproduce the reported diagnostic-test results without accessing the test targets for model selection? Does the current benchmark contain any variation in room layout, ambient illuminance, relay surfaces, or target scale beyond the fixed indoor conference-room configuration described in Section 3.2, and if not, how should the reported results be scoped relative to the broader real-world conditions discussed in the introduction? Extending the benchmark to additional physical scenes, lighting conditions, and reflective configurations would help establish whether the observed fidelity–perception trade-off holds beyond the current controlled setting.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.