acceptodds
Under review as a conference paper at ICLR 2027

Inferring Objects Around Corners from Indirect Wall Light: Structure-Preserving Silhouette Recovery with RUBIN

Abstract

We ask whether a conventional RGB camera can reveal a target silhouette from indirect wall light around a corner. We propose RUBIN, a structure-preserving framework that addresses acquisition-related radiance variation and boundary ambiguity caused by diffuse transport. Radiance-Consistent Structural Learning (RCSL) constructs position-preserving radiance views and aligns global representations, co-located local features, and dense structural relations. Variance and covariance regularization preserve descriptor diversity and reduce redundancy, while an acquisition-factor branch discourages encoding of sampled acquisition factors. Uncertainty-Gated Boundary Region Refinement (UGBR) combines coarse region predictions with multiscale boundary evidence and weights relational corrections using region uncertainty and boundary agreement. Experiments on five measured datasets show stronger overall silhouette recovery than twenty representative methods. Ablations support the complementary roles of RCSL and UGBR, and observation-variation experiments examine the shape information retained as wall-light patterns change. The recovered foreground extent, major components, and basic contours support preliminary hidden-target shape inference from wall light in this setup.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
computer vision, deep learning, object detection, physics-guided representation learning, light-pattern information mining, hidden support discovery

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_BWYdOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies non-line-of-sight (NLOS) silhouette recovery from indirect wall illumination captured by a conventional RGB camera. The authors propose RUBIN, a framework built on two components: Radiance-Consistent Structural Learning (RCSL) and Uncertainty-Gated Boundary Region Refinement (UGBR). RCSL aims to learn robust structural representations under radiance variations by enforcing global, local, and dense affinity consistency across augmented observations, together with statistical regularization and acquisition-factor suppression. UGBR combines coarse region predictions with multi-scale boundary information to refine uncertain foreground regions. The method is evaluated on five physically measured NLOS datasets built from existing salient object detection collections. RUBIN is compared against both direct-view baselines and NLOS architectures and shows improved silhouette recovery, and ablation studies examine the contribution of each component.

Critical strengths
  1. The paper investigates passive NLOS silhouette recovery using a conventional RGB camera. Evaluation on physically measured observations, rather than purely synthetic data, strengthens the practical relevance and provides useful evidence for learning-based passive NLOS imaging.
  2. Combining radiance-consistent representation learning with uncertainty-guided boundary refinement is a reasonable design for this setting. The method achieves competitive results against adapted baselines on five author-collected NLOS datasets derived from existing salient object detection datasets. The component-level ablations on NLOS-ECSSD suggest that RCSL and UGBR provide complementary improvements.
Critical weaknesses
  1. A highly relevant prior work, Wavefront-Constrained Passive Obscured Object Detection (WavePCNet, AAAI 2026), is not discussed in the paper. WavePCNet addresses a closely related task of recovering obscured-object masks from indirect light observations and evaluates on four physically collected datasets derived from DUTS, HKU-IS, ECSSD, and DUT-OMRON, the same source collections used in this submission. This omission raises concerns about the positioning and novelty of the work, as well as the completeness of the experimental comparison. In particular, WavePCNet appears to be a more directly relevant baseline than several of NLOS methods shown in Table 1. The authors should clarify the differences in task formulation, acquisition setup, and dataset construction, and include a comparison where feasible.
  2. The proposed framework mainly combines established techniques, including photometric augmentation, feature consistency learning, variance/covariance regularization, gradient reversal, and uncertainty-guided boundary refinement. Although Eq. (1) introduces a light-transport formulation, the proposed network does not explicitly incorporate this model. In contrast, WavePCNet (AAAI 2026) embeds a Fresnel-based propagation module into its network for a closely related passive NLOS segmentation task. The authors do not sufficiently explain how their architectural and learning choices are specifically motivated by the characteristics of NLOS imaging. In addition, the experiments do not adequately isolate the sources of the reported improvements. Table 1 does not specify whether the adapted baselines are trained with comparable augmentation and training strategies. Table 2 provides only limited fine-grained ablations of RCSL and UGBR, leaving the individual contributions of several consistency objectives, auxiliary losses, and gating mechanisms unclear. It is therefore difficult to determine how much of the improvement stems from the proposed designs rather than from augmentation, auxiliary training objectives, or model capacity.
Questions for authors

**Q1. Relation to WavePCNet. ** How does this work differ from WavePCNet (AAAI 2026) in terms of task formulation, acquisition setup, and methodological contributions? What is the relationship between the NLOS datasets used in this paper and the OOD datasets reported in WavePCNet? Could the authors provide a comparison with this closely related method, or explain why such a comparison is not feasible?

**Q2. Fairness and reproducibility of baseline comparisons. ** Could the authors provide more details on how the NLOS baselines in Table 1 were adapted to the RGB-to-mask setting, particularly those originally designed for transient measurements or different prediction targets? What pretrained weights, augmentation strategies, and optimization settings were used for each baseline, and how was training fairness ensured? Training a strong baseline such as GateNet with the proposed radiance augmentation would help isolate the contribution of the network design.

**Q3. Contributions of individual components. ** Could the authors provide finer-grained ablations of RCSL using final segmentation metrics, particularly for local consistency, affinity consistency, variance/covariance regularization, and adversarial factor suppression? For UGBR, how do region uncertainty gating and boundary confidence gating perform individually? How sensitive are the results to the main gating parameters and loss weights?

**Q4. Dataset construction and generalization. ** Could the authors clarify how the measured NLOS datasets and their train/test splits were constructed, particularly whether the same source images appear in both partitions? For example, NLOS-ECSSD contains 1,000 training and 200 test pairs, whereas the original ECSSD dataset contains only 1,000 images. How were the additional observations obtained, and how was train/test independence ensured?

**Q5. Robustness to physical acquisition variations. ** Figure 5 presents qualitative results under physically varied acquisition conditions, whereas Appendix E reports quantitative results based on computational perturbations of recorded NLOS-ECSSD observations. Could the authors clarify how these two experimental settings relate to each other? In particular, are quantitative results available for the acquisition variations shown in Figure 5, including camera pose, camera distance, source brightness, and relay-wall material?

4 comments

Response to Reviewer BWYd

Thank you for the detailed questions. We address each below.

Q1. WavePCNet and our contribution.

We retrained WavePCNet on our measured NLOS datasets under the same data partitions, input resolution, and target-mask supervision used for all methods, and report its best-checkpoint result per dataset. Because WavePCNet's original paper reports results on its own separately collected OOD data rather than our observations, we compare against the retrained model on our data. Each cell is Max-F ↑ / MAE ↓ / Dice ↑:

DatasetWavePCNet (retrained, best checkpoint)RUBIN
DUTS0.678 / 0.140 / 0.6300.692 / 0.136 / 0.641
HKU0.547 / 0.166 / 0.4800.565 / 0.161 / 0.497
ECSSD0.609 / 0.184 / 0.5240.751 / 0.109 / 0.697
DUT-OMRON0.444 / 0.138 / 0.3490.447 / 0.134 / 0.361

RUBIN exceeds the retrained WavePCNet on Max-F, MAE, and Dice on all four shared datasets. The largest margins appear on ECSSD (Dice 0.697 vs 0.524); the remaining datasets show consistent but smaller gains, with the closest on DUT-OMRON.

Both methods address passive mask recovery from indirect light but take different design routes. WavePCNet embeds a Fresnel-based propagation module, committing the network to an explicit transport operator. RUBIN instead treats transport and appearance as latent (Eq. 1) and targets the two degradations that dominate our passive conventional-camera setting: acquisition-dependent radiance variation at fixed target support, and diffuse boundary mixing. RCSL learns radiance-consistent structure across position-preserving radiance views, and UGBR refines uncertain foreground regions using boundary agreement. The consistent advantage over a physics-embedded baseline on our measured data supports this design choice for the passive RGB setting, where the transport operator is unknown and uncalibrated. We will document the measurement provenance and partition correspondence of the shared source collections explicitly.

Q2. Adaptation and fairness.

The adapted methods in Table 1 share RGB wall inputs, target-mask supervision, data partitions, input resolution, and evaluation. Representative adaptations are:

MethodRGB-to-mask adaptation
NLOS-LTMAn RGB encoder estimates a transport vector that modulates decoder features; a one-channel head predicts mask logits.
PAC-NetBoth branches receive the same wall image; propagation and calibration blocks process it, and the dense branch predicts the mask.
NLOSTA ResNet50 encoder provides 2-D RGB feature tokens to attention blocks followed by a mask decoder.
LEAPA learned RGB adapter supplies a latent volume to frequency filtering and 3-D features; a 2-D readout predicts the mask.

These comparisons evaluate the adapted architectures within the RGB-to-mask task. In particular, LEAP's latent volume is learned from RGB input rather than obtained from transient measurements.

The adapted NLOS baseline recipe uses 384×384 inputs, Adam for 100 epochs, encoder/other learning rates of 1e-4/1e-3, weight decay 5e-4, and equally weighted BCE and IoU losses. Learning-rate multipliers are 1, 0.1, and 0.01 over the first 20%, next 30%, and final 50% of training. ResNet50-based adaptations are configured for ImageNet initialization, while added layers and dedicated encoders use random initialization.

RUBIN's configuration follows Section 4.1, including a batch size of 5 and seed 3407 for the main-paper results. Its additional radiance views and RCSL/UGBR objectives are method-specific training components. We therefore distinguish the shared data and evaluation protocol from these additional objectives, whose effects are examined through augmentation controls and fine-grained ablations. Evaluation uses the epoch-100 checkpoint and the same prediction resizing, normalization, 8-bit conversion, and metric computation procedure. The revision will include per-method initialization, augmentation, loss, and optimization details.

NLOS-ECSSD augmentation controls; each cell is Max-F ↑ / MAE ↓ / Dice ↑:

SettingResult
Full RUBIN0.751 / 0.109 / 0.697
GateNet0.621 / 0.184 / 0.565
GateNet + radiance augmentation0.647 / 0.159 / 0.598
Backbone without RCSL/UGBR0.594 / 0.189 / 0.528
Backbone + radiance augmentation0.628 / 0.177 / 0.576

GateNet with radiance augmentation reaches Dice 0.598, compared with 0.565 for the baseline. The backbone with radiance augmentation reaches 0.576. RUBIN reaches 0.697, exceeding these two augmentation-only controls by 0.099 and 0.121, respectively. The following ablations further examine the effects of the structural objectives and refinement mechanisms.

Q3. Objectives, gates, and sensitivity.

ECSSD ablations; each cell is Max-F ↑ / MAE ↓ / Dice ↑:

SettingResult
Full RUBIN0.751 / 0.109 / 0.697
Generic refiner0.701 / 0.136 / 0.637
Without global consistency0.728 / 0.124 / 0.674
Without local consistency0.732 / 0.120 / 0.679
Without affinity consistency0.710 / 0.126 / 0.674
Without variance regularization0.735 / 0.121 / 0.663
Without covariance regularization0.734 / 0.120 / 0.650
Without factor suppression0.730 / 0.122 / 0.682
Without both gates0.709 / 0.128 / 0.664
Region gate only0.716 / 0.129 / 0.656
Boundary gate only0.714 / 0.127 / 0.638
Without region loss0.705 / 0.133 / 0.644
Without boundary loss0.706 / 0.134 / 0.636
Without relation loss0.726 / 0.124 / 0.672
Without topology loss0.725 / 0.123 / 0.673

The tables quantify each RCSL objective, gate, and auxiliary loss using segmentation metrics. Both gates together give Dice 0.697, versus 0.656/0.638 with individual gates and 0.664 with neither. These results support the joint gating configuration; the individual gates do not consistently improve every metric when used alone. Loss removals probe the objectives, while the generic refiner provides a comparison with an alternative refinement design.

Sensitivity results; entries give the range across the listed settings:

Parameter valuesMax-F ↑MAE ↓Dice ↑
tau_band: 0.30, 0.600.7510.1120.665–0.687
T_band: 0.04, 0.160.745–0.7460.1140.669–0.678
gamma_B: 2, 80.736–0.7400.115–0.1200.677–0.679
lambda_RCSL: 0.025, 0.100.730–0.7350.120–0.1220.673–0.678
lambda_UGBR: 0.125, 0.500.726–0.7300.123–0.1240.663–0.681

These results quantify sensitivity over the tested settings in all three metrics.

Q4. Dataset construction.

We expanded the ECSSD-derived collection by 200 examples, yielding 1,200 measured pairs with a 1,000/200 partition. We will detail the extension and source-image/observation assignments to each partition. After acceptance, we will publicly release the dataset, paired masks, and split lists.

Q5. Physical and computational variations.

Figure 5 provides acquired-image examples with changes in camera pose, distance, brightness, and relay surface. Appendix E quantifies computational angle transforms, illumination changes, and compression of recorded observations using fixed models. We will make their scope explicit in the captions and protocol.

Thank you to the authors for the rebuttal. Unfortunately, not all concerns raised are addressed clearly. For example, Q4 & Q5 are not elaborated properly. There will be no change in the rating.

Thank you for your comment.

Reply to Q4: We have added an overview of all 200 additional samples used to extend NLOS-ECSSD in Appendix A. The figure presents the source images, corresponding measured relay-wall observations, and binary ground-truth masks, accompanied by a description of the sample diversity and dataset expansion. We confirm that no source image appears in both the training and test sets after a careful check. Please do not share these additional images with anyone else bacasue they are under review by the administrative board of my institute for public release.

Reply to Q5: Figure 5 and Appendix E are linked. Figure 5 is the visible results, while Appendix E shows the values (numerical results) of Figure 5. 'computational' is not a proper word in this context, and we will revise this wording issue in the revision.

Thank you for recognizing our clarifications and the strengthened evidence. We sincerely appreciate your thoughtful assessment and constructive feedback. If our response resolves your concerns, we respectfully ask you to consider increasing your score to support our work, it is a huge motivation for us.

Reviewer_KmMQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper mainly studies how to recover the foreground mask of a hidden two-dimensional target image from a single RGB observation of indirect light on a relay wall. The paper proposes a method that using alignment learning at different scales between perturbed images and original images to learn structural representations that remain stable under photometric changes, and refine the foreground mask through residual correction guided by uncertainty and multi-scale boundary consistency. The authors report that the method outperforms 20 adapted baselines on three main metrics across five datasets.

Critical strengths
  1. The paper includes controlled ablations of the main modules and comparisons with alternative designs, which helps in understanding the role of each module.

  2. The task is clearly defined and requires only a single RGB observation of the wall, so the hardware requirements for data acquisition are simple.

Critical weaknesses
  1. The paper reports its largest improvement on NLOS-ECSSD. I therefore looked into the ECSSD dataset (https://www.cse.cuhk.edu.hk/leojia/projects/hsaliency/dataset.html). On its project page, I could only find the 200-image CSSD (the earlier version) and the 1,000-image ECSSD (the extended version). However, the paper uses 1,000 samples for training and 200 for testing. It is unclear to me where these data come from and how they were split. Could the authors provide a more detailed description of the data sources?

  2. The subsequent ablation studies are reported only on NLOS-ECSSD, so it remains unclear whether the gains can be reproduced on the other datasets. I would suggest an ablation of the key components on one additional dataset.

  3. The paper states, "We demonstrate the feasibility of recovering basic hidden-target shape from indirect wall light captured by a conventional camera.". However, NLOS-OT has already demonstrated the feasibility of recovering hidden two-dimensional images from indirect light observations. Although this paper further focuses on directly recovering the foreground mask, it remains unclear to me what this adds in terms of feasibility. I would appreciate further clarification of the scope of this contribution.

Questions for authors

Please see Weaknesses 1–3.

3 comments

Response to Reviewer KmMQ

Thank you for highlighting the task formulation and component analysis. We address the dataset, additional evaluation, and scientific contribution below.

1. ECSSD source and the 1,000/200 split.

NLOS-ECSSD is our expanded, physically measured benchmark. We added 200 examples ourselves to the ECSSD-derived collection, yielding 1,200 paired wall observations and target masks, partitioned into 1,000 training and 200 test pairs. Each model receives a measured wall RGB image and the corresponding foreground annotation as supervision. We will describe the extension, its construction, and source-image and observation assignments to the partitions. After the paper is accepted, we will publicly release the dataset, paired masks, and split lists.

2. Component benefits on another dataset.

Additional ablation on NLOS-HKU:

VariantMax-F ↑MAE ↓Dice ↑
Without RCSL and UGBR0.5080.1720.326
Without UGBR0.5300.1680.342
Without RCSL0.5450.1660.375
Full RUBIN0.5650.1610.497
Without global consistency0.5540.1610.450
Generic photometric views0.5600.1610.445
Generic refiner replacing UGBR0.5470.1650.412
Relation ablation0.5530.1610.431
Coarse-region ablation0.5490.1620.427

On NLOS-HKU, full RUBIN improves Max-F by 0.057 and Dice by 0.171 over the backbone without either component, while reducing MAE by 0.011. RCSL and UGBR individually raise Dice from 0.326 to 0.342 and 0.375; together they reach 0.497. Full RUBIN also exceeds the global-consistency removal (0.450), generic photometric views (0.445), generic refiner (0.412), relation ablation (0.431), and coarse-region ablation (0.427). These controls support the complementary contributions of radiance-consistent structure and coordinated region/boundary refinement on a second measured dataset.

3. Contribution relative to NLOS-OT and NLOST.

Three-seed comparison of reconstruction and mask prediction. Each entry is seed 42 / 1024 / 2024 (mean ± SD). SD is the population standard deviation of the displayed values, consistent with the training-stability analysis.

MethodMax-F ↑MAE ↓Dice ↑
NLOS-OT reconstruction → GateNet, zero-shot0.415 / 0.417 / 0.413 (0.415 ± 0.002)0.264 / 0.266 / 0.265 (0.265 ± 0.001)0.344 / 0.347 / 0.344 (0.345 ± 0.001)
NLOS-OT reconstruction → GateNet, fine-tuned0.547 / 0.550 / 0.547 (0.548 ± 0.001)0.211 / 0.209 / 0.210 (0.210 ± 0.001)0.481 / 0.484 / 0.481 (0.482 ± 0.001)
NLOS-OT, direct mask0.600 / 0.602 / 0.601 (0.601 ± 0.001)0.185 / 0.183 / 0.184 (0.184 ± 0.001)0.540 / 0.542 / 0.541 (0.541 ± 0.001)
NLOST, direct mask0.589 / 0.591 / 0.590 (0.590 ± 0.001)0.186 / 0.184 / 0.185 (0.185 ± 0.001)0.532 / 0.534 / 0.533 (0.533 ± 0.001)
RUBIN0.750 / 0.752 / 0.751 (0.751 ± 0.001)0.110 / 0.108 / 0.109 (0.109 ± 0.001)0.696 / 0.698 / 0.697 (0.697 ± 0.001)

NLOS-OT provides an important foundation for reconstructing hidden images from steady-state indirect light. To examine the relationship directly, we compare its reconstruction followed by GateNet segmentation with direct mask prediction. With fine-tuned GateNet, the reconstruction route achieves mean Dice 0.482, while direct-mask NLOS-OT reaches 0.541. RUBIN reaches 0.697, an increase of 0.156 over direct-mask NLOS-OT and 0.215 over the fine-tuned reconstruction route. The zero-shot reconstruction route reaches 0.345. In these tested settings, direct mask supervision preserves more task-relevant shape, and RUBIN further improves silhouette recovery.

RUBIN targets the hidden foreground's spatial support from a single steady-state RGB wall observation. RCSL encourages structural consistency across radiance changes that preserve geometry, and UGBR combines coarse foreground estimates with multiscale boundary evidence to refine uncertain regions. Together, they address the mixing of target structure with acquisition-dependent radiance and weak, diffuse contour evidence. The resulting silhouette directly describes foreground extent, orientation, and shape.

NLOST is a distinct prior method developed for transient measurements. In our direct-mask comparison, its adapted RGB-to-mask version achieves mean Dice 0.533; RUBIN improves this by 0.164. Our feasibility claim concerns basic silhouette recovery with the proposed framework and a single ordinary-camera steady-state observation. We will clarify this scope and the relationship to prior NLOS reconstruction in the revision.

Reviewer_KmMQOpenReview ↗

Comment by Reviewer KmMQ

Thank you for the response. The new ablation on NLOS-HKU and the multi-seed comparison address my concerns in W2 and W3. However, W1 remains unresolved: the response does not specify the source of the 200 added images, which split(s) they are assigned to, or whether any source image appears in both the training and test sets. Since NLOS-ECSSD accounts for the largest performance gains and the main ablations, I will maintain my current rating.

Thank you for your comment. We have added an overview of all 200 additional samples used to extend NLOS-ECSSD in Appendix A. The figure presents the source images, corresponding measured relay-wall observations, and binary ground-truth masks, accompanied by a description of the sample diversity and dataset expansion. We confirm that no source image appears in both the training and test sets after a careful check. Please do not share these additional images with anyone else because they are under review by the administrative board of my institute for public release. If our response resolves your concerns, we respectfully ask you to consider increasing your score to support our work, it is a huge motivation for us.

Reviewer_PvsSOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RUBIN for recovering hidden-object silhouettes from indirect wall illumination. It combines structure-consistent learning with uncertainty-guided boundary refinement and evaluates the method on measured NLOS datasets.

Critical strengths
  1. The problem is interesting, and the proposed components are well motivated.
  2. The evaluation includes real measurements, multiple baselines, and ablation studies showing the effectiveness of the proposed modules.
Critical weaknesses
  1. The adaptation of different baseline methods to the same task could be explained more clearly to establish comparison fairness.
  2. Some performance improvements are relatively small, and their statistical stability is not fully demonstrated.
Questions for authors
  1. Could the authors clarify the baseline adaptation and training protocols?
4 comments

Response to Reviewer PvsS

Thank you for recognizing the motivation and measured-data evaluation. We address stability and baseline fairness below.

1. Stability across seeds.

RUBIN's results for three seeds are shown below. Mean ± SD uses the population standard deviation of the displayed values.

DatasetMetricSeed 42Seed 1024Seed 2024Mean ± SD
NLOS-HKUMax-F ↑0.5640.5660.5650.565 ± 0.001
NLOS-HKUMAE ↓0.1620.1600.1610.161 ± 0.001
NLOS-HKUDice ↑0.4960.4980.4970.497 ± 0.001
NLOS-ECSSDMax-F ↑0.7500.7520.7510.751 ± 0.001
NLOS-ECSSDMAE ↓0.1100.1080.1090.109 ± 0.001
NLOS-ECSSDDice ↑0.6960.6980.6970.697 ± 0.001
NLOS-DUTSMax-F ↑0.6910.6930.6920.692 ± 0.001
NLOS-DUTSMAE ↓0.1360.1350.1360.136 ± 0.000
NLOS-DUTSDice ↑0.6400.6420.6410.641 ± 0.001
NLOS-DUT-OMRONMax-F ↑0.4480.4460.4470.447 ± 0.001
NLOS-DUT-OMRONMAE ↓0.1330.1350.1340.134 ± 0.001
NLOS-DUT-OMRONDice ↑0.3630.3600.3610.361 ± 0.001
NLOS-PASCALMax-F ↑0.4890.4870.4880.488 ± 0.001
NLOS-PASCALMAE ↓0.2080.2090.2080.208 ± 0.000
NLOS-PASCALDice ↑0.4110.4120.4100.411 ± 0.001

Results are closely grouped on all five datasets with seeds 42, 1024, and 2024. The largest seed-to-seed range is 0.002 for Max-F and MAE and 0.003 for Dice. Standard deviations are at most 0.001 at the reported precision; 0.000 indicates rounding to three decimals. ECSSD Dice ranges from 0.696 to 0.698, and HKU Dice from 0.496 to 0.498. These results support RUBIN's stability across the tested seeds. Table 1 reports a separate run using seed 3407.

2. Baseline adaptation and comparison protocol.

The adapted methods in Table 1 map a single steady-state RGB wall observation to a binary target mask. They share the data partitions, input resolution, target-mask supervision, and evaluation procedure. Representative adaptations are:

MethodRGB-to-mask adaptation
NLOS-LTMAn RGB encoder estimates a transport vector that modulates decoder features; a one-channel head predicts mask logits.
PAC-NetBoth branches receive the same wall image; propagation and calibration blocks process it, and the dense branch predicts the mask.
NLOSTA ResNet50 encoder provides 2-D RGB feature tokens to attention blocks followed by a mask decoder.
LEAPA learned RGB adapter supplies a latent volume to frequency filtering and 3-D features; a 2-D readout predicts the mask.

Transient-based adaptations learn features from RGB observations and mask supervision; LEAP's latent volume is a learned representation rather than a measured transient volume. These experiments compare the adapted architectures within the RGB-to-mask task, rather than their performance under their original sensing modalities.

3. Training and evaluation details.

Inputs and masks are resized to 384×384. RGB inputs use ImageNet normalization, and masks are binarized. As reported in Section 4.1, RUBIN uses Adam for 100 epochs, a batch size of 5, weight decay 5e-4, and initial learning rates of 1e-4 for the encoder and 1e-3 for the remaining parameters. The main-paper results use seed 3407.

The adapted NLOS baseline recipe also uses Adam for 100 epochs, weight decay 5e-4, and encoder/other learning rates of 1e-4/1e-3. Its learning-rate multipliers are 1, 0.1, and 0.01 over the first 20%, next 30%, and final 50% of training. ResNet50-based adaptations are configured for ImageNet initialization; newly added layers and dedicated encoders in NLOS-LTM, PAC-Net, and LEAP use random initialization.

Baseline mask supervision combines BCE and IoU losses with equal weights. RUBIN additionally constructs position-preserving radiance views and optimizes L_seg + 0.05 L_RCSL + 0.25 L_UGBR. Thus, the shared data and evaluation protocol does not imply identical training objectives: the additional objectives are part of RUBIN and are examined through the ablations.

Evaluation uses the epoch-100 checkpoint, bilinear resizing to target-mask resolution, probability-map normalization, and 8-bit conversion before metric accumulation. RUBIN requires only one RGB observation at inference. We will include a per-method configuration summary covering initialization, augmentation, losses, and optimization settings in the revision.

4. Augmentation control.

To examine the role of radiance augmentation, we evaluate GateNet and the backbone without RCSL/UGBR with and without the proposed augmentation.

MethodMax-F ↑MAE ↓Dice ↑
GateNet0.6210.1840.565
GateNet + radiance augmentation0.6470.1590.598
Backbone without RCSL/UGBR0.5940.1890.528
Backbone + radiance augmentation0.6280.1770.576
Full RUBIN0.7510.1090.697

The GateNet augmentation variant achieves Dice 0.598, compared with 0.565 for the baseline. Full RUBIN exceeds this augmentation variant by 0.099 in Dice and 0.104 in Max-F, while reducing MAE by 0.050. The backbone with radiance augmentation reaches Dice 0.576, compared with RUBIN's 0.697. RUBIN therefore retains an advantage over both augmentation-only controls in these experiments. The component and objective ablations further examine the contributions of structural learning and guided refinement. We will add the protocol and results to the revision.

Thank you for the detailed responses and additional experiments. The clarifications and new results have helped address my concerns. I appreciate the authors' efforts.

Thank you for recognizing our clarifications and the strengthened evidence, and the additional evidence, specifically the multi-seed stability results, the baseline adaptation protocol, and the detailed training and evaluation specifications, have addressed your concerns. We sincerely appreciate your thoughtful assessment and constructive feedback.

If this resolves your concerns, we respectfully ask you to consider increasing your score to support our work, it is a huge motivation for us.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.