acceptodds
Under review as a conference paper at ICLR 2027

ReflectWorld: Seeing Two Worlds Through Glass via Multi-View Geometry-Routed Diffusion

Abstract

Image reflection separation aims to recover the transmission and reflection layers from mixed observations captured through glass. Recent advances in large-scale pretrained models have substantially improved reflection recovery, yet most existing methods remain limited to single-image inference, where severe layer entanglement leaves the decomposition fundamentally ambiguous. Multi-view observations offer a natural way to reduce this ambiguity, as transmission is largely consistent across viewpoints whereas reflection is more view-dependent. However, directly exploiting such cross-view cues is difficult because reflections themselves corrupt the geometric correspondences required to establish reliable associations across views. To address this, we propose ReflectWorld, a geometry-routed diffusion framework that employs GeoRCA to establish reliable cross-view associations from corrupted multi-view geometry, while MGR-Diff drives differential generation with the distinct cross-view variations of the two layers. In addition, we introduce MVRefSep, a multi-view reflection separation dataset that captures the physical characteristics of reflections across views, establishing a benchmark for evaluating both transmission and reflection recovery. Experiments demonstrate that ReflectWorld achieves state-of-the-art performance. The code and dataset will be publicly released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-view reflection separation, Image layer decomposition, Diffusion model, Cross-view geometry

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_JX5aOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes ReflectWorld, a diffusion-based framework for jointly recovering transmission and reflection layers from multi-view images captured through glass. It combines a pretrained feed-forward geometry model with a one-step diffusion model. Its Geometry-Routed Cross-View Association module projects reference-view features into the target view and aggregates them using geometric confidence and content-based attention. The resulting cross-view features guide the two output branches through opposite-sign modulation, intended to exploit differences in the layers’ cross-view behavior. The paper also introduces MVRefSep, a dataset containing 40 synthetic scenes with ground truth for both layers and 10 real-world scenes with transmission ground truth. Validation includes comparisons with single-image and multi-image methods, component and design ablations, geometric correspondence analyses, and downstream novel-view synthesis and depth-consistency evaluations. The reported results show improved recovery of both layers on the synthetic benchmark and improved transmission recovery on the real-world dataset.

Critical strengths
  1. The paper addresses a central difficulty in multi-view reflection separation: a geometrically correct correspondence may still connect reflection-contaminated content. Combining geometry-constrained candidate routing with content-dependent attention provides a well-motivated way to use imperfect geometric information for joint transmission and reflection recovery. This task-specific design is the strongest methodological consideration supporting acceptance.

  2. The proposed dataset and reported improvements provide promising empirical contributions. MVRefSep contains 40 synthetic scenes with ground truth for both layers and 10 real-world scenes with transmission ground truth. On the synthetic benchmark, the method improves average transmission and reflection PSNR over the strongest respective competitors in Table 1 by 1.64 dB and 2.64 dB. These results suggest meaningful potential.

Critical weaknesses
  1. Section 5.4 states that all ablations are conducted on MVRefSep-Test, yet Appendix D.4 explicitly uses those results to select the default backbone, diffusion timestep, VAE LoRA configuration, and loss weights. No separate validation set is described. As written, the experimental protocol therefore appears to use the test set for configuration selection and then report performance on that same set. This can produce optimistic results even when training and test scenes do not overlap. The authors are recommended to identify the independent data used for model selection or evaluate the finalized configuration on previously unused test scenes.

  2. The interpretation of the reversed-direction ablation in Table 5 is inconsistent with Eqs. (2) and (9). With $ Δ=WOh\Delta=W_Oh $, the proposed configuration gives $ FT=F+ΔF_T=F+\Delta $ and $ FR=F−ΔF_R=F-\Delta $. Setting $ WO′=−WOW'_O=-W_O $ in the reversed configuration yields identical branch features, so both configurations have the same representational capacity if the output projection is freely trainable. Their performance difference therefore cannot establish that the chosen signs intrinsically reinforce transmission and suppress reflection. The authors should clarify whether the variants were retrained or modified only at inference, identify any constraints breaking this equivalence, and revise the mechanistic claim. This somehow weakens the differential-decoding explanation without ruling out the empirical benefit of opposite-sign updates.

Questions for authors
  1. Please clarify whether the reversed-direction variants in Table 5 were retrained or changed only at inference, and whether any constraint prevents the output projection from absorbing the sign reversal through $ WO′=−WOW'_O=-W_O $.

  2. Did reflection separation or geometry estimation use any of the 43 held-out NVS target images before the seven context views were selected?

  3. Which baselines were retrained or fine-tuned on the same data as ReflectWorld, including MVRefSep-Train, and how many input views did each multi-image method receive?

6 comments
AuthorsOpenReview ↗

Response to Reviewer JX5a (Part 1 of 4): W1 — Cross-dataset generalization

Response to Reviewer JX5a (Part 1 of 4): W1 — Cross-dataset generalization

W1:

“Section 5.4 states that all ablations are conducted on MVRefSep-Test, yet Appendix D.4 explicitly uses those results to select the default backbone, diffusion timestep, VAE LoRA configuration, and loss weights. ...”


Thank you for your constructive suggestion regarding evaluation independence and cross-dataset generalization. We previously conducted the component and configuration ablations on MVRefSep-Test because it is the only multi-view reflection separation benchmark providing ground truth for both transmission and reflection layers (GT-T and GT-R). It covers daytime, nighttime, indoor, and outdoor scenes, enabling comprehensive quantitative comparisons of both recovered layers across diverse imaging conditions. We agree that additional evaluation on independent real-world data is important for further assessing the generalization of the selected model configurations.

To further assess generalization beyond MVRefSep-Test, we have added ablation experiments on the real-world MVRefSep-Real dataset. These results are reported in Tables 2–3 (p. 8), Table 5 (p. 9), and Tables 15–29 (pp. 24–31, where applicable). The trends across simulated and real-world datasets are broadly consistent, providing further support for the selected model configurations.

Table 2: Component ablation.

SettingMVRefSep-Test T\boldsymbol{T}-PSNRT\boldsymbol{T}-SSIMR\boldsymbol{R}-PSNRR\boldsymbol{R}-SSIMMVRefSep-Real PSNRSSIM
w/o GeoRCA20.730.84421.640.75922.930.918
w/o Geometric Encoding22.280.85222.390.76524.020.917
w/o T/R\boldsymbol{T}/\boldsymbol{R} Interaction22.980.85322.180.76825.760.923
Full23.000.85822.560.77125.570.924

Table 3: Effect of reference-view count.

Refs.MVRefSep-Test T\boldsymbol{T}-PSNRT\boldsymbol{T}-SSIMR\boldsymbol{R}-PSNRR\boldsymbol{R}-SSIMMVRefSep-Real PSNRSSIM
020.560.83520.380.66222.650.907
422.610.85922.480.76325.150.922
822.700.86022.510.76525.270.923
1222.730.86022.510.76525.330.923
1622.750.86022.530.76625.360.923

Additional ablation results are provided in Table 5 (p. 9) and Tables 15–29 (pp. 24–31, where applicable).

Cross-dataset generalization evaluation on the TNeRF dataset. We have additionally evaluated ReflectWorld on the external real-world TNeRF dataset, using the same evaluation protocol for all comparison methods (Table 8, p. 20).

Table 8: Quantitative comparison on the TNeRF dataset. ReflectWorld achieves the best performance across all scenes. (p. 20)

All results are reported as PSNR (dB) / SSIM.

Methodboxmonkey1monkey2sofaAverage
DSRNet (ICCV'23)20.13/0.77624.27/0.92023.40/0.84821.18/0.82822.25/0.843
DSIT (NIPS'24)20.03/0.78126.88/0.93523.82/0.85626.51/0.85724.31/0.857
RDRNet (CVPR'24)21.17/0.77624.98/0.92626.87/0.87025.62/0.83724.66/0.852
RDNet (CVPR'25)19.64/0.77527.23/0.93324.84/0.84925.87/0.84224.40/0.850
IRLR (TIP'26)19.53/0.76225.40/0.92522.16/0.81224.38/0.82522.87/0.831
DMDNet (AAAI'26)20.05/0.77726.53/0.93126.54/0.87024.73/0.84824.46/0.857
ReflexSplit (CVPR'26)20.39/0.77926.16/0.93325.93/0.86326.04/0.84524.63/0.855
GFRRN (CVPR'26)21.22/0.79226.80/0.93726.60/0.86826.53/0.85425.29/0.863
JLD (CVPR'26)23.27/0.77128.84/0.92826.76/0.86124.03/0.82825.73/0.847
ReflectWorld (Ours)23.81/0.81629.64/0.93827.72/0.88928.45/0.87527.41/0.880

ReflectWorld achieves the highest PSNR and SSIM across all four TNeRF scenes. Together with the newly added MVRefSep-Real ablations, these results provide complementary evidence of generalization beyond the dataset used for the original configuration analyses.

AuthorsOpenReview ↗

Response to Reviewer JX5a (Part 2 of 4): W2/Q1 — Differential modulation directions

Response to Reviewer JX5a (Part 2 of 4): W2/Q1 — Differential modulation directions

W2:

“The interpretation of the reversed-direction ablation in Table 5 is inconsistent with Eqs. (2) and (9). ...”

Q1:

“Please clarify whether the reversed-direction variants in Table 5 were retrained or changed only at inference, and whether any constraint prevents the output projection from absorbing the sign reversal through (WO′=−WOW'_O = -W_O).”


Thank you for your valuable comments. The reversed-direction variant was trained with a direction constraint. Specifically, we use geometry-aligned cross-view feature differences to determine the correction direction, allowing the output projection to control the correction magnitude while preventing its parameters from absorbing the prescribed modulation direction through sign reversal. We have moved the reversed-direction experiment from Table 5 to Appendix D.4 (Table 17, p. 26) for a detailed explanation.

AuthorsOpenReview ↗

Response to Reviewer JX5a (Part 3 of 4): Q2 — Held-out NVS target images

Response to Reviewer JX5a (Part 3 of 4): Q2 — Held-out NVS target images

Q2:

“Did reflection separation or geometry estimation use any of the 43 held-out NVS target images before the seven context views were selected?”


Thank you for your careful question. None of the 43 held-out novel-view synthesis (NVS) target images were used during reflection separation or geometry estimation. Thus, the predicted transmission images and geometry used for NVS are obtained without accessing any of those held-out target images, preventing information leakage from the NVS evaluation targets. We have further clarified this evaluation protocol in Appendix C.1, Cross-View Consistency Evaluation (p. 17) of the revised manuscript.

Reviewer_sqpjOpenReview ↗
Rating
3
Confidence
2
Informativeness
3(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ReflectWorld, a framework that combines multi-view geometry and diffusion models to jointly recover transmission and reflection layers from images captured through glass. GeoRCA establishes cross-view associations, while differential modulation guides the two generation branches. The authors also introduce the MVRefSep dataset and report promising results on synthetic and real-world scenes.

Critical strengths
  1. The paper addresses a meaningful problem. Combining multi-view geometric information with diffusion models for transmission and reflection separation is well motivated and offers an interesting approach.
  2. The experiments are relatively comprehensive, including component ablations, modulation-direction comparisons, and geometric correspondence analyses. The newly introduced MVRefSep dataset is also a valuable contribution to related research.
Critical weaknesses
  1. According to the supplementary-code analysis, valid VGGT-Ω confidence values appear to be clipped to 1, potentially reducing the confidence-weighted aggregation in Eq. (6) to simple averaging and making the confidence prior in Eq. (8) ineffective. This appears inconsistent with the proposed use of geometric confidence to assess cross-view correspondence reliability and raises concerns about the effectiveness of this mechanism.
  2. The implementation includes additional decoder input channels and skip connections that are not sufficiently described in the paper. It is also unclear whether these architectural changes are controlled in the ablation experiments. Furthermore, potential issues with set_eval() and non-default chunk settings may affect inference behavior, raising concerns about consistency between the described method and its implementation.
  3. The explanation of differential modulation is not sufficiently clear. According to Eq. (2), the transmission and reflection branches use the same cross-view correction with opposite signs. However, Figure 7(b) is interpreted as showing different spatial responses in the two branches. If the visualization represents the original correction, its magnitude should be identical in both branches. Otherwise, the authors should clarify how these responses are computed.
  4. The model-selection procedure is unclear. The paper explicitly states that all ablation experiments are conducted on MVRefSep-Test, while the appendix discusses selecting the backbone, diffusion timestep, and other hyperparameters based on experimental results. Without a clear description of an independent validation set, it is difficult to assess whether the final test evaluation is independent of model selection.
Questions for authors
  1. Could the authors provide the actual distribution of VGGT-Ω confidence values during inference and compare the results using original versus constant confidence to verify whether the mechanisms in Eqs. (6) and (8) function as intended?
  2. Could the authors explain the additional decoder channels and skip connections, and clarify whether set_eval() and chunk settings affect the reported results?
  3. Does Figure 7(b) visualize the original cross-view correction or the feature responses after subsequent network processing? If it shows the original correction, why do the two branches exhibit different spatial responses?
  4. Could the authors clarify how hyperparameters and model configurations were selected? Was an independent validation set used, or were the final configurations selected based on results from MVRefSep-Test?
7 comments
AuthorsOpenReview ↗

Response to Reviewer sqpj (Part 1 of 4): W1/Q1 — Native VGGT-Ω confidence

Response to Reviewer sqpj (Part 1 of 4): W1/Q1 — Native VGGT-Ω confidence

W1:

“According to the supplementary-code analysis, valid VGGT-Ω confidence values appear to be clipped to 1, potentially reducing the confidence-weighted aggregation in Eq. (6) to simple averaging and making the confidence prior in Eq. (8) ineffective. ...”

Q1:

“Could the authors provide the actual distribution of VGGT-Ω confidence values during inference and compare the results using original versus constant confidence to verify whether the mechanisms in Eqs. (6) and (8) function as intended?”


Thank you for your careful observation. The actual inference uses the native VGGT-Ω\Omega confidence values, as shown in Table 20 (p. 28). When preparing the code for the initial submission, we inadvertently left an old confidence-clipping operation active, although it had been disabled in the code used for our experiments. We have re-uploaded the revised code. We will release the complete training and inference code in the future.

Table 20: Distribution of VGGT-Ω\Omega confidence. (p. 28)

Confidence intervalMVRefSep-TestMVRefSep-Real
[1,2)[1,2)47.91%2.68%
[2,5)[2,5)24.00%7.59%
[5,10)[5,10)16.40%14.36%
[10,20)[10,20)9.06%37.04%
[20,50)[20,50)2.59%36.21%
[50,100)[50,100)0.04%2.11%

Constant-confidence control. As shown in Table 21 (p. 28), native confidence achieves better recovery performance than constant confidence. However, constant confidence still maintains good recovery performance, indicating that the model does not fully rely on confidence guidance and exhibits robustness to changes in confidence weighting.

Table 21: Recovery with constant or native VGGT-Ω\Omega confidence.

ConfidenceMVRefSep-Test T-PSNRT-SSIMR-PSNRR-SSIMMVRefSep-Real PSNRSSIM
Constant22.960.85622.550.76925.540.923
Native23.000.85822.560.77125.570.924

How confidence affects routing. To further investigate the role of confidence, we use Total Variation Distance (TV) to quantify the differences between the probability distributions induced by native and constant confidence in Eqs. (6) and (8), as shown in Table 22 (p. 28).

Table 22: Total Variation Distance (TV) between the distributions induced by native and constant confidence. (p. 28)

DatasetSplatting (Eq. 6)Attention (Eq. 8)
MVRefSep-Test0.0360.084
MVRefSep-Real0.0730.113

The differentiated weights provided by native confidence modulate the contributions of cross-view features during splatting in Eq. (6) and guide cross-view attention allocation through the confidence bias in Eq. (8).

AuthorsOpenReview ↗

Response to Reviewer sqpj (Part 2 of 4): W2/Q2 — Decoder architecture and controls

Response to Reviewer sqpj (Part 2 of 4): W2/Q2 — Decoder architecture and controls

W2:

“The implementation includes additional decoder input channels and skip connections that are not sufficiently described in the paper. ...”

Q2:

“Could the authors explain the additional decoder channels and skip connections, and clarify whether set_eval() and chunk settings affect the reported results?”


Thank you for your valuable questions about the implementation details. We have clarified the skip-connection architecture and evaluation settings in Appendix A.1 (p. 15) and added architectural controls in Table 16 (p. 25) of the revised manuscript. ReflectWorld employs two distinct types of skip connections for VAE encoder-decoder feature propagation and cross-view information fusion, respectively.

(1) VAE skip connections. To improve the training stability of VAE low-rank adaptation, we introduce four learnable skip connections between the VAE Encoder and Decoder. Specifically, four 1×11\times1 convolutions project multi-scale Encoder features to the corresponding Decoder channels, followed by additive fusion at each upsampling stage.

(2) Cross-view decoder connections. In the diffusion UNet, we introduce cross-view features from GeoRCA as additional skip connections to the corresponding Decoder ResBlocks. Specifically, these features are concatenated with the original Decoder inputs. To accommodate the additional features, we expand the input channels of the corresponding Decoder ResBlocks and adjust the associated normalization and convolution layers, enabling them to process both the original and cross-view features.

(3) Architectural controls for ablation studies. All four VAE skip connections are retained in the full model and in the main component ablations. The cross-view decoder path remains in place when we ablate Geometric Encoding or T/R\boldsymbol{T}/\boldsymbol{R} Interaction. Ablating GeoRCA also removes its associated cross-view feature input path. To control for the additional decoder channels, we compare with a capacity-matched Target-Only variant. This control retains the expanded decoder ResBlocks of the full model and replaces reference-view features with features constructed from the target view alone. Table 16 (p. 25) shows that the full model outperforms the Target-Only control on both datasets.

Table 16: Architectural controls for the GeoRCA ablation.

SettingMVRefSep-Test T-PSNRT-SSIMR-PSNRR-SSIMMVRefSep-Real PSNRSSIM
w/o GeoRCA20.730.84421.640.75922.930.918
Capacity-matched Target-Only21.650.84422.060.76323.470.904
Full ReflectWorld23.000.85822.560.77125.570.924

(4) set_eval() versus model.eval(). The evaluation mode settings do not affect the reported results. The set_eval() function explicitly switches the main modules, including the UNet, VAE, and VGGT geometry projection module, to evaluation mode, whereas model.eval() recursively switches the entire model and its submodules to evaluation mode. In our actual evaluation, we use set_eval() together with an outer model.eval(). Experimental comparisons show that the PSNR differences are below 0.0001 dB, with similarly negligible SSIM differences.

(5) Chunk setting. The chunk setting does not affect the reported results. We consistently set target_chunk_size=1 during both training and inference, processing target views sequentially in the Decoder Forward pass. This setting primarily reduces peak GPU memory consumption without changing the reference-view information available to each target view.

AuthorsOpenReview ↗

Response to Reviewer sqpj (Part 3 of 4): W3/Q3 — Interpretation of Fig. 7(b)

Response to Reviewer sqpj (Part 3 of 4): W3/Q3 — Interpretation of Fig. 7(b)

W3:

“The explanation of differential modulation is not sufficiently clear. ...”

Q3:

“Does Figure 7(b) visualize the original cross-view correction or the feature responses after subsequent network processing? ...”


Thank you for your insightful question. Figure 7(b) illustrates the spatial responses of the transmission and reflection layers to the cross-view correction Δcv\Delta_{\mathrm{cv}} after propagation through subsequent network layers.

Since Δcv\Delta_{\mathrm{cv}} lies in a high-dimensional feature space and lacks intuitive visual interpretation, Fig. 7(b) adopts output-space response visualization. Specifically, we perform inference with cross-view correction enabled and disabled, respectively, and compute the pixelwise differences between the TT and RR predictions under the two settings to illustrate the spatial impact of the correction on the recovery results.

We have clarified the visualization procedure and its interpretation in Appendix A.1 (p. 15) of the revised manuscript.

Reviewer_GLtjOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies reflection separation from glass using multi-view inputs and proposes ReflectWorld, a framework that combines a diffusion-based decomposition model with geometry-guided feature routing. The method uses VGGT-derived geometric information, injects geometric features into the diffusion model through MVGT, and uses GeoRCA to project and select cross-view features before applying a cross-view correction to the transmission and reflection branches. The paper also introduces MVRefSep, a benchmark containing synthetic Blender scenes and real scenes. Training combines the synthetic data with DL3DV-1K and several single-image datasets.

Critical strengths
  1. It is encouraging to see a classical assumption in reflection separation revisited through newer techniques, with the reported results suggesting improved performance. This offers a fresh perspective for the field. In particular, MVGT and GeoRCA address complementary aspects of the problem: incorporating geometric features into the model and routing information across views.

  2. MVRefSep includes synthetic scenes with layer supervision, as well as real scenes. A benchmark of this kind could help support further work on multi-view reflection separation.

  3. The paper compares with both single-image and multi-view methods and includes ablations of components and data sources.

Critical weaknesses
  1. The reported evaluation is not fully independent because the test set is used for extensive model and hyperparameter selection. Sec. 5.4 states that all ablation experiments are conducted on MVRefSep-Test. Appendix D.4 further uses these experiments to determine the default configuration, including the number of reference views, the feed-forward 3D backbone, the VAE LoRA configuration, the training-data composition, and the one-step diffusion timestep. I do not consider a small number of test-set ablations to justify the selected model configuration to be a major issue. However, the paper reports a relatively large number of test-set-based ablations, including choices involving the one-step diffusion timestep and the training-data composition. These settings can be highly dependent on the particular test distribution. Optimizing such choices on MVRefSep-Test substantially weakens confidence in the proposed method’s ability to generalize to real-world data.

  2. GeoRCA projects candidate features between views using camera parameters and geometric information, but the paper does not make sufficiently clear whether these parameters are ground-truth or VGGT-estimated in each evaluation setting. This distinction is important for practical use. More fundamentally, VGGT is not specifically trained to distinguish transmitted scene structure from reflection-induced structure. Strong reflections, occlusions, and moving objects can corrupt its depth, point maps, and correspondences—the very situations in which reflection separation is needed. The paper does not provide enough robustness analysis to show that GeoRCA’s projection and filtering remain reliable under these errors.

  3. The method adds a correction to the transmission features and subtracts it from the reflection features. Since this operates on internal feature representations rather than on physically meaningful image layers, the signs alone do not guarantee that transmission becomes more cross-view consistent or that reflection becomes more view-dependent. The same operation could suppress legitimate view-dependent transmission content, such as a moving object, or mis-handle a reflection that is stable across views. The paper would be stronger with direct measurements of the claimed effects and analysis of failure cases.

  4. MVRefSep has 40 synthetic Blender scenes, with 8 used for training, and 10 real scenes in the evaluation set. Real scenes do not have reflection-layer ground truth, so the real-world evidence for recovering both layers is largely qualitative. In addition, the DL3DV-based training data is synthesized by combining transmission and reflection videos from different scenes with matched camera-motion magnitudes. This is a useful way to construct additional data, but it does not reproduce the physical image formation or scene coupling of a real glass reflection. The paper would benefit from clearer evidence about how much this data helps and how well the resulting model transfers to real scenes.

  5. The baseline protocols are not described in enough detail to assess comparability. In particular, the paper should clarify for each baseline whether it uses official pretrained weights, is fine-tuned on MVRefSep or other training data, and receives one view or multiple views at inference. Since the comparison includes both single-image and multi-view methods, these settings should be clearly separated and reported.

Questions for authors
  1. Camera and geometry inputs: For each evaluation setting, are the camera intrinsics and extrinsics used by GeoRCA ground truth, estimated by VGGT, or obtained from another source? What geometric inputs are available at real-world inference time? Could you report results using estimated parameters and quantify sensitivity to pose and depth errors?
  2. Robustness of GeoRCA: How does GeoRCA perform when VGGT’s point maps or correspondences are corrupted by strong reflections, occlusions, or dynamic transmitted objects? Could you provide targeted experiments or failure cases under these conditions?
  3. Effect of the cross-view correction: What evidence shows that adding and subtracting the correction produces the intended changes in the transmission and reflection representations? Could you measure cross-view consistency and view-dependent variation directly, and show examples where the correction helps or harms dynamic transmission content?
  4. Role of single-image data: Since the 2D diffusion backbone is shared between single-image and multi-view inputs, can the authors evaluate whether multi-view/3D supervision improves its single-image layer decomposition capability? In particular, please show the predicted transmission and reflection layers on representative images from the Real dataset introduced by Zhang et al., preferably comparing models trained with and without the multi-view/3D sources such as DL3DV-1K and MVRefSep-Train. Table 13 also requires a more detailed explanation of how the training-data composition affects performance on MVRefSep-Test. The effect of removing Real and Nature is relatively intuitive, but it is unclear why removing VOC2012 leads to a noticeable degradation, why increasing the proportion of MVRefSep-Train from 10% to 20% results in worse performance than the default mixture, and why the different mixture settings affect transmission and reflection recovery differently. Are these effects caused by differences in scene diversity, domain distribution, data quality, or optimization and sampling imbalance? Controlled comparisons with matched training budgets and sample counts would help disentangle the source-specific benefits from the effects of changing the data distribution.
  5. Role of DL3DV-1K: How much of the improvement comes specifically from DL3DV-1K-based synthetic training data? Could the authors compare against an equivalent amount of video data constructed using a different source or sampling strategy, to clarify whether the benefit comes from DL3DV-1K itself or from additional camera-motion-consistent training examples?
  6. Baseline protocols: For every method in Table 1, could the authors report the training data, use of pretrained weights or fine-tuning, number of input views, and inference protocol? Are single-image and multi-view results directly comparable under the reported settings? Flag For Ethics Review
9 comments
AuthorsOpenReview ↗

Response to Reviewer GLtj (Part 1 of 7): W1 — Evaluation protocol and generalization

Response to Reviewer GLtj (Part 1 of 7): W1 — Evaluation protocol and generalization

W1:

“The reported evaluation is not fully independent because the test set is used for extensive model and hyperparameter selection. ...”


Thank you for your constructive suggestion. We previously conducted ablation studies on MVRefSep-Test because it is the only multi-view reflection separation benchmark with ground truth for both transmission and reflection layers (GT-T and GT-R). It covers daytime, nighttime, indoor, and outdoor scenes, allowing comprehensive quantitative comparisons of both recovered layers across diverse imaging conditions. We agree that additional evaluation on real-world data is important for further assessing generalization.

To further evaluate generalization beyond MVRefSep-Test, we have added ablation results on the real-world MVRefSep-Real dataset. These results are reported in Tables 2–3 (p. 8), with further ablations in Table 5 (p. 9) and the appendix (Tables 15–29, pp. 24–31, where applicable). The trends across simulated and real-world datasets are broadly consistent, supporting the reported configuration as an effective choice for recovering both transmission and reflection. Representative results from Tables 2 and 3 are presented below.

Table 2: Component ablation.

SettingMVRefSep-Test T-PSNRT-SSIMR-PSNRR-SSIMMVRefSep-Real PSNRSSIM
w/o GeoRCA20.730.84421.640.75922.930.918
w/o Geometric Encoding22.280.85222.390.76524.020.917
w/o T/R Interaction22.980.85322.180.76825.760.923
Full23.000.85822.560.77125.570.924

Table 3: Effect of reference-view count.

Refs.MVRefSep-Test T-PSNRT-SSIMR-PSNRR-SSIMMVRefSep-Real PSNRSSIM
020.560.83520.380.66222.650.907
422.610.85922.480.76325.150.922
822.700.86022.510.76525.270.923
1222.730.86022.510.76525.330.923
1622.750.86022.530.76625.360.923

... Further ablation results are provided in Table 5 (p. 9) and Tables 15–29 (pp. 24–31), where applicable.

We additionally evaluate on the independent TNeRF dataset, comparing ReflectWorld with the comparison methods under the same evaluation protocol (Table 8, p. 20):

Table 8: Quantitative comparison on TNeRF dataset. ReflectWorld achieves the best across all scenes. (p. 20) All results are reported as PSNR (dB) / SSIM.

Methodboxmonkey1monkey2sofaAverage
DSRNet (ICCV'23)20.13/0.77624.27/0.92023.40/0.84821.18/0.82822.25/0.843
DSIT (NIPS'24)20.03/0.78126.88/0.93523.82/0.85626.51/0.85724.31/0.857
RDRNet (CVPR'24)21.17/0.77624.98/0.92626.87/0.87025.62/0.83724.66/0.852
RDNet (CVPR'25)19.64/0.77527.23/0.93324.84/0.84925.87/0.84224.40/0.850
IRLR (TIP'26)19.53/0.76225.40/0.92522.16/0.81224.38/0.82522.87/0.831
DMDNet (AAAI'26)20.05/0.77726.53/0.93126.54/0.87024.73/0.84824.46/0.857
ReflexSplit (CVPR'26)20.39/0.77926.16/0.93325.93/0.86326.04/0.84524.63/0.855
GFRRN (CVPR'26)21.22/0.79226.80/0.93726.60/0.86826.53/0.85425.29/0.863
JLD (CVPR'26)23.27/0.77128.84/0.92826.76/0.86124.03/0.82825.73/0.847
ReflectWorld (Ours)23.81/0.81629.64/0.93827.72/0.88928.45/0.87527.41/0.880

ReflectWorld obtains the highest PSNR and SSIM in all four scenes. Together with the added MVRefSep-Real ablations, these results provide additional evidence of cross-dataset generalization.

AuthorsOpenReview ↗

Response to Reviewer GLtj (Part 2 of 7): W2/Q1 — Estimated geometry and sensitivity

Response to Reviewer GLtj (Part 2 of 7): W2/Q1 — Estimated geometry and sensitivity

W2:

“GeoRCA projects candidate features between views using camera parameters and geometric information, but the paper does not make sufficiently clear whether these parameters are ground-truth or VGGT-estimated in each evaluation setting. ...”

Q1:

“Camera and geometry inputs: For each evaluation setting, are the camera intrinsics and extrinsics used by GeoRCA ground truth, estimated by VGGT, or obtained from another source? ...”


Thank you for your careful review and valuable suggestions. The camera intrinsics and extrinsics used by GeoRCA are estimated by VGGT-Ω\Omega. At real-world inference time, VGGT-Ω\Omega estimates camera intrinsics, extrinsics, and depth from the mixed input views, which are used to establish cross-view geometric relationships. All reported reflection-separation results use estimated geometry. We have clarified this in Appendix A.1, Additional Implementation Details (p. 15) of the revised manuscript.

Sensitivity to geometry errors. To quantify the impact of geometry estimation errors, we introduce varying magnitudes of rotation, relative translation, and depth perturbations into the estimated geometry, and reconstruct the 3D point maps accordingly. As shown in Fig. 8 (p. 10), recovery performance generally decreases as geometric errors increase, but the rate of degradation gradually slows at higher perturbation levels. T\boldsymbol{T} is more sensitive to these perturbations, whereas R\boldsymbol{R} exhibits smaller variations due to its weaker dependence on cross-view geometric alignment. This trend demonstrates that ReflectWorld maintains relatively stable recovery performance even under large perturbations, indicating its robustness to geometry estimation errors.

For example, on MVRefSep-Test, a rotation perturbation of 20° changes TT-PSNR by approximately −0.281 dB, while the corresponding RR-PSNR change is approximately −0.074 dB. We also report the response to translation and depth perturbations and include real-world transmission results from MVRefSep-Real in Fig. 8.

Sensitivity of ReflectWorld to geometry estimation errors (PSNR, dB; Fig. 8, p. 10).

Error typeMagnitudeMVRefSep-Test T-PSNRR-PSNRΔT-PSNRΔR-PSNRMVRefSep-Real PSNRΔPSNR
None—23.00722.5620.0000.00025.5640.000
Rotation1°22.92122.549-0.086-0.01325.539-0.025
Rotation2°22.86222.533-0.145-0.02925.517-0.047
Rotation5°22.78222.499-0.226-0.06325.476-0.088
Rotation8°22.75722.493-0.250-0.06825.435-0.130
Rotation10°22.74522.492-0.262-0.07025.416-0.149
Rotation12°22.73922.491-0.268-0.07125.401-0.164
Rotation20°22.72622.488-0.281-0.07425.357-0.207
Translation1%22.94922.557-0.059-0.00525.558-0.007
Translation5%22.82022.520-0.187-0.04225.508-0.056
Translation10%22.77422.494-0.233-0.06825.458-0.106
Translation15%22.75622.480-0.251-0.08225.423-0.142
Translation20%22.74222.479-0.265-0.08225.406-0.158
Translation30%22.72422.484-0.283-0.07725.377-0.188
Depth2%22.98622.557-0.022-0.00425.563-0.001
Depth5%22.94822.548-0.059-0.01325.560-0.004
Depth10%22.90322.538-0.104-0.02425.552-0.012
Depth15%22.87122.530-0.136-0.03225.544-0.020
Depth20%22.84922.524-0.159-0.03725.539-0.025
Depth30%22.82222.515-0.186-0.04625.529-0.035

These experiments support relative robustness to imperfect geometry, while showing that geometry errors can still reduce restoration quality.

GeoRCA does not assume that geometrically projected features are necessarily clean: geometry restricts the candidate search space, and the subsequent association step selects more reliable cross-view evidence. Figures 6 (p. 8) and 13 (p. 23) further illustrate routing and candidate selectivity under reflection interference.

AuthorsOpenReview ↗

Response to Reviewer GLtj (Part 3 of 7): W3/Q2/Q3 — Dynamic transmission and cross-view correction

Response to Reviewer GLtj (Part 3 of 7): W3/Q2/Q3 — Dynamic transmission and cross-view correction

W3:

“The method adds a correction to the transmission features and subtracts it from the reflection features. ...”

Q2:

“Robustness of GeoRCA: How does GeoRCA perform when VGGT’s point maps or correspondences are corrupted by strong reflections, occlusions, or dynamic transmitted objects? ...”

Q3:

“Effect of the cross-view correction: What evidence shows that adding and subtracting the correction produces the intended changes in the transmission and reflection representations? ...”


Thank you for requesting more direct evidence. We have added controlled evaluations of dynamic transmission content, cross-view consistency, and failure cases.

(1) Dynamic transmission objects. To evaluate the impact of dynamic transmission content on reflection separation, we construct a simulated scene with five vehicle motion levels, S0–S4, as detailed in Table 11 (p. 22) and illustrated in Fig. 14 (p. 23). We use five input views, with each view serving as the target in turn and the remaining four as references. To avoid fixed associations between vehicle positions and camera views, we cyclically assign the five vehicle positions across views at each level from S1 to S4 and average the results over the five assignments. We compare the full model (Full) with cross-view correction disabled (Off).

Table 11: Vehicle motion levels in the dynamic transmission experiment. (p. 22)

LevelSettingVehicle position step (m)Mean target–reference position difference (m)Additional GT displacement (px)
S0Synchronous000
S1Asynchronous0.090.1826.51
S2Asynchronous0.170.3450.07
S3Asynchronous0.270.5479.53
S4Asynchronous0.480.96141.40

Layer recovery versus motion magnitude. As shown in Tables 12–13 (p. 22), cross-view correction improves both PSNR and SSIM for the transmission and reflection layers under the synchronous setting S0. As vehicle motion increases, the correction gains for transmission gradually diminish, indicating that dynamic geometric misalignment weakens transmission recovery and may even have negative effects. In contrast, the PSNR and SSIM gains for reflection remain positive at all motion levels. This is because vehicle motion directly disrupts cross-view geometric correspondences in the transmission layer, while the reflected scene itself remains unchanged across motion settings and is therefore less affected.

Table 12: Comparison of PSNR (dB) across vehicle motion levels. Transmission gains diminish with increasing motion, while reflection gains remain positive. (p. 22)

LevelTransmission OffFullΔImprovedReflection OffFullΔImproved
S016.08216.178+0.097✓26.43226.646+0.214✓
S116.26216.358+0.096✓26.74826.993+0.245✓
S216.44716.541+0.093✓26.98127.244+0.263✓
S316.53116.604+0.073✓27.11027.358+0.248✓
S416.61516.659+0.045✓27.07327.323+0.249✓

Table 13: Comparison of SSIM across vehicle motion levels. Transmission gains diminish and become negative at the largest motion magnitude, while reflection gains remain positive. (p. 22)

LevelTransmission OffFullΔ (×10⁻³)ImprovedReflection OffFullΔ (×10⁻³)Improved
S00.84680.8478+1.063✓0.90590.9072+1.375✓
S10.85050.8512+0.685✓0.90830.9104+2.062✓
S20.85380.8545+0.647✓0.91080.9132+2.420✓
S30.85510.8554+0.233✓0.91210.9144+2.356✓
S40.85650.8562-0.297✗0.91130.9138+2.519✓

Continued in Part 4 of 7: Cross-view measurements and failure cases.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

We sincerely thank the three reviewers for their careful reading and constructive suggestions. We have revised the manuscript and added experiments to address the central concerns about (i) generalization and evaluation protocols, (ii) the reliability of geometry-routed cross-view association and differential modulation, and (iii) implementation details and fair baseline comparisons. All additions and revisions are highlighted in blue in the revised PDF for ease of reference.

Revision map — locations refer to the revised manuscript.

ReviewersMain additions / clarificationsLocation
GLtj (W1), sqpj (W4/Q4), JX5a (W1)Additional ablations on MVRefSep-Real and evaluation on the TNeRF datasetTables 2–3 (p. 8), Table 5 (p. 9); Table 8 (p. 20); Tables 15–29 (pp. 24–31)
GLtj (W2/Q1)Estimated geometry, its source, and sensitivity to rotation, translation, and depth errorsAppendix A.1 (p. 15); Fig. 8 (p. 10)
GLtj (W3/Q2/Q3)Dynamic transmission, cross-view consistency, and failure under strong reflectionsFig. 9 (p. 10); Fig. 14 (p. 23), Tables 11–14 (pp. 22–24)
GLtj (W4/Q4/Q5)Single-image transfer and contributions of training sources / DL3DV-1KFig. 18, Table 26 (p. 30); Table 25 (p. 30)
GLtj (W5/Q6), JX5a (Q3)Baseline weights, input protocols, and fine-tuning comparisonsSection 5.3 (p. 8); Appendix A.1 (pp. 15–16); Appendix C.4, Tables 9–10 (pp. 20–21)
sqpj (W1/Q1)VGGT-Ω\Omega confidence, constant-confidence control, and TV analysisTables 20–22 (p. 28)
sqpj (W2/Q2)Decoder architecture and capacity-matched GeoRCA ablationTable 16 (p. 25); architecture clarification in revised text
sqpj (W3/Q3)Meaning of the Δcv\Delta_{\mathrm{cv}} visualizationExplanation in Appendix A.1 (p. 15); Fig. 7(b) (p. 9)
JX5a (W2/Q1)Direction-constrained reversed modulationTable 5 (p. 9); Appendix D.4, Table 17 (p. 26)
JX5a (Q2)Held-out NVS target images and evaluation protocolAppendix C.1 (p. 17)

Main additional evidence. (i) Ablations on MVRefSep-Real: We have extended the ablation experiments to MVRefSep-Real, where the results show trends broadly consistent with those on MVRefSep-Test (Tables 2–3, 5, and additional appendix tables), further supporting the generalization of our configuration choices. (ii) Evaluation on the TNeRF dataset: ReflectWorld achieves the highest PSNR and SSIM across all four scenes (Table 8). (iii) Fine-tuned baseline comparisons: We fine-tuned the baseline methods using the same training data and number of training steps as ours, and ReflectWorld still achieves the best overall performance (Tables 9–10). (iv) Robustness to geometry errors: Sensitivity tests show gradual rather than catastrophic degradation under increasingly inaccurate geometry (Fig. 8). (v) Limitations and failure cases: Large inter-view motion reduces the gains from cross-view correction, while strong reflections with similar cross-view variations can cause separation failure (Fig. 9; Tables 11–14).

Detailed responses to each reviewer are provided separately in the following comments, with references to the revised manuscript. We appreciate the opportunity to clarify and strengthen the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.