acceptodds
Withdrawn from ICLR 2027

SW-Mamba: Selective Weather Forgetting for Self-Supervised Monocular Depth Estimation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Self-supervised monocular depth estimation relies on cross-frame photometric consistency, an assumption severely challenged by adverse weather where appearance changes deviate from scene geometry. To address these challenges, we propose Selective Weather-forgetting Mamba (SW-Mamba), which formulates robust adverse-weather depth estimation as selective latent-state correction. Rather than treating inter-frame variation as a weather mask, SW-Mamba interprets temporal cues jointly with multidirectionally propagated spatial context to estimate a spatiotemporal correction gate. The gate modulates reference-guided state correction, selectively updating unreliable responses while preserving geometric consistency. During paired training, aligned clean states provide supervision for gate calibration and serve as structural references. At inference, an internal structural proxy enables deployment without aligned clean observations. Extensive experiments on WeatherKITTI, nuScenes, and DrivingStereo show consistent gains under adverse weather and strong cross-domain generalization.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Monocular depth estimation, self-supervised learning, adverse-weather vision, state space model, selective forgetting

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_oLzpOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses self-supervised monocular depth estimation under adverse weather conditions, aiming to improve depth prediction robustness against weather-induced appearance variations. The authors propose SW-Mamba, a framework consisting of three main components: (1) State Propagation, which employs multidirectional Mamba to aggregate spatial context; (2) State Gating, which combines inter-frame RGB differences with spatial features to estimate correction weights; and (3) State Correction, which selectively updates latent features using clean references during training and spatially averaged feature proxies at inference. Experiments on WeatherKITTI, nuScenes, and DrivingStereo demonstrate improvements in depth estimation accuracy under adverse weather conditions, including zero-shot cross-dataset generalization, with relatively low computational overhead.

Critical strengths
  • The paper addresses self-supervised monocular depth estimation under adverse weather conditions, which is a challenging and practically important problem for robust visual perception and autonomous driving.
  • The proposed method achieves competitive or superior results compared with existing approaches across WeatherKITTI, nuScenes, and DrivingStereo, demonstrating improvements under various adverse weather conditions and promising cross-dataset generalization.
Critical weaknesses
  • The paper identifies weather-induced violations of inter-frame photometric consistency as a major challenge in self-supervised depth estimation. However, the proposed method primarily corrects latent features without directly addressing unreliable photometric supervision. The connection between the identified problem and the proposed solution is not sufficiently established.
  • The method relies on raw inter-frame RGB differences to guide weather-related feature correction. However, such differences can arise from camera motion, moving objects, and occlusions rather than weather corruption. For example, a moving vehicle or an occluded object may produce large frame differences, even though the corresponding features do not necessarily require weather-related correction. Conversely, severe but temporally stable degradations, such as dense fog, may produce relatively small frame differences. Therefore, the motivation for using raw frame differences as weather-related guidance is insufficiently justified.
  • The method assumes that weather corruption is spatially irregular and temporally unstable. While this may hold for isolated raindrops or snowflakes, many adverse weather conditions, such as heavy rain and dense fog, affect large portions of an image. The paper does not adequately explain how the proposed mechanism handles these different degradation patterns.
  • During inference, the method constructs reference features using local and global average pooling of weather-corrupted features. However, there is no convincing justification that spatial averaging can recover clean structural representations. For example, averaging cannot reliably remove widespread degradation caused by fog. Moreover, local averaging may suppress important geometric details, such as object boundaries and thin structures. In regions with large correction weights, the method may therefore replace useful geometric information with oversmoothed features.
  • The paper lacks direct evidence that inter-frame differences can effectively identify weather-related corruption or that the predicted correction gates distinguish weather disturbances from genuine geometric changes. For example, visualizations of temporal cues and correction gates, as well as comparisons with geometry-aligned frame differences, would help validate the central hypothesis. Similarly, the authors do not directly demonstrate that the constructed reference features are more geometrically reliable than the original features. The existing ablations mainly demonstrate improvements in final depth accuracy rather than validating the claimed working mechanism.

Overall, although the proposed method achieves promising quantitative results across several benchmarks, I have substantial concerns about its methodological soundness and underlying motivation. Several key assumptions, particularly the use of inter-frame differences to identify weather corruption and spatially averaged features as reliable correction references, are neither sufficiently justified nor directly validated. Consequently, the reported performance improvements alone are not convincing evidence that the method operates according to the proposed selective weather-forgetting mechanism. The improvements may instead originate from additional feature modeling or regularization effects, rather than the claimed identification and correction of weather-corrupted representations.

Questions for authors

Please see the Weaknesses part.

Reviewer_VJobOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SW-Mamba for adverse-weather MDE via selective latent-state correction, validated on three benchmarks. However, data credibility and mechanism attribution are the two core issues — baseline gaps, contradictory ablations, and gains that persist even when core modules are disabled.

Critical strengths

This paper formulates adverse-weather MDE as a selective latent-state correction problem, distinguishing transient weather corruption from geometrically meaningful representations. The proposed SW-Mamba integrates multidirectional state propagation, spatiotemporal gating, and reference-guided correction to determine spatial context, correction strength, and correction direction, while an internal proxy enables inference without clean observations. Extensive experiments on WeatherKITTI, nuScenes, and DrivingStereo demonstrate consistent improvements across adverse-weather depth estimation, real-world conditions, and zero-shot transfer, while retaining favorable accuracy–efficiency trade-offs.

Critical weaknesses

This paper proposes a pluggable feature-refinement module, SW-Mamba, for self-supervised monocular depth estimation (MDE) under adverse weather conditions. This module is inserted into stages {2,3,4} of the WeatherDepth encoder. This paper reports results for three benchmarks, WeatherKITTI, nuScenes, and Zero-shot DrivingStereo. It also reports results for component ablation, reference ablation, and efficiency ablation.

  1. SW-Mamba is built on the WeatherDepth baseline. Table 1 reports a WeatherDepth with an absolute reliability of 0.103 on WeatherKITTI. However, the Base Model in Table 4 and the “Weather Depth Baseline” in Table 6 on WeatherKITTI report 0.128. This represents a gain of approximately 12% compared to published WeatherDepth values . Compared to the baseline set by the paper itself, the gain is approximately 29%. The paper does not explain why its reproduced baseline is 25% worse than the published baseline. If the reproduction is so poor, the comparisons in Table 1 are a mixture of published data from well-tuned systems and data from reimplementations based on poorly performing methods. Alternatively, the ablation baseline used a different configuration.

Table 11 reports an absolute relativity of 0.116 and a root mean square error of 4.882 for the complete model (with all three design choices), instead of 0.091/4.078. Its settings are never defined. Table 9 gives a clear-day baseline of 0.102, which is almost identical to the severe weather WeatherDepth in Table 1. In general, the figures in Tables 1, 4, 6, 9, and 11 cannot be reconciled with the manuscript.

  1. The definition of the partial ablation row is ambiguous according to the equations of the method itself. According to formula (2)/(13), gate α has no effect without reference B_r. Therefore, the "+ time-gated" row in Table 4 does not correspond to any of the described calculations. - The missing "+ Reference Correction" line (0.097) for deploying the agent is also unclear. During inference, there is no clear reference, so other references must be used.

Table 5 shows a result of 0.105 when using the current state B_w as the inference reference. In this case, H = B_w is fully valid, and the gate becomes irrelevant again. This contradicts the 0.097 in Table 4.

The "+ State Propagation" (0.117) in Table 4 and the "+ Mamba Adapter" (0.103) in Table 6 seem to describe similar modules, but they are actually quite different. The specific names of the CNN, Transformer, and Mamba Adapter are not explicitly stated.

  1. During the inference phase, the mechanism simplifies to a learned gated low-pass filter on the features, so the claim of "weather forgetting" is somewhat exaggerated. Substituting equation (11) into equation (13), we get H = (1−α)B_w + α[0.75·Avg5×5(B_w) + 0.25·GAP(B_w)]. Therefore, the deployed operation is a spatially adaptive mixture between the feature and its local/global average. No pure state information exists at test time. The term "selective forgetting" indicates a correction to the cyclic forgetting dynamics of SSM. In fact, the correction is performed as a post-hoc interpolation outside the scan (Equations 6, 13). This paper does not provide any evidence that α can truly locate weather noise in real data (e.g., α plots on nuScenes or DrivingStereo), nor does it demonstrate a correlation with the |B_w − B_c| difference on the retained data pairs. Figures 1 and 3 are schematic diagrams.

  2. The results obtained are not attributable to the claimed mechanism; confounding factors have not been examined. For clean pairings, L_proxy backpropagates to B_w via the unseparated B_p (page 6, lines 310–312). This is equivalent to the low-frequency clean feature distillation loss on the encoder. In 75% of the training iterations, the features received by the decoder are mixed with the clean state B_c, which is privileged information. Both effects can explain the gains of WeatherKITTI, regardless of the gating mechanism. We need a control group: maintaining the same loss and distillation results without gating or with a constant α value. In nuScenes, gated calibration and surrogate alignment are disabled (page 8, Appendix E.2). At this point, the module degenerates into a Mamba adapter plus unsupervised gated blurring, but it still reports the largest relative gain (5.6%–6.2%, Table 2). This undermines the claim that clean state calibration is crucial. On nuScenes, there is no ablation effect between Mamba propagation and surrogate blur correction.

  3. The timing cues are coarse and change input settings. The calculation of ΔI = |I_t − I_j| does not consider its own motion compensation (Equation 7). For moving vehicles, it is mainly affected by parallax and edges. It is then normalized channel by channel, thus discarding the global magnitude (Equation 8). Therefore, the model requires adjacent frames during inference (Appendix D.2: “Previous Frame”). Thus, the comparisons with single-frame methods in Tables 1-3 are not entirely identical. This paper does not evaluate the effect of individual cues (e.g., Q=0 versus learned cues), does not evaluate the behavior of cues at the beginning of a sequence, and does not evaluate the performance of cues on static cameras.

  4. Lack of statistical rigor and sources of comparison. Variance, seeds, or confidence intervals are not given. The improvement is small compared to the strongest baseline: an absolute relative value of 0.003 compared to WeatherMono (Table 1); and an absolute relative value of 0.004 compared to D4RD on a sunny-day scene in nuScenes.

It remains unclear which baselines were re-evaluated and which were replicated (Appendix E.4, page 20). This is particularly important for WeatherDepth and D4RD on nuScenes and all baselines on DrivingStereo, as these baselines are highly sensitive to scale alignment rules. Depth Anything V2 and UniDepth appear in Table 1, but the alignment of relative or metric depths is not specified. Figure 8 compares WeatherKITTI with WeatherMono, but otherwise with D4RD. It claims that "gain increases under more challenging conditions" (page 9), but this claim is not supported by a systematic, rigorous analysis.

  1. Insufficient Specificity in Composite Condition Experiments. Table 10 includes three conditions on the WeatherKITTI dataset: "Night + Rain," "Snow + Blur," and "Fog + Low Light." The WeatherKITTI dataset described in Appendix E.1 only includes four conditions: rain, snow, rain-snow mixture, and fog. The paper does not explain how conditions such as nighttime, snow, or low light are generated, which segmentation method is used, or the specific meaning of "benchmark."
Questions for authors
  1. Please explain the difference between the WeatherDepth value of 0.103 in Table 1 and the “WeatherDepth baseline” of 0.128 in Tables 4 and 6. Which result did you reproduce? Did training differences cause this gap? Which baseline values ​​in Tables 1-3 were reruns and which were copies?

  2. What are the settings in Table 11? Why does the full model get 0.116/4.882 instead of 0.091/4.078? What exactly does “no multi-scale refinement” mean?

  3. In Table 4, assuming α only works through B_r, what is the result of “+time gating” without reference correction? For “+reference correction” without deploying agents, which reference is used during testing? Why does it give a result of 0.097, while the result given by B_w reference in Table 5 is 0.105? 4. Please define the CNN, Transformer, and Mamba adapters in Table 6. What is the relationship between "+Mamba adapter" (0.103) and "+state propagation" (0.117) in Table 4?

  4. Can you provide some control methods to isolate the gating process from the clean feature distillation process?

  • (a) Train using the same L_proxy and clean reference, but with a fixed α value or a globally learned value.

  • (b) Train using a clean reference, but force the α value to be 0 during evaluation.

  • (c) Remove ΔI (Q=0).

  1. Can you show the α mapping on real nuScenes or DrivingStereo frames? Can you quantify whether α is related to |B_w − B_c| on preserved WeatherKITTI frame pairs, for example, using the AUROC metric?

  2. Since the temporal cues require the previous frame during inference, how do you handle the first frame of the sequence? How does performance vary with ego-motion compensation differences or greater camera motion?

  3. Please report the mean ± standard deviation of the main WeatherKITTI and nuScenes results across multiple seed points, considering the bias relative to the strongest baseline of 0.003–0.004 absolute relative values.

  4. How were the composite conditions (night, blurred, low light) in Table 10 generated? Based on which segmentation? What does "Base" refer to?

  5. Why is the blurred, corrupted state B_p expected to be closer to the sharp target than B_w (a premise of Proposition 4)? Can you provide empirical results for ||B_p ​​− B_c|| and ||B_w − B_c||?

  6. What alignment methods (median scaling, least-squares scaling/translation) were used for Depth Anything V2 and UniDepth in Table 1? What alignment method is used for all the methods on DrivingStereo?

Reviewer_2Qp5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SW-Mamba for adverse-weather self-supervised monocular depth estimation. Multidirectional Mamba propagation supplies spatial context; inter-frame RGB differences modulate a gate that blends latent states toward a reference. Aligned clean views supervise gate calibration and proxy alignment during paired training, while an averaged-feature proxy is used at inference. The authors evaluate on WeatherKITTI, nuScenes, and DrivingStereo, with component and efficiency ablations.

Critical strengths
  1. The evaluation goes beyond one synthetic benchmark: the paper reports results on WeatherKITTI, real-world nuScenes conditions, and zero-shot DrivingStereo transfer. In particular, the nuScenes result is useful because aligned clean training views are unavailable there.

  2. The paper makes a serious attempt to separate design choices. Table 6 includes CNN, Transformer, and Mamba adapters at roughly comparable cost, while Table 5 tests reference construction and reference dropout. The clear-condition and compound-weather analyses in the appendix are also relevant checks for a weather-robust depth model.

Critical weaknesses
  1. The evidence for selective weather forgetting is indirect. The gate uses an unwarped RGB difference between adjacent frames (Eq. 7), which is also large around parallax, moving vehicles, and occlusion boundaries. Figure 3 is schematic, and no gate maps or region-wise analysis show whether the model treats weather transients differently from genuine scene motion. Also, when the gate is zero, Eq. 13 still adds a projected propagated state to the encoder feature. The gate controls interpolation within the block; it does not guarantee that the original geometric features are retained.

  2. Test-time input parity is unclear. Eq. 7 and Algorithm 1 require a neighboring RGB frame for the temporal cue, whereas many standard monocular-depth baselines predict from one test image. Some listed baselines also use multiple frames, so the comparison is not uniformly unequal; the issue is that Tables 1–3 do not distinguish the available inference inputs. A target-frame-only SW-Mamba and matched-input comparisons would separate the benefit of an extra observation from the proposed correction mechanism. Table 6 should also clarify whether the cost of preparing the temporal input is included in latency.

  3. The deployment proxy may act as a low-pass filter rather than a reliable clean structural reference. Eq. 11 forms it by averaging the corrupted state itself, so spatially coherent fog, low light, or broad corruption can remain. This matters on nuScenes, where both clean-dependent auxiliary losses are disabled but the method still improves. Table 5 varies pooling and dropout, yet does not show whether the proxy is closer to a clean state or whether an equally sized spatial-smoothing module with matched losses explains the gain.

  4. Several ablation rows are difficult to interpret. In Table 4, what does 'temporal gating' modify before reference correction is enabled, and what reference is used when correction is enabled but the deployment proxy is absent? The WeatherDepth result in Table 1 is 0.103 Abs Rel, but the base model in Tables 4 and 6 is 0.128; Appendix Table 11 uses yet another score for its 'with' setting. These may reflect different experimental setups, but the differences must be stated. The small margin over the strongest WeatherKITTI baseline also lacks run-to-run variation.

Questions for authors
  1. Can the authors visualize learned gates and measure them separately on weather artifacts, moving objects, parallax, and occlusion boundaries? How does a spatial-only gate perform at matched capacity?

  2. Is the adjacent RGB frame available to SW-Mamba at test time on all benchmarks, and do comparisons use the same temporal inputs? What is the result when the gate is limited to the target frame?

  3. How close is the pooled proxy to the aligned clean state on WeatherKITTI, especially under fog and low illumination? Does it outperform simple feature smoothing under matched losses and compute?

  4. Please define the intermediate Table 4 variants, explain the differing baseline settings across Tables 1, 4, 6, and Appendix Table 11, and report variability over multiple training runs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.