acceptodds
Under review as a conference paper at ICLR 2027

Learning When to Trust Depth: Scale-Wise Latent Reliability Routing for RGB-D Salient Object Detection

Abstract

Depth benefits RGB-D salient object detection only when its geometric evidence is reliable and relevant to the target; noisy, incomplete, or misaligned depth can instead cause negative cross-modal transfer. We formulate RGB-D fusion as inference of a scale-wise latent reliability variable that controls how the modalities interact at each representation level. Our framework, MPLoRA-MoE, combines shared semantic encoding, modality-specific low-rank adaptation, and four heterogeneous fusion experts. At each scale, an independent router predicts a soft expert distribution from pre-fusion RGB and depth features, allowing depth evidence to be preserved, attenuated, or reconciled without quality or profile annotations. The routed hierarchy is decoded through a single edge-semantic pathway, so conditional computation changes the fused representation rather than selecting among multiple outputs. Across eight RGB-D SOD benchmarks, MPLoRA-MoE achieves state-of-the-art performance on multiple datasets and remains competitive on the others. Ablations further show that the gains depend on reliability-oriented expert specialization, while moderate routing sharpness and compact modality-specific adaptation are sufficient. These results support learning when and how strongly depth should contribute instead of treating it as uniformly trustworthy. Code will be released upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
RGB-D Salient Object Detection, RGB-D Modality Reliability, Scale-Wise Reliability Routing, Mixture of Experts, Low-Rank Adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GT1kOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MPLoRA-MoE for RGB-D salient object detection. The main idea is that depth isn't always reliable, so the model should learn how much to trust it at each feature scale. RGB and depth share a PVT-v2-B5 encoder, and each modality gets its own LoRA residuals. At each of the four scales, a small router softly mixes four fusion experts that take different combinations of raw and cross-aligned RGB/depth features. The router gets no reliability labels and is trained only through the saliency and edge losses. The fused features then go into an ESCNet-style edge-semantic decoder. On eight benchmarks, the method is best on ReDWeb-S, SIP, COME-E, and COME-H, and roughly on par with prior methods on NJU2K, NLPR, DUT-RGBD, and STERE. Ablations cover edge guidance, the number and type of experts, the routing temperature, and the LoRA rank.

Critical strengths
  1. Good cross-dataset results. The method is best on the four unseen benchmarks (e.g., ReDWeb-S S_alpha .773 vs. .752 for the best baseline), which is where adaptive fusion should matter most. This is the main reason I don't lean toward a clear reject.
  2. Table 3 is a useful control. Four heterogeneous experts beat four or eight homogeneous ones, so the gain is not just from adding branches. This partly supports the idea that different fusion paths are needed.
Critical weaknesses
  1. The central claim, that the router learns when to trust depth, is never shown. There is no plot of routing weights, no per-scale statistics, and no link to depth quality. Fig. 1(c) even shows depth-oriented routing at shallow scales and RGB-oriented routing at deep scales, which is the opposite of what the text says (L88-92, L463-465). Since the title and the framing rest on this claim, it's the main reason for my score.
  2. The gains can't be attributed to routing. The obvious ablation, the same four experts with uniform weights (no router), is missing, and so is an image-level router. The backbone is PVT-v2-B5 with no params/FLOPs comparison and no same-backbone baselines, so part of the gain may come from the encoder. Apart from the router, most parts are borrowed (LoRA, coordinate attention, deformable conv, the ESCNet decoder), so without these controls the contribution feels thin for ICLR.
Questions for authors
  1. What do the routing weights look like per scale and per dataset, and do they shift when depth is corrupted or dropped at test time? A clear yes would directly support the main claim and could change my score.
  2. How does the model do with uniform expert weights, or with a single image-level router?
  3. Can the authors compare with a few baselines using the same backbone and report params/FLOPs?
2 comments
AuthorsOpenReview ↗

We have addressed your concerns with new experimental results in the main paper and additional details in the Appendix.

We thank the reviewer for highlighting the need for direct routing evidence, controlled ablations, and complexity reporting. We have added experiments and revised the relevant discussion.

Q1–Q2: Reliability-responsive and scale-wise routing (Tables 3–4; Appendix Table 6)

1. Controlled degradation (Table 3). We select 547 RGB-D pairs from eight benchmarks using independently trained RGB-only and depth-only F3Net models, retaining images with high S-measure in both modalities. The same images and masks are evaluated under clean, RGB-degraded, depth-degraded, and jointly degraded conditions (Appendix A.1).

Table 3 reports E1–E4 routing weights at all four scales. Averaged across scales, the dominant expert shifts from E1 (both-oriented, 0.415) on clean inputs to E3 (depth-oriented, 0.427) with degraded RGB, E2 (RGB-oriented, 0.428) with degraded depth, and E4 (all-source, 0.374) when both degrade. Responses also differ by scale: RGB degradation increases E3 by 0.283 at S1 vs. 0.149 at S4; depth degradation increases E2 by 0.202 vs. 0.170.

These results directly link controlled modality corruption to expert-weight redistribution. We interpret routing weights as task-relative fusion preferences, not calibrated depth-quality estimates. Figure 1(c) and the text now clarify that scale-wise routing does not impose a fixed shallow-to-deep modality preference.

2. Matched routing ablations (Table 4). With the backbone, four experts, and decoder fixed, we compare static scale-wise routing (learned input-independent weights), image-level dynamic routing (one shared distribution), and our scale-wise dynamic routing.

Their S-measure/MAE results are 0.906/0.043, 0.909/0.041, and 0.915/0.035 on SIP, and 0.844/0.078, 0.848/0.076, and 0.855/0.069 on COME-H. These results demonstrate benefits from input-dependent, scale-specific expert weighting beyond merely having four branches. The static control uses learned weights rather than a literal uniform mixture.

3. Expert structure (Table 5(a)). Four heterogeneous E1–E4 experts improve SIP S-measure from 0.909 to 0.915 over four homogeneous E4 experts while using fewer expert parameters (3.09M vs. 10.77M).

4. Corruption robustness (Appendix Table 6). On the same degraded images, our S-measure is 0.943 (RGB), 0.959 (depth), and 0.803 (both), versus the best competing results of 0.925, 0.953, and 0.788. This complements the routing statistics with predictive performance under matched corruptions. We test controlled degradation rather than complete depth removal or a severity sweep.

Q3: Backbone and computational complexity (Table 1; Appendix Table 7)

Table 1 now reports parameter counts; Appendix Table 7 additionally provides GFLOPs and input resolutions. MPLoRA-MoE has 105.17M parameters and 341.72 GFLOPs at 416×416, compared with 223.10M parameters for CPNet, 453.03M for LAFB, 272.10M for CATNet, and 368.42M for DPPNet.

All four experts execute under dense soft routing, so we do not claim computational savings from sparse expert activation. Different methods also use different backbones and resolutions; these comparisons are not backbone-controlled.

Although we have not retrained external baselines with PVT-v2-B5, Table 4 fixes our backbone and architecture across routing variants, isolating the routing strategy rather than encoder capacity.

We hope the new controlled routing statistics, matched ablations, and complexity results address the reviewer's core concerns.

Reviewer_GT1kOpenReview ↗

Feedback

Thanks to the authors for carefully adding degradation experiments, routing comparisons and parameter statistics. Routing weights will change in the expected direction with mode degradation, which answered my most important question, so I raised the score to 3. However, except for routing, most of them are existing modules, and the incremental improvements of the routing mechanism compared to existing adaptive fusion methods (such as LAFB) are also limited. I still have reservations about this.

Reviewer_uqSAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose MPLoRA-MoE to address the usages of unreliable depth information in RGB-D salient object detection. The model employs sample-dependent routing at multiple feature scales to softly combine heterogeneous experts, enabling RGB-oriented, depth-oriented, and other complementary fusion strategies for integrating RGB and depth features before decoding. Experiments on eight benchmarks show competitive performance, while ablation studies on SIP demonstrate the effectiveness of edge guidance and MoE.

Critical strengths
  1. The paper presents a clear modality-reliability-aware fusion framework. The multiple specialized experts to model different RGB/depth reliability conditions provides an interesting perspective on adaptive RGB-D fusion.
  2. The model achieves the best or joint-best results across all reported metrics on ReDWeb-S, SIP, COME-E, and COME-H. Qualitative comparisons further illustrate effective use of complementary modality information and accurate segmentation across multiple challenging scenarios.
Critical weaknesses
  1. Although the ablation results support the effectiveness of the overall architecture, direct evidence of the intended expert specialization and reliability-aware routing behavior remains limited. The current comparisons do not sufficiently distinguish the benefits of these mechanisms from those of expert structure and model capacity.
  2. The incremental contribution over LAFB requires clearer justification, since LAFB already adaptively combines multiple fusion schemes at different feature levels. The paper would benefit from clarifying what is substantively new in its reliability-oriented expert design and routing mechanism.
Questions for authors
  1. Could the authors provide empirical evidence of the router’s effectiveness and analyze whether it correlates well with task-relevant modality reliability?
  2. Could the authors compare individual experts across sample groups with different modality reliability conditions to validate their specialization? For example, how does the depth-oriented expert’s performance change between depth-dominant and RGB-dominant samples?
  3. Could the authors provide parameter- and compute-matched comparisons to distinguish the benefits of the proposed design from those of increased model capacity?
1 comment
AuthorsOpenReview ↗

We have addressed your concerns with new experimental results in the main paper and additional details in the Appendix.

We thank the reviewer for the constructive feedback. We have added controlled modality degradation, routing analyses, and complexity comparisons to clarify the proposed contributions.

Q1: Reliability-aware routing (Tables 3–4)

We construct a diagnostic subset of 547 RGB-D pairs from eight benchmarks using independently trained RGB-only and depth-only F3Net models, selecting images with high S-measure from both. Identical samples and masks are evaluated under clean, RGB-degraded, depth-degraded, and jointly degraded conditions (Appendix A.1).

Table 3 reports mean routing weights at four scales. The dominant expert changes from E1 (both-oriented, 0.415) on clean inputs to E3 (depth-oriented, 0.427) when RGB is degraded, E2 (RGB-oriented, 0.428) when depth is degraded, and E4 (all-source, 0.374) when both are degraded. Responses differ across scales: RGB degradation increases E3 by 0.283 at S1 versus 0.149 at S4; depth degradation increases E2 by 0.202 versus 0.170. These shifts support quality-responsive routing, although the weights reflect task-relative fusion preferences, not calibrated physical depth-quality estimates.

Table 4 further isolates routing with the same backbone, experts, and decoder. Static scale-wise, image-level dynamic, and scale-wise dynamic routing achieve S-measure/MAE of 0.906/0.043, 0.909/0.041, and 0.915/0.035 on SIP, and 0.844/0.078, 0.848/0.076, and 0.855/0.069 on COME-H, respectively. The static variant learns input-independent per-scale weights; the results support both input dependence and scale-specific adaptation.

Q2: Expert specialization and novelty over LAFB (Tables 3, 5(a); Appendix Table 6)

The condition-dependent routing shifts in Table 3 align with the experts' intended roles: clean inputs favor E1, RGB degradation E3, depth degradation E2, and joint degradation E4. We clarify that these are routing-weight statistics, not standalone expert accuracy measurements; they support specialization of the learned mixture rather than prove independent expert performance.

Table 5(a) tests whether the heterogeneous structure helps beyond adding branches. Homogeneous configurations repeat the all-source E4 expert; heterogeneous K=4* combines E1–E4. On SIP, heterogeneous fusion improves S-measure/MAE from 0.909/0.037 to 0.915/0.035, using 3.09M versus 10.77M expert parameters. On COME-H, it improves 0.852/0.071 to 0.855/0.069.

We acknowledge that LAFB already performs hierarchical adaptive fusion. Our novelty is not multi-branch adaptation itself: LAFB adaptively reweights challenge-specific fusion features, whereas our router predicts a four-expert mixture from pre-fusion modality features at each scale, with paths structured around joint RGB–depth reliability relations. Under controlled RGB, depth, and joint degradation (Appendix Table 6), our S-measure exceeds LAFB's: 0.943 vs. 0.925, 0.959 vs. 0.941, and 0.803 vs. 0.778, respectively. These comparisons support the design's practical value without isolating every difference between architectures.

Q3: Capacity and computation (Tables 1, 4, 5(a); Appendix Table 7)

We now report parameter counts in Table 1 and parameter/GFLOPs/input-resolution details in Appendix Table 7. Our method uses 105.17M parameters and 341.72 GFLOPs at 416×416, versus 223.10M parameters for CPNet, 453.03M for LAFB, 272.10M for CATNet, and 368.42M for DPPNet. Since all experts run in dense soft mixing, we do not claim computational savings; input resolutions also differ across models.

A strict parameter- and FLOPs-matched comparison against external architectures is unavailable. Nevertheless, Table 4 holds the backbone and expert configuration fixed to isolate routing, while Table 5(a) shows that a smaller heterogeneous expert set outperforms a larger homogeneous one. These controls address important capacity confounds, without eliminating them entirely.

We hope these revisions strengthen the evidence for reliability-responsive routing and clarify the contribution beyond existing adaptive fusion approaches.

Reviewer_9Xr2OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes scale-wise routing of RGB and depth features for salient object detection. A shared encoder with modality-specific adapters feeds four fusion experts, whose outputs are mixed by a router at each scale. The method is tested on eight benchmarks.

Critical strengths

The evaluation is broad. The method performs particularly well on benchmarks not used for training, and the paper reports all four metrics rather than selecting a single favorable result. The ablations examine expert structure, routing temperature, and adapter rank.

Critical weaknesses

The paper calls the routing weights a measure of when to trust depth, but it does not directly test whether those weights respond to reliable or corrupted depth as intended. Strong saliency scores alone cannot establish that interpretation. A controlled depth degradation experiment and per-scale routing analysis would make the central claim much more convincing. The gains from heterogeneous experts are not fully separated from model capacity. One proposed expert uses deformable convolution, whereas the homogeneous comparison uses repeated experts of another structure. The paper also gives no parameters or inference time. Since all four experts are evaluated for soft mixing, the method should not imply a computational saving from conditional routing without measuring one.

Questions for authors
  1. The eight-dataset evaluation is a strength, but it does not directly show that the router learns when depth should be trusted. A controlled test with progressively noisy, missing or misaligned depth would test that claim.
  2. Please show how the routing weights change across scales for cases where RGB is useful, depth is useful, or the two disagree. The paper currently interprets the experts in these terms without showing that the learned routes follow them.
  3. A uniform mixture and a single image-level router are important controls. Without them, the results do not establish that learned scale-wise routing is needed, rather than simply having four fusion paths.
  4. The heterogeneous expert comparison is not fully matched. The proposed set includes a deformable convolution expert, while the homogeneous alternatives repeat a different structure. Please compare models with similar parameter counts and computation.
  5. All four experts are evaluated before their outputs are mixed. The method therefore does not appear to save computation through routing. Parameter count, memory use and inference time should be reported alongside accuracy.
  6. Table 5 varies the LoRA rank but has no model without LoRA. A shared encoder without modality-specific adapters is needed to establish their contribution.
  7. It is unclear whether the competing methods in Table 1 were evaluated under matched training and input conditions. SATNet also uses estimated depth while the other methods use dataset depth. These differences should be made explicit when interpreting the rankings.
  8. The gains on unseen datasets are encouraging, but they are attributed to changing modality reliability without a direct breakdown by depth quality. Results across several runs, together with performance on different depth conditions and representative failure cases, would make that interpretation more convincing.
1 comment
AuthorsOpenReview ↗

We have addressed your concerns with new experimental results in the main paper and additional details in the Appendix.

We thank the reviewer for the constructive suggestions. We have added controlled degradation experiments, matched routing ablations, and complexity analysis.

Q1–Q2: Reliability-responsive routing (Table 3; Appendix Table 6)

We construct a diagnostic subset of 547 RGB-D pairs from eight benchmarks using independently trained RGB-only and depth-only F3Net models, selecting images with high S-measure in both modalities. We evaluate the same images and masks under Both-clean, RGB-degraded, Depth-degraded, and Both-degraded conditions (Appendix A.1).

Table 3 reports four expert weights at each of four scales. The dominant expert changes from E1 (both-oriented, 0.415) for clean inputs to E3 (depth-oriented, 0.427) with degraded RGB, E2 (RGB-oriented, 0.428) with degraded depth, and E4 (all-source, 0.374) when both are degraded. These are averages over the four scales. Responses also vary by scale: RGB degradation increases E3 by 0.283 at S1 vs. 0.149 at S4; depth degradation increases E2 by 0.202 vs. 0.170. Thus, learned routing responds to modality corruption with scale-dependent adjustments. We interpret weights as task-relative fusion preferences, not calibrated depth-quality scores. We evaluate controlled degradation, not progressive corruption or complete removal.

Q3: Routing controls (Table 4)

Holding the backbone, four experts, and decoder fixed, we compare static scale-wise (learned input-independent weights), image-level dynamic (one distribution shared across scales), and scale-wise dynamic routing (ours). Their S-measure/MAE results are 0.906/0.043, 0.909/0.041, and 0.915/0.035 on SIP, and 0.844/0.078, 0.848/0.076, and 0.855/0.069 on COME-H. These controls demonstrate benefits beyond simply providing four fusion paths. The static control is more flexible than literal uniform averaging.

Q4: Expert structure and capacity (Table 5(a))

Homogeneous models replicate the E4 deformable-convolution expert; heterogeneous K=4* combines E1–E4. On SIP, heterogeneous K=4* improves S-measure/MAE from 0.909/0.037 to 0.915/0.035, despite using fewer expert parameters (3.09M vs. 10.77M). On COME-H, performance improves from 0.852/0.071 to 0.855/0.069. Thus, gains are not explained by greater expert parameter count alone, although the configurations are not FLOPs-matched.

Q5: Computational cost (Table 1; Appendix Table 7)

We now report parameters in Table 1 and input resolutions and GFLOPs in Appendix Table 7. Our model has 105.17M parameters and 341.72 GFLOPs at 416×416. All four experts are evaluated during dense soft routing, so we do not claim computational savings from expert activation. Peak memory and inference latency have not been measured; cross-model GFLOPs also reflect different input sizes.

Q6: LoRA-free control (Table 2)

With Edge and MoE retained under the same shared backbone, removing modality-specific LoRA decreases SIP S-measure from 0.915 to 0.907 and increases MAE from 0.035 to 0.039. This matched comparison supports the contribution of LoRA; Table 5(c) reports rank sensitivity.

Q7–Q8: Evaluation protocol and robustness (Section 4; Appendix Tables 6–7)

We clarify that SATNet uses estimated monocular depth, while the other methods use dataset depth, and disclose each method's input resolution. The first three benchmarks use held-out test splits of training-source datasets; the other five are unseen during training. Cross-method results should not be interpreted as identical-backbone or identical-training comparisons.

Appendix Table 6 compares five methods on the same 547 images under clean, RGB-, depth-, and jointly degraded conditions. Our S-measure under the three corruptions is 0.943, 0.959, and 0.803, exceeding the best competitors' 0.925, 0.953, and 0.788. This directly evaluates corruption robustness rather than inferring it from cross-dataset accuracy. Repeated-run uncertainty estimates and systematic failure-case analysis remain outside the present evaluation.

Together, Tables 3–4 directly address quality-responsive routing and scale-wise necessity, while Tables 1–2, 5, and Appendix Tables 6–7 clarify capacity, LoRA, computational cost, and robustness.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.