acceptodds
Under review as a conference paper at ICLR 2027

Learning to Complement: Rethinking Focal Selection for Light Field Salient Object Detection

Abstract

Light field salient object detection benefits from complementary focal cues, but independent slice scoring can favor redundant observations and overlook slices that become informative when combined. Effective fusion further requires accounting for dependencies between the selected focal evidence and all-focus features. We propose a framework that integrates complementarity-aware focal selection with relation-conditioned state-space fusion. Specifically, Complementarity-Aware Focal Selection and Aggregation (CFSA) estimates the conditional prediction gains of individual slices and slice pairs given the accumulated evidence. Its pairwise look-ahead captures complementary combinations whose predictive value is underestimated by individual scoring. The estimated gains guide progressive evidence selection and aggregation, with adaptive stopping under an encoding budget. Relation-Conditioned State-Space Fusion (RCSF) then models all-focus and focal features as observations of a shared spatial state. Cross-modal relations determine the observation mappings and weights in a regularized joint correction. The resulting closed-form update jointly incorporates the observation residuals while accounting for overlap between observation directions, allowing cross-modal dependencies to shape state correction and subsequent spatial propagation. Extensive experiments on four public light field salient object detection benchmarks demonstrate that the proposed method outperforms state-of-the-art approaches.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Light Field Salient Object Detection, Focal Slice Selection, Cross-Modal Fusion

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_tcdhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This study is for the light field salient object detection. In particular, it takes pairwise slices and all-focus representation into consideration and proposes two main models, CFSA and RCSF. CFSA combines single-slice and pairwise again estimation to identify complementary focal evidence for focal selection and aggregation while RCSF takes cross-modal reations to determine the observation mappings and weights for joint state correction. Combinging CFSA and RCSF progressively fulfills saliency prediction.

Critical strengths

The strength of this work lies in the two collaborated modules. CFSA aims at selecting proper focus while RCSF aims at mapping and weighting for fusion with the all-focus features. They combined through different layers for final saliency prediction.

Critical weaknesses
  1. The proposed method looks like a 'local+global' idea, with locally CFSA and globally RCSF. What are the key contributions of CFSA and RCSF? For CFSA, it relies on a pre-trained model to do the focus choice. For RCSF, it seems the state space is a copied idea of Longhorn.
  2. The progressive idea is not new, or even rather old, which makes the contribution weak again.
  3. The ablation is also rather rough. For example, CFSA takes several components, such as pariwise slices, all-focus features etc while RCSF having a joint target. It's better to have more indepth analyses on them.
  4. The motiviation shown in Figure 1 is not clear. Why do the top-K idea can better than weighted aggregation? The top-K still needs aggregation!
Questions for authors

The saliency detection is interesting, but the method proposed is not very attractive. The main reason is that the ideas inside are not very creative in my view. Please check the Weakness Section for my questions. In addition, the article is not easy to follow and it's better to polish again.

Reviewer_zhktOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the challenge of selecting and fusing evidence from focal stack slices in Light Field Salient Object Detection (LFSOD). We propose a mechanism called Complementarity-Aware Focal Selection and Aggregation (CFSA), alongside a method termed Relation-Conditioned State-Space Fusion (RCSF).

  1. Drawing inspiration from conditional mutual information in dynamic feature selection and loss-augmented supervision, CFSA evaluates the conditional prediction gains of individual slices and slice pairs based on the currently accumulated evidence. This approach overcomes the limitations of independent slice scoring—which often overlooks the complementarity between slices—and enables adaptive termination of the selection process under a fixed encoding budget.

  2. RCSF adopts an online regression perspective rooted in state-space models, treating the all-focus image and the selected focal-plane features as joint observations of a shared latent state. It dynamically parameterizes the observation mapping and weighting through cross-modal relationships, while explicitly modeling the overlap and coupling between observation modalities within a closed-form update solution.

  3. Extensive evaluations on four public light field salient object detection benchmarks—DUTLF-FS, HFUT, LFSD, and DUTLF-V2—demonstrate that our proposed method achieves state-of-the-art (SOTA) performance across MAE, SαS_{\alpha}, ESE_{S}, and FβwF_{\beta}^{w} metrics.

Critical strengths
  1. Previous methods often relied on heuristics—such as variance maximization, visual saliency-based retrieval, or static dense fusion—which frequently introduced redundant background information. This paper introduces, for the first time, a paired "look-ahead" evaluation mechanism driven by the reduction in downstream loss (loss-improvement supervision). This approach effectively identifies complementary combinations—elements that appear insignificant in isolation but become critical when paired—offering a highly insightful methodological contribution.

  2. Rather than relying solely on simple tensor concatenation or attention-based interactions, RCSF employs a closed-form update for joint state estimation, incorporating regularization to explicitly penalize or model the overlap between all-focus features and focal-specific features. This enables residual correction across modalities to be mutually constrained and coupled during propagation; the mathematical derivation is clear, and the method demonstrates high computational efficiency.

Critical weaknesses
  1. Although the authors claim in the "Method" section that CFSA "reuses lightweight previews and cached features" during inference, Equation (2) indicates that CFSA requires evaluating the joint gain for all unselected individual slices—as well as all unordered pairs of slices—within the candidate set at each step. As the depth of the focal stack increases, the number of such pairs grows quadratically. The paper lacks a comparison between the actual wall-clock inference latency (or FPS) of the CFSA selection stage and that of traditional dense aggregation methods, raising questions about its efficiency for practical deployment.

  2. In the implementation details (Section 4.1), the authors state: "We first train the saliency predictor and then optimize CFSA's conditional-gain estimator, keeping the encoders, focal aggregation component, RCSF, and decoder fixed during the second stage." This decoupled training strategy—fixing the downstream predictor before training the gain estimator—carries a significant risk of distribution shift. As the gain estimator refines its selection policy during the second training stage, the downstream predictor is not adapted to the distribution of subsets generated by this updated policy. The authors provide no ablation study comparing this two-stage approach against joint end-to-end training.

  3. In Section 3.2, the authors note: "Compact two-row heads (ds=2d_s=2) preserve the full transition... each head is read out through an input-conditioned query." By forcing complex, full-focal-stack features into an extremely compact 2-row subspace (represented by a 2×22 \times 2 system matrix GtG_t), the method ensures that off-diagonal interactions remain analytically tractable. However, it is highly questionable whether such a low-rank, 2D state can losslessly encode the rich, multi-scale saliency semantics present across the entire image. The authors lack a sensitivity analysis regarding the subspace dimension ds∈{2,4,8,… }d_s \in \{2, 4, 8, \dots\}.

  4. In the ablation study presented in Table 3, the authors compare the full model against "w/o CFSA" and "w/o CFSA & RCSF" (i.e., a dense concatenation baseline); however, they fail to include an ablation setting for "w/o RCSF" (i.e., retaining the CFSA selection mechanism while using standard concatenation for fusion). Consequently, readers cannot clearly discern from the table to what extent the performance gains are attributable to "CFSA-based focal selection" versus "RCSF-based state-space fusion." The synergy between these two components lacks quantitative support for the individual contributions of each part.

Questions for authors
  1. During the inference phase, when the number of candidate slices is large, what proportion of the total end-to-end forward-pass latency (or FPS) is accounted for by the computation of gains for individual slices and all possible pairs? Is there any empirical data comparing the actual runtime speed of your method against direct dense aggregation or baseline algorithms (such as LF-Mamba or VMKNet)?

  2. When training the Gain Estimator while keeping the downstream saliency predictor fixed, is the distribution of sampled subsets seen by the Gain Estimator identical to the distribution encountered during the training of the downstream predictor? Furthermore, why was an end-to-end joint optimization approach—such as one utilizing Reinforcement Learning or a Straight-Through Estimator—not adopted?

  3. What is the rationale behind choosing ds=2d_s=2 (for the "compact two-row heads")? If the state subspace dimension dsd_s were increased (e.g., to ds=4d_s=4 or 88), would the model's performance on the DUTLF-FS and HFUT datasets saturate or decline? Would such an increase impose an excessive computational burden?

  4. To strictly decouple the contributions of individual modules, could you provide ablation study results for a configuration where only the RCSF component is removed (i.e., retaining the CFSA selection mechanism while replacing RCSF with standard feature concatenation), and include these results in Table 3?

Reviewer_zCiaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper studies focal-slice selection and fusion for light field salient object detection. Given the available evidence, CFSA predicts how much adding one or two slices would reduce the prediction loss and progressively selects slices accordingly. RCSF treats the selected focal features and all-focus features as observations of a shared spatial state and fuses them through a relation-conditioned joint update. The paper reports strong results on four public datasets.

Critical strengths
  1. Selecting slices according to their contribution to the current prediction is a reasonable motivation. Considering the joint value of two slices could also address limitations of selection based only on individual slices.

  2. RCSF provides an explicit optimization objective and closed-form update, making it possible to understand how the two observations jointly affect the state. The reported results are competitive, particularly the comparisons with existing light field methods in Table 1.

Critical weaknesses
  1. My main concern is that the significance and novelty of the additional contributions are not sufficiently established. The paper's direct contribution concerns selection and fusion for light field salient object detection. It explains why complementary focal information is useful for this task, but the current argument mainly emphasizes improved detection metrics. The practical importance of the difficulty being addressed and the new understanding gained from the study are less clear.

Methodologically, estimating feature value through loss reduction, selecting features conditional on current evidence, and adapting stopping criteria build on Gadgil et al. [1]. Deriving state-space updates through online regression builds on Longhorn [2]. The specific extensions here are candidate-slice previews, selection over singleton and pairwise actions, and relation-conditioned state correction that retains coupling between two observations within a small state. These are concrete new designs, but the advance over the existing frameworks appears incremental, and its important benefits have not been directly established.

  1. The experiments mainly establish that the complete network performs better, while providing limited insight into the central mechanisms. The most interesting claim is that two slices with little individual value can provide a substantial joint benefit. However, Table 3 removes CFSA as a whole rather than directly comparing conditional singleton selection with a policy that also considers pairs; Table 4 only compares fixed slice counts. The results therefore do not establish whether this type of complementarity is important in real data or which failures of singleton selection the pairwise strategy resolves. A comparison using the same predictor and matched actual or average encoded slice counts, together with representative real examples, would be more informative about the contribution than further aggregate metrics.

  2. For readability, the spacing between figures and text could be improved. Please reduce unnecessary blank space, keep related material together, and use the recovered space for essential explanations. This is a minor layout issue.

References [1] Soham Gadgil, Ian Covert, and Su-In Lee. Estimating Conditional Mutual Information for Dynamic Feature Selection. ICLR, 2024. https://openreview.net/forum?id=Oju2Qu9jvn

[2] Bo Liu, Rui Wang, Lemeng Wu, Yihao Feng, Peter Stone, and Qiang Liu. Longhorn: State Space Models are Amortized Online Learners. ICLR, 2025. https://openreview.net/forum?id=8jOqCcLzeO

Questions for authors

Please see weakness. If the authors can address the issues above, I will consider raising my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.