acceptodds
Under review as a conference paper at ICLR 2027

PoEM: Reference-Guided Camouflaged Object Detection via Probabilistic Environment Modeling

Abstract

Reference-guided camouflaged object detection (RGCOD) aims to localize camouflaged objects in a query image using reference information. Existing approaches predominantly formulate RGCOD as feature matching: text-based methods are constrained by the cross-modal semantic gap, whereas vision-based methods typically rely on paired images with pixel-level annotations, limiting scalability and generalization. We propose **PoEM**, a training-free framework that recasts RGCOD as **P**r**o**babilistic **E**nvironment **M**odeling, requiring only target-free reference environment images and no paired annotations. The central idea is to estimate the query image’s background distribution from reference environments and identify camouflaged objects as regions that deviate from this distribution. To this end, we introduce a Describe–Sample–Filter (DSF) agent that automatically constructs high-quality reference environment images that are semantically and structurally aligned with the query image. We then use probabilistic principal component analysis (PPCA) to model the reference environment distribution in DINOv3 feature space and score each query patch by its negative log-likelihood under the resulting model. Higher scores indicate lower compatibility with the reference environment, providing cues for camouflaged object localization. When integrated with the Segment Anything Model (SAM), PoEM consistently outperforms state-of-the-art SAM-based RGCOD methods across multiple benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Camouflaged Object Detection, Environment Modeling, Probabilistic Principal Component Analysis

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XMxbOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PoEM addresses RGCOD by generating query-conditioned background images and identifying query patches that deviate from their feature distribution. The framework combines a Describe-Sample-Filter agent, multilayer DINOv3 features, PPCA with BIC-based rank selection, likelihood-based scoring, and SAM refinement. Validation includes four benchmark datasets, component ablations, alternative environment models, computational measurements, and supplementary failure analyses.

Critical strengths
  1. DSF extends the shared category-based synthetic gallery in EASE with per-query descriptions and candidate filtering. The filtering ablation improves results for all four tested language-model/generator combinations.
  2. The scoring ablation supports using covariance-aware deviations within this pipeline, with PPCA improving performance over the reported cosine-matching baseline. BIC also achieves performance comparable to a fixed rank of 2000 while reducing the reported number of estimable parameters from 2.10 million to 1.50 million, supporting its practical role in controlling model complexity.
Critical weaknesses
  1. The claimed conceptual departure is broader than the verified methodological advance. EASE already uses unannotated synthetic environment references, while PaDiM [ref.1] establishes Gaussian modeling of multilayer features and Mahalanobis localization. [ref.2] further fits a single Gaussian to pooled spatial features. Given the exact normalized NLL and squared Mahalanobis equivalence established in Eq. 19, PoEM’s contribution lies in query-conditioned filtered references combined with PPCA and BIC for camouflage detection, which narrows the originality claim to this specific integration.
  2. The evidence establishes benchmark effectiveness more clearly than the claimed causes and scope of the advantage. The ablation design and reporting leave the separate benefits of query conditioning and PPCA’s covariance constraint incompletely isolated, while the evaluation does not separately test open-world generalization or robustness to viewpoint and illumination changes. These uncertainties limit how strongly the results support the proposed paradigm’s superiority.

ref:

  1. Defard, T., Setkov, A., Loesch, A., & Audigier, R. (2021, January). Padim: a patch distribution modeling framework for anomaly detection and localization. In International conference on pattern recognition (pp. 475-489). Cham: Springer International Publishing.
  2. Bertoldo, J. P., & Arrustico, D. (2023, October). Visualization for multivariate gaussian anomaly detection in images. In 2023 Twelfth International Conference on Image Processing Theory, Tools and Applications (IPTA) (pp. 1-6). IEEE.
Questions for authors
  1. Which cited methods justify the cross-attention and supplied-annotation requirements assigned to prior approaches in Section 2.1 and Figure 1, given GenSAM’s visual self-attention and normalized text-image feature comparisons, and EASE’s unannotated environment references?
  2. What operational definition of open-world generalization is intended in Section 2.1, and which evaluation tests it? Please distinguish this property from class-agnostic segmentation, which is already explicitly defined in Concealed Object Detection, published in TPAMI 2022.
  3. What correspondence assumption does PPCA remove in Section 3.2 that EASE’s cross-position retrieval still requires, and what reported evidence supports the associated viewpoint and illumination robustness claim?
  4. What covariance regularization was used for the full-Gaussian baseline in Table 8, and does the PPCA advantage persist with numerical stabilization comparable to Appendix A.1?
  5. Is there a matched comparison between generating four filtered references for each query and retrieving four relevant filtered references from a shared gallery, with the foundation models and segmentation procedure fixed? Such a comparison would isolate query conditioning from the filtering benefit already demonstrated.
1 comment

The authors clarified the differences between PoEM and existing work and verified the benefits of generating four reference images per query image through controlled experiments.

The rebuttal does not specify the regularization method or strength used for the new baseline, making it difficult to attribute PPCA’s 0.001 advantage in weighted F-measure on NC4K to its covariance structure.

I maintain my rating.

Reviewer_SKXNOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PoEM, a training-free framework for reference-guided camouflaged object detection that reformulates the task as probabilistic environment modeling rather than explicit feature matching. The proposed Describe–Sample–Filter (DSF) agent generates target-free reference environment images conditioned on the query image. PoEM then employs DINOv3 features and probabilistic principal component analysis (PPCA) to characterize the background distribution, using negative log-likelihood scores to identify potential camouflaged regions. The resulting deviation maps are used to prompt SAM for segmentation.Experiments on four COD benchmarks demonstrate competitive performance against existing training-free methods. Ablation studies further investigate the effectiveness of DSF, probabilistic environment modeling, multi-layer features, and BIC-based component selection.

Critical strengths
  • An interesting environment-centric formulation of reference-guided detection.

Instead of relying on explicit target-level feature matching, the proposed framework identifies camouflaged objects through deviations from a modeled background distribution. This formulation relaxes the dependence on annotated, target-specific reference images and provides an alternative perspective for training-free camouflaged object detection. The integration of automatically generated environment references further improves its practical applicability.

  • Strong empirical performance with reasonably comprehensive component analysis.

PoEM achieves consistent improvements over existing training-free approaches across four benchmark datasets. The authors also provide useful ablations examining the DSF agent, feature representations, probabilistic modeling alternatives, and reference-set size. These experiments offer empirical support for the effectiveness of the overall framework.

Critical weaknesses
  • The methodological novelty of the probabilistic modeling component is somewhat limited.

Although the environment-centric formulation is interesting, its core operations rely on established techniques, including PPCA, BIC-based model selection, and Mahalanobis-distance-based anomaly scoring.

Moreover, the prior EASE [1] method has already explored environment-centered object discovery.

  • The experimental evidence could better isolate the contributions and robustness of the proposed framework.

Although comparisons with several SAM2-based baselines are included, additional backbone-controlled comparisons with the strongest competing approaches would strengthen the attribution of performance gains to the proposed environment modeling.

The DSF agent relies on stochastic image generation and language-model-based filtering, but the reliability of the generated reference images is not quantitatively evaluated.

[1] Shift the Lens: Environment-Aware Unsupervised Camouflaged Object Detection

Questions for authors

Q1. Contribution of probabilistic modeling versus segmentation backbones.

Table 1 reports substantial improvements when replacing SAM with SAM2 in PoEM. Could the authors provide additional comparisons using the same segmentation backbone for the strongest competing methods, where feasible?

Q2. Reliability and generalizability of the DSF agent.

How reliably does the DSF agent generate target-free reference images?

Q3. Additional clarification on PPCA and BIC.

Appendix A.2 acknowledges the working-independence approximation used for BIC-based rank selection. Could the authors report the distribution of selected ranks and explain the practical benefits of BIC-based selection relative to a fixed high-dimensional PPCA model?

Reviewer_HazaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PoEM, a training-free framework for reference-guided camouflaged object detection (RGCOD) that reformulates the task from feature matching to probabilistic environment modeling. The method introduces a Describe–Sample–Filter (DSF) agent to automatically generate query-conditioned, target-free reference environment images without requiring paired annotations. It then employs probabilistic principal component analysis (PPCA) on multi-layer DINOv3 features to model the background distribution, uses the Bayesian Information Criterion (BIC) to select the latent dimension, and computes negative log-likelihood scores to guide SAM-based segmentation. Experiments on four benchmark datasets (CHAMELEON, CAMO, COD10K, and NC4K) demonstrate improvements over existing training-free baselines, while ablation studies validate the effectiveness of the proposed components.

Critical strengths
  1. The paper provides an interesting perspective on reference-guided camouflaged object detection by shifting from conventional target-oriented feature matching to probabilistic environment modeling. This formulation removes the need for target-related reference images and annotations, making the framework more flexible and potentially scalable. The integration of query-conditioned reference generation with probabilistic deviation scoring is intuitive and well motivated.

  2. PoEM achieves competitive and consistently strong performance across four established benchmarks, outperforming the evaluated training-free baselines. The paper also provides meaningful ablation studies on the DSF agent, probabilistic environment modeling, multi-layer features, and BIC-based component selection. These experiments provide convincing evidence for the effectiveness of the proposed framework and support the paper's main claims.

Critical weaknesses
  1. The proposed method relies on PPCA to model environmental features under a low-dimensional Gaussian assumption. However, real-world backgrounds may exhibit complex and multimodal feature distributions that cannot be adequately represented by a single Gaussian model. Although the empirical results are promising, further analysis of the robustness of this assumption across diverse and challenging environments would strengthen the paper.

  2. The DSF agent requires multiple interactions with a multimodal language model and an image generation model to construct query-specific reference images. This introduces additional computational overhead compared with methods that directly process the query image. Although the paper reports inference time comparisons, a more detailed breakdown of the computational cost of each component, particularly the reference generation and filtering stages, would help clarify the practical scalability of the proposed framework.

Questions for authors
  1. In Appendix A.8 (Table 8), PPCA achieves Fβω=0.880F_\beta^\omega=0.880 on NC4K, compared with 0.876 for FG and 0.874 for FA. Since Appendix A.3 establishes the equivalence between normalized NLL and normalized squared Mahalanobis distance maps under the same covariance model, could the authors compare PPCA and FG/FA in fitting time, memory usage, and sensitivity to the number of reference patches? This would help clarify the practical advantage of the PPCA covariance structure beyond the relatively small segmentation gains.

  2. Appendix A.5 filters generated references using the prompt "Is there any animal or human in the image?" How does the method handle camouflage categories that are neither animals nor humans? Can the authors evaluate an object-agnostic filtering prompt under the same generation and PPCA settings, and report whether reference contamination affects the resulting deviation maps and segmentation accuracy?

  3. Table 5 shows that BIC-based selection reduces the reported number of estimable PPCA parameters from 2.10M at fixed s=2000s=2000 to 1.50M, while the COD10K weighted F-measure changes only from 0.865 to 0.866. Since Eqs. (8)–(10) require selecting the latent dimension by minimizing BIC over candidate dimensions, could the authors report the actual fitting and model-selection times, together with the distribution of selected dimensions across queries? Does adaptive selection offer a meaningful end-to-end computational advantage over a well-chosen fixed rank?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.