acceptodds
Under review as a conference paper at ICLR 2027

Cluster-Consensus Bayes for Open-Vocabulary Camouflaged Object Segmentation

Abstract

Open-vocabulary camouflaged object segmentation requires recognizing visually subtle objects from unseen categories. Existing methods typically process each test image independently, overlooking the latent consensus among related samples. We introduce Cluster-Consensus Bayes (CCB), a lightweight, plug-and-play empirical Bayes module that turns this consensus into a source of semantic evidence. CCB groups unlabeled test images by visual similarity and estimates cluster-level class priors from their predicted categories. It then adaptively fuses these priors with individual vision-language predictions according to prediction confidence. This allows uncertain samples to draw on shared evidence while confident samples retain their individual information, without assigning a single label to an entire cluster. CCB supports iterative offline refinement across complementary encoder views and online inference through confidence-weighted centroid memory. Both modes refine predictions at test time without gradient updates, accommodating collection-based and streaming inference. Experiments across five base models and two additional single-target datasets demonstrate consistent recognition gains. On OVCamo, CLIP-only CCB improves OVCoser’s accuracy from 68.8% to 73.2%, with a 6.2% average relative gain across six class-aware segmentation metrics. An enhanced system combining auxiliary DINO feature fusion with multi-view CCB achieves a 14.3% average relative gain over original OVCoser.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Open-Vocabulary Recognition, Empirical Bayes, Test-Time Adaptation, Vision-Language Models, Cluster Consensus, Transductive Inference, Streaming Inference

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_AwLhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3(+1)
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper introduces a lightweight, plug-and-play module called “Clustering Consensus Bayesian” (CCB) for open-vocabulary object segmentation with camouflage (OV-COS). CCB uses a frozen encoder to group visually similar unlabeled test images. It combines a Dirichlet-smoothed Top-1 voting-based class prior with individual predictions through adaptive fusion using a confidence-weighted geometric mean based on predicted entropy. CCB runs entirely during the testing phase without requiring gradient updates. It supports both offline batch processing and streaming processing modes (PC-WOC) and demonstrates higher segmentation accuracy across multiple benchmarks and five different base models.

Critical strengths

This method is driven by practical and intuitive motivations and ingeniously leverages the shared semantic context in related test images to resolve ambiguous objects. Its plug-and-play design requires no model retraining or architectural modifications, making it widely applicable. This paper also presents a comprehensive empirical evaluation across various models and introduces two complementary inference modes; notably, the streaming mode maintains constant memory usage, which offers significant advantages for real-world deployment.

Critical weaknesses

The most critical flaw of this paper is its lack of fundamental innovation. Its core concepts bear a striking resemblance to well-established techniques such as test-time adaptation, transduction reasoning, and self-training, yet the authors fail to rigorously distinguish their work from these existing approaches. Furthermore, the performance gains are too modest to justify the added complexity, and the paper severely lacks fair comparisons with key baseline models such as TENT and TPT. The theoretical contributions are superficial (e.g., the absence of convergence proofs), the performance of the streaming mode lags far behind that of the offline variants, and the extensive yet redundant ablation experiments fail to explore critical failure modes or hyperparameter sensitivities.

Questions for authors

The authors are required to conduct a comprehensive comparison with a correctly implemented runtime adaptive baseline and provide a detailed analysis of computational overhead (number of floating-point operations, memory usage, and actual runtime) for both modes. In addition, the reviewers requested that the authors justify the reasonableness of the core hyperparameters through sensitivity analysis, examine failure cases (such as rare classes or skewed distributions), provide theoretical convergence guarantees for offline optimization, analyze the risk of noise introduced by misaligned clustering encoders, and propose solutions to narrow the performance gap between the streaming and offline modes.

4 comments
AuthorsOpenReview ↗

Response to Reviewer AwLh (Part 1/3)

We thank Reviewer AwLh for acknowledging CCB's practical motivation, retraining-free plug-and-play integration across five base models, and constant-memory streaming mode. Below we address each technical point with theoretical analysis and empirical measurements.

Comparison with TTA Baselines, Computational Overhead, Orthogonal Combinations, and Distinct Contributions (W1 & Q1)

We evaluated gradient-based (TENT, TPT) and gradient-free (T3A, TDA, LAME) baselines under the exact same frozen OVCoser base model and protocol (Table R1). Existing test-time methods hit clear bottlenecks on OV-COS:

  1. Self-training and gradient-based TTA (TENT, TPT): Updating weights via hard pseudo-labels and multi-step backpropagation is costly and prone to confirmation bias when camouflaged foreground and background entangle. TENT (Table 5, Appendix Table 9) improves class-agnostic masks (agIoU 0.583→0.602) but lacks cross-image context, slightly lowering classification accuracy; TPT relies solely on single-image crop consistency without cross-image semantic consensus.
  2. T3A assumes a closed-set setting and bins samples by hard pseudo-labels to update prototypes, causing prototype contamination under open-vocabulary predictions.
  3. TDA uses a moderate-entropy negative cache that mis-suppresses visually similar camouflaged classes, and its instance-level retrieval uses a fixed residual weight ignoring target sample uncertainty.
  4. LAME targets small batches via unnormalized pairwise graph Laplacian smoothing. Over the full dataset with its default dense RBF kernel, accumulated node degrees overwhelm unary logits and collapse accuracy (4.8%). For a fair comparison, Table R1 therefore reports LAME with RBF at B=16B=16 and a sparse kNN5 kernel on the full set.

By contrast, CCB combines three complementary designs for OV-COS. First, CCB decouples visual clustering from class assignment: it clusters in frozen visual space and computes normalized soft class distributions per cluster rather than single-class clusters, avoiding T3A's hard-label contamination and LAME's degree explosion. Second, instead of TDA's fixed residual weights and negative cache, CCB modulates the cluster prior weight per sample by its prediction confidence, preserving confident predictions while correcting ambiguous ones. Third, CCB unifies full-set offline transductive and online streaming inference within one training-free Bayesian formulation across batch sizes, stream orderings, and encoders.

Under this unified evaluation (Table R1), CCB outperforms baselines within each paradigm with negligible overhead:

  • Online streaming: CCB (Streaming) surpasses T3A and TDA across accuracy, segmentation quality, and memory. On an Intel Xeon Gold 5218R CPU, it needs only 0.086 MFLOPs/image—negligible vs. one base OVCoser forward pass on an RTX 3090 (464,094 MFLOPs, 63.71 ms).
  • Offline transductive: Even with a sparse kNN5 kernel on the full set, CCB (Offline, CCC) surpasses LAME-kNN5 in accuracy and Macro-F1 with 4.5x faster runtime, 78.8% less memory, and 9.18 vs. 51.79 MFLOPs/image; cross-encoder multi-view refinement widens this margin further.
  • Gradient-based TTA: Gradient-free CCB (Streaming) and CCB (Offline, CCC) surpass TENT and TPT by a wide margin at a fraction of the cost, whereas backpropagation and multi-view passes add ~124 ms (TENT) and ~5,018 ms (TPT) adaptation latency/image on an RTX 3090.

More importantly, because CCB operates on output probability distributions, it stacks orthogonally with both gradient-free and gradient-based TTA: adding online CCB after TDA raises accuracy from 69.6% to 72.4% (66.7% Macro-F1); offline CCB after LAME-kNN5 raises accuracy to 73.9%; and offline CCB after spatially adapted TENT preserves mask quality while lifting cIoU to 0.478. These orthogonal gains confirm CCB's population-level structural prior is complementary to existing TTA mechanisms.

ParadigmMethodAcc(%)↑Macro-F1(%)↑cSmcS_m↑cFβωcF_\beta^\omega↑cMAE↓cFβcF_\beta↑cEmcE_m↑cIoU↑CPU(ms/img)RSS(MB)
BaselineBase OVCoser68.861.70.5790.4900.3360.5200.6160.443——
Gradient TTATENT68.661.60.5800.5090.3340.5350.6260.456——
TPT70.564.00.5920.5000.3190.5310.6300.451——
StreamingT3A65.358.30.5510.4680.3690.4960.5860.4230.14034.65
TDA69.662.80.5850.4940.3270.5240.6230.4460.66735.01
CCB (Streaming)72.465.50.6050.5060.3010.5370.6430.4570.1943.80
OfflineLAME (RBF, B=16)69.061.80.5810.4910.3340.5210.6180.4440.09035.60
LAME (kNN5, Full)72.765.30.6100.5150.2990.5460.6490.4650.829204.63
CCB (Offline, CCC)73.266.60.6110.5130.2940.5450.6510.4620.18243.45
AuthorsOpenReview ↗

Response to Reviewer AwLh (Part 2/3)

Well-Posedness and Cross-Round Convergence of Offline Optimization (W1 & Q3)

CCB (Offline) runs a short-horizon alternating loop in which each round consists of two deterministic steps: K-means clustering in feature space and variational fusion on the probability simplex:

  1. Per-round convergence and unique solution: Within each round, K-means monotonically decreases within-cluster sum-of-squared errors over a finite space of sample partitions and is guaranteed to terminate in finite steps. Given the resulting cluster distributions rai(r)∈ΔK−1r_{a_i}^{(r)} \in \Delta^{K-1} and sample confidence weights αi(r)∈[0,1]\alpha_i^{(r)} \in [0, 1], the variational fusion step minimizes the weighted KL objective Ji(q)=αiDKL(q∥pˉi(r−1))+(1−αi)DKL(q∥rai(r))J_i(q) = \alpha_i D_{\text{KL}}(q \parallel \bar{p}_i^{(r-1)}) + (1-\alpha_i) D_{\text{KL}}(q \parallel r_{a_i}^{(r)}) (Appendix A, Eqs. 9–11). Because DKL(⋅∥⋅)D_{\text{KL}}(\cdot \parallel \cdot) is strictly convex in qq, this subproblem admits a unique closed-form global optimum on the probability simplex, qi∗(y)∝[pˉi(r−1)(y)]αi[rai(r)(y)]1−αiq_i^{\ast}(y) \propto [\bar{p}_i^{(r-1)}(y)]^{\alpha_i} [r_{a_i}^{(r)}(y)]^{1-\alpha_i}, requiring no step-size tuning and admitting neither divergence nor oscillation.
  2. Confidence damping and short-horizon schedule: Cross-round refinement is not intended to be iterated to R→∞R \to \infty—under a static prior, infinite iteration would decay the cumulative weight of the initial prediction, ∏r=1Rαi(r)\prod_{r=1}^R \alpha_i^{(r)}, causing collapse to a cluster-prior-only regime (similar to the over-sharpening of Prior-only). Instead, each sample's update magnitude is scaled by (1−αi(r))(1 - \alpha_i^{(r)}): most confident samples (median initial αi(0)=0.921\alpha_i^{(0)} = 0.921 on OVCamo) satisfy (1−αi(r))≈0(1 - \alpha_i^{(r)}) \approx 0, anchoring the cluster distributions, while the remaining uncertain samples lose entropy after Round 1, increasing αi(r)\alpha_i^{(r)} and naturally damping update step sizes in later rounds.

Sensitivity Analysis of Core Hyperparameters (W1 & Q1)

All default hyperparameters in the paper were fixed a priori based on intuitive physical meaning without tuning on the test set. To assess sensitivity, we ran a full 200-run grid sweep on OVCamo covering 20 positive-smoothing offline settings (ϵ×λ\epsilon \times \lambda) and 36 online settings x 5 random seeds (τ×β×κ\tau \times \beta \times \kappa; sweep ranges in Table R2). Every offline configuration (20/20; Acc 71.0%–73.3%) and every online configuration (36/36; Acc 71.7%–73.0%) outperforms the base model by a clear margin, and the grid maxima (73.3% offline, 73.0% online) sit close to our preset defaults. As Table R2 shows, varying τ\tau and β\beta induces minimal variance, while combining entropy and margin (λ=0.5\lambda=0.5) outperforms either extreme (λ=0\lambda=0 or 11), supporting the soundness and robustness of the defaults.

Hyperparameter & ModeSweep RangeDefaultAcc Range(%)Max Span(pp)
Offline smoothing ϵ\epsilon{0.01,0.05,0.1,0.5}\lbrace 0.01,0.05,0.1,0.5\rbrace0.0572.4–72.90.58
Offline balance λ\lambda{0.0,0.25,0.5,0.75,1.0}\lbrace 0.0,0.25,0.5,0.75,1.0\rbrace0.571.5–73.31.72
Online gate τ\tau{0.75,0.80,0.85,0.90}\lbrace 0.75,0.80,0.85,0.90\rbrace0.8572.2–72.60.37
Online quality exp β\beta{1,2,3}\lbrace 1,2,3\rbrace272.3–72.30.04
Online temperature κ\kappa{0.05,0.07,0.10}\lbrace 0.05,0.07,0.10\rbrace0.0771.7–72.70.93
AuthorsOpenReview ↗

Response to Reviewer AwLh (Part 3/3)

Performance Under Skewed Distributions and Rare-Class Failure-Case Analysis (W1 & Q2)

First, regarding skewed distributions, the original OVCamo test set is already markedly long-tailed (Table R3). We further sampled more extreme label-prior-shifted subsets, including exponential decay IR100 and compound exponential decay Decay20, using 5 independent random seeds per setting with K-means run from scratch. As shown in Table R3, heavier skew produces larger Macro-F1 gains from CCB. Clustering operates purely in the frozen visual space decoupled from noisy predictions, and αi\alpha_i gating prevents error spread across clusters. Under severe skew, ambiguous samples from frequent classes are frequently misclassified by the base model into rare classes, generating many false positives; CCB pulls them back into their dense frequent-class clusters, sharply reducing false positives into rare classes while confident rare-class samples keep high αi\alpha_i weights and retain their predictions.

Test Subset DistributionNNIRGiniBase→CCB Acc(%)Δ\DeltaAcc(pp)Base→CCB F1(%)Δ\DeltaF1(pp)
Original OVCamo Test Set3,77065.40.53268.8→73.2+4.461.7→66.6+4.8
Exponential-Decay IR1001,3121000.60568.5→73.1+4.757.4→62.7+5.3
Compound Extreme Decay202,2553270.74368.0→74.9+7.048.6→60.0+11.4

Second, to inspect failure cases on rare classes, we partitioned the 61 OVCamo classes into Many (n≥100n \ge 100: 12 classes, 2,211 images), Medium (30≤n<10030 \le n < 100: 22 classes, 1,116 images), and Rare (n<30n < 30: 27 classes, 443 images). Macro-F1 rises substantially across all three frequency bands: Many increases from 74.0% to 75.4%, Medium from 66.5% to 72.5%, and Rare from 52.4% to 57.8% (+5.4 pp). Only Rare accuracy shows a minor −0.7 pp fluctuation (70.2%→69.5%), with 26 corrected vs. 29 harmed across all 443 rare images—nearly breaking even, in contrast to the +164 net gain on Medium and Many. Case-by-case inspection of these few errors reveals a clear cause: under equal-weight voting across clusters averaging ~62 images, an occasional rare-class sample whose initial top-1 prediction happens to be correct—but with a nearly flat probability distribution and very low confidence αi\alpha_i—can be diluted by equal-weight votes from visually similar frequent-class members when it falls into a mixed cluster. As discussed in Appendix H, introducing confidence-weighted voting within clusters can further mitigate this dilution on low-confidence boundary samples.

Clarifying the Risk of "Misaligned Clustering Encoders" (Q4)

We clarify a key architectural property of multi-view CCB (Offline): feature vectors from different encoders are never mixed or compared in the same metric space. In round rr, K-means clustering runs independently inside the single frozen encoder's own feature space Z(r)∈RN×drZ^{(r)} \in \mathbb{R}^{N \times d_r}—clustering solely on DINOv3 features in D rounds, ResNet features in R rounds, and CLIP features in C rounds. The only information passed between rounds is the KK-dimensional class probability distribution P(r)∈ΔK−1P^{(r)} \in \Delta^{K-1} over the shared class vocabulary C\mathcal{C}, avoiding semantic drift caused by over-clustering in a single non-language-aligned encoder.

Narrowing the Gap Between Streaming and Offline Modes (W1 & Q5)

Under causal zero-latency streaming, the model only observes arrived samples 1…t1 \dots t rather than all N=3,770N = 3,770 images simultaneously; even so, CCB (Streaming, PC-WOC) trails full-set CCB (Offline) by only 0.8 pp with CLIP-only embeddings (Tables 4, 7; 75.0% vs. 75.9% with DINO feature fusion). Moreover, zero-latency CCB (Streaming) already outperforms offline Batch 512 (71.3%) as well as the online baselines T3A and TDA.

To narrow this small gap further under tight memory and latency budgets, deployment can incorporate either a sliding-window buffer or micro-batch offline aggregation: adding a lightweight Ring Buffer (B=64B=64, occupying only 0.402 MB of state memory) delays final predictions by at most 64 arrivals so early samples query more mature class centroids, raising online accuracy from 72.4% to 72.6% on the exact same cached features; likewise, the micro-batch offline mode in Table 4 (Batch 1024, 72.5% Acc) provides a smooth midpoint between online streaming and full-dataset offline inference.

AuthorsOpenReview ↗

Polite Follow-up Before Discussion Deadline

Dear Reviewer AwLh,

Thank you again for your detailed review and valuable suggestions. With the author-reviewer discussion period closing tomorrow, we wanted to kindly check in to see if our response has adequately addressed your concerns.

We remain readily available to answer any follow-up questions, clarify any remaining points, or run additional checks before the deadline. We would greatly appreciate your feedback on our rebuttal.

Best regards, The Authors

Reviewer_nzknOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Cluster-Consensus Bayes (CCB), a lightweight test-time semantic refinement framework for open-vocabulary camouflaged object segmentation. CCB estimates cluster-level semantic priors from visually similar unlabeled images and adaptively combines them with individual predictions based on confidence. The framework further supports iterative multi-view clustering and online inference through confidence-weighted centroid memory. Experiments across multiple segmentation models and datasets demonstrate consistent improvements in category recognition and class-aware segmentation performance without gradient-based adaptation.

Critical strengths
  1. Interesting motivation and practical methodology. The paper explores cross-sample semantic consensus for improving open-vocabulary recognition, providing a useful alternative to conventional image-independent inference. The proposed framework is lightweight and can be integrated into existing models without gradient-based adaptation.

  2. Comprehensive evaluation and consistent improvements. CCB demonstrates consistent improvements across multiple segmentation architectures and datasets. The experiments cover offline refinement, online inference, different embedding representations, and several ablation studies, providing solid empirical support for the proposed framework.

Critical weaknesses
  1. Limited methodological novelty compared with existing test-time adaptation methods. CCB mainly combines visual clustering, pseudo-label-based semantic priors, and confidence-weighted prediction fusion, which are closely related to existing transductive inference and test-time adaptation techniques. Although the combination is effective, comparisons with representative methods such as T3A, TDA, and LAME would better establish its methodological novelty and advantages.

  2. Dependence on test-set composition and clustering quality. The effectiveness of CCB depends on the availability of visually related test samples and reliable cluster-level predictions. Table 4 shows that performance varies with collection size and streaming order, particularly under the hard-to-easy setting. Further analysis of class imbalance, mixed-category clusters, and distribution shifts would strengthen its practical applicability.

  3. Limited improvement in actual segmentation quality. CCB primarily refines category predictions without changing the predicted segmentation masks, as demonstrated in Table 5. Therefore, the improvements in class-aware segmentation metrics largely originate from more accurate semantic classification rather than better object localization. The authors should clarify this limitation and further discuss its implications for general open-vocabulary segmentation.

  4. Evaluation protocol and generalization to multi-object scenarios. The evaluation mainly focuses on single-target images, while the general open-vocabulary segmentation setting often involves multiple objects from different categories. Moreover, some evaluated segmentation models employ oracle mask selection. Additional experiments under realistic non-oracle and multi-object settings would strengthen the generality of CCB.

Questions for authors

Please see the weaknesses part.

4 comments
AuthorsOpenReview ↗

Response to Reviewer nzkn (Part 1/2)

We thank Reviewer nzkn for the constructive feedback and for recognizing our motivation on cross-sample semantic consensus and comprehensive evaluation. Below we address all four concerns (random-order streaming averaged over 5 seeds).

Comparison with T3A, TDA, and LAME, Orthogonal Combinations, and Distinct Contributions (W1)

We evaluated T3A, TDA, and LAME under the exact same frozen OVCoser base model and protocol (Table R1). Generic TTA methods hit clear bottlenecks on OV-COS:

  1. T3A assumes closed-set classes and bins samples by hard pseudo-labels to update prototypes, causing prototype contamination under open-vocabulary predictions.
  2. TDA uses a moderate-entropy negative cache that mis-suppresses visually similar camouflaged classes, and its instance-level key-value retrieval uses a fixed residual weight ignoring target sample uncertainty.
  3. LAME targets small batches via unnormalized pairwise graph Laplacian smoothing. Over the full dataset with its default dense RBF kernel, accumulated node degrees overwhelm unary logits and collapse accuracy to 4.8%. For a fair comparison, Table R1 therefore reports LAME with RBF at B=16B=16 and with a sparse kNN5 kernel on the full set.

By contrast, CCB combines three complementary designs for OV-COS. First, CCB decouples visual clustering from class assignment: it clusters in frozen visual space and computes normalized soft class distributions per cluster rather than single-class clusters, avoiding both T3A's hard-label contamination and LAME's degree explosion. Second, instead of TDA's fixed residual weights and negative cache, CCB modulates the cluster prior weight per sample by its prediction confidence, preserving confident predictions while correcting ambiguous ones. Third, CCB unifies full-set offline transductive and online streaming inference within one training-free Bayesian formulation across batch sizes, stream orderings, and encoders.

Under this unified evaluation (Table R1), CCB (Streaming) outperforms T3A and TDA across accuracy, segmentation quality, and memory, while CCB (Offline, CCC) surpasses LAME-kNN5 in accuracy and Macro-F1 while running 4.5x faster with 78.8% less peak memory, and cross-encoder multi-view refinement widens this margin further.

More importantly, because CCB operates directly on output probability distributions, it stacks orthogonally with existing TTA methods in the same paradigm: adding online CCB after TDA raises accuracy from 69.6% to 72.4% (66.7% Macro-F1), and adding offline CCB after LAME-kNN5 raises accuracy to 73.9%. These consistent orthogonal gains confirm that CCB does not duplicate existing TTA mechanisms, but delivers distinct and irreplaceable incremental value.

ParadigmMethodAcc(%)↑Macro-F1(%)↑cSmcS_m↑cFβωcF_\beta^\omega↑cMAE↓cFβcF_\beta↑cEmcE_m↑cIoU↑CPU(ms/img)RSS(MB)
BaselineBase OVCoser68.861.70.5790.4900.3360.5200.6160.443——
StreamingT3A65.358.30.5510.4680.3690.4960.5860.4230.14034.65
TDA69.662.80.5850.4940.3270.5240.6230.4460.66735.01
CCB (Streaming)72.465.50.6050.5060.3010.5370.6430.4570.1943.80
OfflineLAME (RBF, B=16)69.061.80.5810.4910.3340.5210.6180.4440.09035.60
LAME (kNN5, Full)72.765.30.6100.5150.2990.5460.6490.4650.829204.63
CCB (Offline, CCC)73.266.60.6110.5130.2940.5450.6510.4620.18243.45

Mixed-Category Clusters (W2, Part 1)

CCB does not rely on pure clusters. Base OVCoser has a 31.2% initial error rate on OVCamo, and Round-1 K-means purity is 0.723 (0.728 in Round 3; Section 4.5, Appendix Table 17). Hard-replacing predictions with mixed-cluster majorities (Prior-only) spreads errors and drops accuracy from 68.8% to 62.6% in Round 1 and 50.1% in Round 3 (Table 4, Appendix Table 16). Instead, CCB retains the full soft within-cluster vote distribution rj∈ΔK−1r_j \in \Delta^{K-1} via Eq. (1) to preserve competing hypotheses, and fuses it via the log-odds gate in Eq. (5): confident samples retain their individual predictions without cluster-noise corruption, while uncertain samples draw on neighborhood consensus.

AuthorsOpenReview ↗

Response to Reviewer nzkn (Part 2/2)

Class Imbalance, Distribution Shift, Collection Size, and Streaming Order (W2, Part 2)

For label-prior shift, OVCamo is already long-tailed; we further resampled exponential-decay IR100 and compound extreme-decay Decay20 across 5 random seeds with K-means run from scratch (Table R2). CCB achieves large Accuracy and Macro-F1 gains under extreme skew. Visual clustering is frozen and decoupled from noisy predictions: ambiguous frequent-class samples misclassified as rare classes are pulled back into dense majority clusters, cutting rare-class false positives (explaining larger Macro-F1 gains under stronger skew), while confident rare-class samples keep high αi\alpha_i and avoid majority override. For visual-domain/vocabulary shift, zero-retraining transfer on COCO-S (Table 2: 27.6%→33.0%) confirms effectiveness when both domain and vocabulary change completely.

Test Subset DistributionNNIRGiniBase→CCB Acc(%)Δ\DeltaAcc(pp)Base→CCB F1(%)Δ\DeltaF1(pp)
Original OVCamo Test Set3,77065.40.53268.8→73.2+4.461.7→66.6+4.8
Exponential-Decay IR1001,3121000.60568.5→73.1+4.757.4→62.7+5.3
Compound Extreme Decay202,2553270.74368.0→74.9+7.048.6→60.0+11.4

On collection size and streaming order, offline CCB still yields clear gains on smaller collections (Table 4, Appendix Table 17: Batch 512 71.3%, Batch 1024 72.5% vs. Base 68.8%), with Round-3 NMI rising monotonically with size (0.628→0.659→0.764). Without pre-aggregation, streaming PC-WOC reaches 72.4% in constant memory. Under adversarial Hard-to-Easy order, the first 1,600 images (42.4%) hold 84.6% of base errors and all satisfy αt≤τ=0.85\alpha_t \le \tau=0.85, so the gate excludes all of them from memory; from image 1,601 onward, all have αt>0.85\alpha_t > 0.85, yielding sample-for-sample identical predictions to the base model with zero harm. On a three-stage Hard-Easy-Hard stream (Table R3, 5 seeds), once Stage B populates centroids, Stage C zero-latency accuracy jumps immediately, whereas τ=0\tau=0 degrades recovery due to early contamination.

Gate SettingStage A (800 Hard)Stage B (2,170 Easy)Stage C (800 Hard)Full (3,770)Stage C Corrected/Harmed
Base OVCoser38.791.737.168.80/0
CCB (Streaming, τ=0.85\tau=0.85)38.791.747.4 (+10.3 pp)71.099.2/17.0
CCB (Streaming, τ=0\tau=0)39.991.745.2 (+8.1 pp)70.880.6/15.8

Semantic Classification vs. Segmentation Mask Refinement (W3)

We agree and will expand this discussion. As shown in Table 5, CCB is a test-time semantic refinement module optimizing category probabilities. In OV-COS, class-agnostic localization is already relatively strong (OVCoser reaches 0.583 agIoU vs. 0.443 cIoU), so unseen-class confusion from camouflage textures is the main bottleneck; resolving semantics directly recovers well-localized masks. Moreover, proposal-aggregation OVSS baselines (e.g., SAN, OVSeg, FC-CLIP) compute Msem(h,w,c)=∑mPm(c)Mm(h,w)M_{\text{sem}}(h,w,c)=\sum_m P_m(c)M_m(h,w), so refining Pm(c)P_m(c) directly alters activated spatial regions and mask category assignments. Still, CCB does not modify mask spatial boundaries; in general OVSS its upper bound depends on localization quality (agIoU), and it can therefore be paired with spatial adaptation such as TENT (Table 5).

Single-Target Protocol, Non-Oracle Mask Selection, and Multi-Object Generalization (W4)

OV-COS benchmarks adopt a single-foreground protocol. Dedicated OV-COS models in Table 1 (OVCoser, CVLM) output one mask directly and never use oracle selection. Oracle selection in Table 1 applies only to multi-region OVSS baselines (e.g., SAN, CAT-Seg), strictly following OVCoser (Pang et al., ECCV 2024) to adapt them to single-target evaluation—favoring those baselines. Replacing GT-Oracle on SAN and CAT-Seg with a label-free highest-confidence rule (holding the selected mask fixed across Base, +CCB, and +PC-WOC) sharply reduces baseline agIoU and accuracy (Table R4), yet offline +CCB and online +PC-WOC still yield consistent gains.

Base ModelMask SelectionGT?agIoUBase Acc(%)+CCB (Offline)(%)+PC-WOC (Streaming)(%)
OVCoser——0.58368.873.2 (+4.4)72.4 (+3.6)
SANHighest IoUYes0.38166.669.3 (+2.7)68.8 (+2.2)
SANHighest Conf.No0.35065.2 (−1.4)66.8 (+1.6)66.7 (+1.4)
CAT-SegHighest IoUYes0.37470.373.3 (+3.0)70.9 (+0.5)
CAT-SegHighest Conf.No0.30861.5 (−8.8)63.1 (+1.6)61.7 (+0.2)

Finally, while VOC-S and COCO-S include multi-object clutter and verify transfer to general categories, our current formulation uses image-level features suited to single-dominant-object scenes. Extending to scenes with multiple co-occurring distinct classes requires region-level mask-pooled features and background filtering, which we will discuss as a limitation and future direction.

I appreciate the authors’ constructive responses, which have largely addressed my concerns, and I therefore maintain my positive rating. However, it is essential that all clarifications and revisions discussed above be incorporated into the revised manuscript.

We sincerely thank the reviewer for their constructive feedback and positive assessment of our rebuttal. We will carefully incorporate the discussed clarifications and revisions into the revised manuscript.

Reviewer_U6VHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Cluster-Consensus Bayes (CCB), a lightweight, plug-and-play framework for open-vocabulary camouflaged object segmentation that exploits semantic consensus among visually similar test samples. The method clusters unlabeled test images to estimate class-level priors and adaptively combines these priors with individual vision-language predictions based on prediction confidence. It further introduces iterative multi-view refinement for offline inference and a confidence-weighted centroid memory for online inference, both without gradient-based adaptation. Experiments across five base models on OVCamo, together with evaluations on VOC and COCO subsets, demonstrate consistent improvements in recognition accuracy and class-aware segmentation performance. Additional ablation studies validate the effectiveness of the proposed consensus fusion, multi-view refinement, and streaming inference mechanisms.

Critical strengths
  1. The paper presents a well-motivated approach to improving open-vocabulary recognition by exploiting semantic consensus across visually related test samples. The confidence-adaptive fusion mechanism effectively balances individual predictions with cluster-level evidence, allowing uncertain samples to benefit from shared information while preserving confident predictions. The method is lightweight, requires no gradient updates, and can be integrated into existing segmentation frameworks with minimal modifications.

  2. The proposed method demonstrates consistent improvements across five different base models, including both open-vocabulary semantic segmentation and camouflaged object segmentation approaches. The evaluation on additional VOC and COCO subsets further supports its applicability beyond the primary benchmark. The paper also provides extensive ablation studies investigating fusion strategies, clustering configurations, multi-view refinement, and online inference, which strengthen the empirical support for the proposed approach.

Critical weaknesses
  1. Limited methodological novelty relative to existing test-time adaptation approaches. The proposed framework builds on established ideas such as clustering, pseudo-label aggregation, and confidence-weighted prediction refinement. Although their combination is effective, the underlying components are relatively straightforward. The paper discusses related approaches such as T3A, TDA, and LAME, but does not provide direct empirical comparisons with these methods under a unified evaluation setting. Such comparisons would help clarify the distinct contribution of CCB beyond existing test-time semantic adaptation techniques.

  2. Dependence on the distribution and composition of test samples. CCB relies on the assumption that visually similar test images provide reliable semantic evidence for one another. However, the effectiveness of this assumption may vary with class imbalance, noisy predictions, and changes in the test distribution. Although the paper investigates different batch sizes and streaming input orders, the results show that performance can be sensitive to the available context. For example, the online accuracy drops from 72.4% under random ordering to 68.8% under hard-to-easy ordering. Further evaluation under imbalanced or distribution-shifted test collections would strengthen the claims regarding robustness and practical generalization.

Questions for authors
  1. Sensitivity to streaming input order. Table 4 shows that PC-WOC achieves 72.4% accuracy under random ordering and 73.6% under class-grouped ordering, but drops to 68.8% under hard-to-easy ordering, eliminating the improvement over the original OVCoser baseline (68.8%). Since Eq. (7) updates the class centroids using confident predictions without an explicit mechanism for correcting erroneous assignments, could the authors report recognition accuracy at different stages of the stream under random and hard-to-easy ordering? In particular, does performance recover as reliable samples become available, or do early memory errors continue to affect subsequent predictions?

  2. Selection of multi-view refinement sequences. Table 3 shows that, under the same DINO-fusion baseline, the D-D-C refinement sequence achieves 75.9% accuracy, compared with 73.5% for D-D-D and 73.7% for C-C-C. Furthermore, Appendix E.1 reports that the preferred sequence varies across auxiliary encoders. How are these sequences selected without using test-set labels? Could the authors evaluate a fixed sequence selected on a separate validation set and report whether its advantage over single-view refinement remains consistent across different datasets?

  3. Contribution of confidence-adaptive fusion. Eq. (3) defines the fusion coefficient using normalized prediction entropy and the top-two probability margin, while Table 4 shows that the proposed adaptive fusion achieves 73.2% accuracy compared with 72.0% using fixed confidence. Could the authors clarify how the fixed-confidence baseline is configured and provide a comparison against a fixed coefficient tuned on a separate validation set? This would help determine whether the 1.2-point gain arises from sample-specific confidence adaptation rather than the choice of fusion weight.

3 comments
AuthorsOpenReview ↗

Response to Reviewer U6VH (Part 1/2)

We thank Reviewer U6VH for the positive evaluation and for recognizing our motivation on cross-sample semantic consensus, confidence-adaptive fusion, and broad evaluation across five base models and three benchmarks. Below we address each concern and question (streaming results averaged over 5 seeds).

Comparison with T3A, TDA, and LAME, Orthogonal Combinations, and Distinct Contributions (W1)

We evaluated T3A, TDA, and LAME under the exact same frozen OVCoser base model and protocol (Table R1). Generic TTA methods hit clear bottlenecks on OV-COS:

  1. T3A assumes closed-set classes and bins samples by hard pseudo-labels to update prototypes, causing prototype contamination under open-vocabulary predictions.
  2. TDA uses a moderate-entropy negative cache that mis-suppresses visually similar camouflaged classes, and its instance-level key-value retrieval uses a fixed residual weight ignoring target sample uncertainty.
  3. LAME targets small batches via unnormalized pairwise graph Laplacian smoothing. Over the full dataset with its default dense RBF kernel, accumulated node degrees overwhelm unary logits and collapse accuracy to 4.8%. For a fair comparison, Table R1 therefore reports LAME with RBF at B=16B=16 and with a sparse kNN5 kernel on the full set.

By contrast, CCB combines three complementary designs for OV-COS. First, CCB decouples visual clustering from class assignment: it clusters in frozen visual space and computes normalized soft class distributions per cluster rather than single-class clusters, avoiding both T3A's hard-label contamination and LAME's degree explosion. Second, instead of TDA's fixed residual weights and negative cache, CCB modulates the cluster prior weight per sample by its prediction confidence, preserving confident predictions while correcting ambiguous ones. Third, CCB unifies full-set offline transductive and online streaming inference within one training-free Bayesian formulation across batch sizes, stream orderings, and encoders.

Under this unified evaluation (Table R1), CCB (Streaming) outperforms T3A and TDA across accuracy, segmentation quality, and memory, while CCB (Offline, CCC) surpasses LAME-kNN5 in accuracy and Macro-F1 while running 4.5x faster with 78.8% less peak memory, and cross-encoder multi-view refinement widens this margin further.

More importantly, because CCB operates directly on output probability distributions, it stacks orthogonally with existing TTA methods in the same paradigm: adding online CCB after TDA raises accuracy from 69.6% to 72.4% (66.7% Macro-F1), and adding offline CCB after LAME-kNN5 raises accuracy to 73.9%. These consistent orthogonal gains confirm that CCB does not duplicate existing TTA mechanisms, but delivers distinct and irreplaceable incremental value.

ParadigmMethodAcc(%)↑Macro-F1(%)↑cSmcS_m↑cFβωcF_\beta^\omega↑cMAE↓cFβcF_\beta↑cEmcE_m↑cIoU↑CPU(ms/img)RSS(MB)
BaselineBase OVCoser68.861.70.5790.4900.3360.5200.6160.443——
StreamingT3A65.358.30.5510.4680.3690.4960.5860.4230.14034.65
TDA69.662.80.5850.4940.3270.5240.6230.4460.66735.01
CCB (Streaming)72.465.50.6050.5060.3010.5370.6430.4570.1943.80
OfflineLAME (RBF, B=16)69.061.80.5810.4910.3340.5210.6180.4440.09035.60
LAME (kNN5, Full)72.765.30.6100.5150.2990.5460.6490.4650.829204.63
CCB (Offline, CCC)73.266.60.6110.5130.2940.5450.6510.4620.18243.45

Robustness to Prediction Noise (W2, Part 1)

CCB does not assume clean base-model outputs or pure visual clusters. Base OVCoser has a 31.2% initial error rate on OVCamo, and Round-1 clustering purity is 0.723 (Section 4.5). Hard-replacing each sample's prediction with its cluster majority (Prior-only) propagates errors across the cluster, dropping accuracy from 68.8% to 62.6% in Round 1 and 50.1% in Round 3 (Table 4, Appendix Table 16). Moreover, on COCO-S (Table 2), even when base accuracy is 27.6%, CCB improves it to 33.0%.

AuthorsOpenReview ↗

Response to Reviewer U6VH (Part 2/2)

Robustness to Class Imbalance and Distribution Shift (W2, Part 2)

The original OVCamo test set is already markedly long-tailed (Table R2). To test harsher label-prior shifts, we resampled two more skewed subsets: exponentially decaying IR100 and extreme long-tailed Decay20 (exponential decay applied atop the original long-tailed set), running 5 random subsets and K-means from scratch per setting. Table R2 shows substantial gains in Accuracy and Macro-F1, confirming frequent-class priors do not overwhelm rare classes under severe skew.

Mechanistically, clustering runs strictly in frozen visual space decoupled from noisy predictions, so misclassifications do not corrupt feature neighborhoods and confidence gating prevents error spread unlike Prior-only. Under heavy imbalance, ambiguous frequent-class samples are often misclassified into rare classes, depressing Macro-F1; CCB pulls them back into dense majority clusters, cutting rare-class false positives (explaining larger Macro-F1 gains under stronger skew). Meanwhile, confident rare-class samples retain high αi\alpha_i weights and keep their original predictions.

Test Subset DistributionNNIRGiniBase→CCB Acc(%)Δ\DeltaAcc(pp)Base→CCB F1(%)Δ\DeltaF1(pp)
Original OVCamo Test Set3,77065.40.53268.8→73.2+4.461.7→66.6+4.8
Exponential-Decay IR1001,3121000.60568.5→73.1+4.757.4→62.7+5.3
Compound Extreme Decay202,2553270.74368.0→74.9+7.048.6→60.0+11.4

Stage-Wise Streaming Accuracy, Zero Early Pollution, and Recovery (Q1 & W2)

To examine arrival-order effects and early pollution, we split the test stream into 5 equal stages comparing Random vs. Hard-to-Easy ordering (Table R3). Under Hard-to-Easy, the first 1,600 images (42.4% of the stream) account for 84.6% of all base errors and all satisfy αt≤τ=0.85\alpha_t \le \tau = 0.85. The gate excludes this entire low-confidence prefix from centroid memory, while every sample from image 1,601 onward has αt>0.85\alpha_t > 0.85 (individual confidence dominates). Thus predictions match the base model sample-for-sample across the stream rather than canceling early errors with late corrections.

Stage (754 imgs each)Random: Base→CCB(%)Random Δ\Delta(pp)Hard-to-Easy: Base→CCB(%)H2E Δ\Delta(pp)
Stage 1 (0–20%)70.6→72.2+1.623.5→23.50.0
Stage 2 (20–40%)69.4→72.6+3.248.8→48.80.0
Stage 3 (40–60%)68.3→72.5+4.278.5→78.50.0
Stage 4 (60–80%)68.2→72.9+4.895.1→95.10.0
Stage 5 (80–100%)67.6→71.7+4.198.1→98.10.0
Full Stream (3,770)68.8→72.4+3.668.8→68.80.0

To test recovery on later hard samples, we constructed a three-stage Hard-Easy-Hard stream (Table R4), where Stage B populates all 61 class centroids. Once Stage B establishes centroid memory, zero-latency accuracy on Stage C hard samples jumps immediately, whereas removing the gate (τ=0\tau=0) weakens recovery due to early contamination (averaged over 5 split/order seeds).

Gate SettingStage A (800 Hard)Stage B (2,170 Easy)Stage C (800 Hard)Full (3,770)Stage C Corrected/Harmed
Base OVCoser38.791.737.168.80/0
CCB (Streaming, τ=0.85\tau=0.85)38.791.747.4 (+10.3 pp)71.099.2/17.0
CCB (Streaming, τ=0\tau=0)39.991.745.2 (+8.1 pp)70.880.6/15.8

Label-Free Multi-View Sequence Selection and Cross-Dataset Consistency (Q2)

Our default method (Tables 1–2) uses only a single CLIP view (CCC) without sequence selection; multi-view clustering is an optional extension for auxiliary encoders (Table 3). Because neighborhood structure varies across encoders and datasets, optimal sequences depend on both. Without test labels, we determine round count via unsupervised inter-round prediction agreement, and select the view sequence via k-NN Consistency (agreement between final predictions and initial neighbor-voting in dual-view space).

Even without sequence selection, simply transferring the fixed hybrid sequence DDC from OVCamo (Table 3) to Table 2 benchmarks with zero tuning (frozen DINOv3+CLIP) still preserves its advantage consistently: on VOC-S, DDC reaches 84.4%, outperforming CCC (83.7%; Base 79.6%) and DDD (80.4%); on COCO-S, DDC reaches 35.2%, outperforming CCC (33.0%; Base 27.6%) and DDD (32.9%).

Clarification on Fixed-Confidence Baseline and Validation-Tuned α∗\alpha^* (Q3)

In Table 4, Fixed confidence sets αi≡0.5\alpha_i \equiv 0.5 in Eq. (2) as a neutral equal-weight baseline. Using a separate calibration set of 896 images from the 14 seen training classes (no unseen test labels), sweeping fixed α\alpha yielded α∗=0.7\alpha^* = 0.7, which also turned out to be the oracle-optimal fixed weight on the test set (72.6% Acc). Our tuning-free sample-adaptive αi\alpha_i (73.2% Acc, Table 4) outperforms this tuned static optimum, confirming gains stem from per-sample confidence adaptation.

AuthorsOpenReview ↗

Polite Follow-up Before Discussion Deadline

Dear Reviewer U6VH,

Thank you again for your time and thoughtful feedback on our submission. As the author-reviewer discussion period is drawing to a close tomorrow, we wanted to politely follow up and check whether our response has addressed all of your questions (W1–W2 and Q1–Q3).

We are standing by and would be very glad to provide any further clarifications or additional details if anything remains unclear. If your concerns have been resolved, we would be grateful if you could consider updating your evaluation accordingly.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Outcomes and Revision Plan

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and Reviewers U6VH, nzkn, and AwLh for the constructive review and discussion process. All reviewers recognized CCB's clear motivation, lightweight training-free design, and consistent gains across five models and three benchmarks. Following our rebuttal and discussion, Reviewer AwLh raised their score from Weak Rejection (2) to Weak Acceptance (3), citing "the solid execution and thoroughness of the work." Together with U6VH (3) and nzkn (3), all reviewers now recommend acceptance (3 / 3 / 3). Below we summarize the main outcomes of the discussion and our revision plan.

1. Distinctness, TTA Baselines & Compute Overhead (U6VH/nzkn W1; AwLh W1, Q1)

  • Mechanism: Unlike TENT/TPT, CCB updates no weights or prompts. Unlike T3A (hard pseudo-label prototypes), TDA (fixed-weight cache + negative suppression), and LAME (unnormalized pairwise smoothing), CCB is a model-agnostic, output-space empirical-Bayes framework decoupling frozen visual grouping (supporting external encoders without feature alignment) from soft cluster priors and per-sample confidence gating across offline and streaming modes.
  • Unified OVCamo benchmark & efficiency (OVCoser base: 68.8% Acc): Streaming PC-WOC (72.4%) outperforms T3A and TDA at sub-millisecond CPU latency (<0.2 ms/img) and minimal memory. Offline CLIP-only CCB (73.2%) beats sparse LAME-kNN5 in Acc/F1 while running 4.5× faster with 78.8% less memory (whereas dense LAME-RBF collapses to 4.8%), and outperforms gradient-based TENT and TPT; multi-view DDC reaches 75.9%.
  • Orthogonal gains: Stacking CCB consistently improves every baseline—TDA+CCB, LAME+CCB, and TENT+CCB (+1.2 to +3.8 pp Acc)—confirming distinct value.

2. Noise, Class Imbalance & Rare Classes (U6VH/nzkn W2; AwLh Q2)

  • Noise & skew: Soft priors with confidence gating tolerate noisy base predictions and impure clusters (whereas hard Prior-only collapses) and generalize to low-base regimes (COCO-S). On 5-seed skewed subsets, gains widen with imbalance—reaching up to +7.0 pp Acc / +11.4 pp Macro-F1 under severe decay.
  • Rare classes: Macro-F1 improves across Many/Medium/Rare splits (Rare +5.4 pp). Rare Acc dips slightly (−0.7 pp) due to equal-vote dilution on low-αi\alpha_i samples, which can be mitigated via confidence-weighted voting (to be added in revision).

3. Streaming Order, Recovery & Online–Offline Gap (U6VH W2, Q1; nzkn W2; AwLh Q5)

  • Zero early contamination: Random-order gains rise from +1.6 pp (Stage 1) to +4.1–4.8 pp in later stages. Under Hard-to-Easy, the confidence gate (τ=0.85\tau=0.85) filters out the low-confidence prefix, yielding zero early contamination and exact Base parity (68.8%).
  • Recovery & gap: On Hard-Easy-Hard, once reliable centroids activate, subsequent hard-stage accuracy recovers sharply (+10.3 pp). To narrow the 0.8 pp online–offline gap (72.4% vs. 73.2%), a lightweight Ring Buffer (B=64B=64, 0.4 MB) reaches 72.6%, matching Batch-1024.

4. Convergence, Sensitivity, Adaptive αi\alpha_i & Multi-View Selection (U6VH Q2–3; AwLh Q1, Q3–4)

  • Convergence & cross-encoder safety: Each round pairs finite-step K-means with strictly convex weighted-KL fusion (unique optimum, Appendix A), stabilizing rapidly by Round 3 (93.4% label agreement). External encoder features are never mixed; only KK-dim class distributions transfer.
  • Sensitivity & adaptive αi\alpha_i: Across 200 hyperparameter sweep runs, 100% beat Base. Adaptive αi\alpha_i (73.2%) outperforms both Table 4's fixed α=0.5\alpha=0.5 (72.0%) and the calibration-tuned fixed optimum α∗=0.7\alpha^*=0.7 (72.6%).
  • Label-free multi-view selection: Default CCB is CLIP-only (CCC). Unsupervised agreement and k-NN Consistency guide optional multi-view refinement; transferring fixed DDC without tuning consistently beats CCC/DDD on VOC-S and COCO-S.

5. Non-Oracle Protocol, Mask Refinement & Scope (nzkn W3, W4)

  • Non-Oracle evaluation: OVCoser/CVLM never use oracle masks; Table 1's oracle rule (Pang et al., 2024) applied only to multi-mask OVSS baselines. Under label-free highest-confidence selection, +CCB and +PC-WOC retain consistent gains on both SAN and CAT-Seg.
  • Mask quality & scope: Semantic consensus targets OV-COS's dominant error (agIoU 0.583 vs. cIoU 0.443), directly alters activated regions in proposal-aggregation OVSS, and pairs orthogonally with spatial TTA (TENT+CCB).

Revision Plan

All new baselines, compute profiles, imbalance/streaming diagnostics, hyperparameter sweeps, Non-Oracle results, convergence analyses, and expanded region-level multi-object limitations will be incorporated into the revised manuscript.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.