- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 3(+1)
- Presentation
- 3(+1)
- Soundness
- 2
Read review
Paper summary
This paper introduces a lightweight, plug-and-play module called “Clustering Consensus Bayesian” (CCB) for open-vocabulary object segmentation with camouflage (OV-COS). CCB uses a frozen encoder to group visually similar unlabeled test images. It combines a Dirichlet-smoothed Top-1 voting-based class prior with individual predictions through adaptive fusion using a confidence-weighted geometric mean based on predicted entropy. CCB runs entirely during the testing phase without requiring gradient updates. It supports both offline batch processing and streaming processing modes (PC-WOC) and demonstrates higher segmentation accuracy across multiple benchmarks and five different base models.
Critical strengths
This method is driven by practical and intuitive motivations and ingeniously leverages the shared semantic context in related test images to resolve ambiguous objects. Its plug-and-play design requires no model retraining or architectural modifications, making it widely applicable. This paper also presents a comprehensive empirical evaluation across various models and introduces two complementary inference modes; notably, the streaming mode maintains constant memory usage, which offers significant advantages for real-world deployment.
Critical weaknesses
The most critical flaw of this paper is its lack of fundamental innovation. Its core concepts bear a striking resemblance to well-established techniques such as test-time adaptation, transduction reasoning, and self-training, yet the authors fail to rigorously distinguish their work from these existing approaches. Furthermore, the performance gains are too modest to justify the added complexity, and the paper severely lacks fair comparisons with key baseline models such as TENT and TPT. The theoretical contributions are superficial (e.g., the absence of convergence proofs), the performance of the streaming mode lags far behind that of the offline variants, and the extensive yet redundant ablation experiments fail to explore critical failure modes or hyperparameter sensitivities.
Questions for authors
The authors are required to conduct a comprehensive comparison with a correctly implemented runtime adaptive baseline and provide a detailed analysis of computational overhead (number of floating-point operations, memory usage, and actual runtime) for both modes. In addition, the reviewers requested that the authors justify the reasonableness of the core hyperparameters through sensitivity analysis, examine failure cases (such as rare classes or skewed distributions), provide theoretical convergence guarantees for offline optimization, analyze the risk of noise introduced by misaligned clustering encoders, and propose solutions to narrow the performance gap between the streaming and offline modes.
4 comments
Response to Reviewer AwLh (Part 2/3)
Well-Posedness and Cross-Round Convergence of Offline Optimization (W1 & Q3)
CCB (Offline) runs a short-horizon alternating loop in which each round consists of two deterministic steps: K-means clustering in feature space and variational fusion on the probability simplex:
- Per-round convergence and unique solution: Within each round, K-means monotonically decreases within-cluster sum-of-squared errors over a finite space of sample partitions and is guaranteed to terminate in finite steps. Given the resulting cluster distributions and sample confidence weights , the variational fusion step minimizes the weighted KL objective (Appendix A, Eqs. 9–11). Because is strictly convex in , this subproblem admits a unique closed-form global optimum on the probability simplex, , requiring no step-size tuning and admitting neither divergence nor oscillation.
- Confidence damping and short-horizon schedule: Cross-round refinement is not intended to be iterated to —under a static prior, infinite iteration would decay the cumulative weight of the initial prediction, , causing collapse to a cluster-prior-only regime (similar to the over-sharpening of Prior-only). Instead, each sample's update magnitude is scaled by : most confident samples (median initial on OVCamo) satisfy , anchoring the cluster distributions, while the remaining uncertain samples lose entropy after Round 1, increasing and naturally damping update step sizes in later rounds.
Sensitivity Analysis of Core Hyperparameters (W1 & Q1)
All default hyperparameters in the paper were fixed a priori based on intuitive physical meaning without tuning on the test set. To assess sensitivity, we ran a full 200-run grid sweep on OVCamo covering 20 positive-smoothing offline settings () and 36 online settings x 5 random seeds (; sweep ranges in Table R2). Every offline configuration (20/20; Acc 71.0%–73.3%) and every online configuration (36/36; Acc 71.7%–73.0%) outperforms the base model by a clear margin, and the grid maxima (73.3% offline, 73.0% online) sit close to our preset defaults. As Table R2 shows, varying and induces minimal variance, while combining entropy and margin () outperforms either extreme ( or ), supporting the soundness and robustness of the defaults.
| Hyperparameter & Mode | Sweep Range | Default | Acc Range(%) | Max Span(pp) |
|---|---|---|---|---|
| Offline smoothing | 0.05 | 72.4–72.9 | 0.58 | |
| Offline balance | 0.5 | 71.5–73.3 | 1.72 | |
| Online gate | 0.85 | 72.2–72.6 | 0.37 | |
| Online quality exp | 2 | 72.3–72.3 | 0.04 | |
| Online temperature | 0.07 | 71.7–72.7 | 0.93 |
Response to Reviewer AwLh (Part 3/3)
Performance Under Skewed Distributions and Rare-Class Failure-Case Analysis (W1 & Q2)
First, regarding skewed distributions, the original OVCamo test set is already markedly long-tailed (Table R3). We further sampled more extreme label-prior-shifted subsets, including exponential decay IR100 and compound exponential decay Decay20, using 5 independent random seeds per setting with K-means run from scratch. As shown in Table R3, heavier skew produces larger Macro-F1 gains from CCB. Clustering operates purely in the frozen visual space decoupled from noisy predictions, and gating prevents error spread across clusters. Under severe skew, ambiguous samples from frequent classes are frequently misclassified by the base model into rare classes, generating many false positives; CCB pulls them back into their dense frequent-class clusters, sharply reducing false positives into rare classes while confident rare-class samples keep high weights and retain their predictions.
| Test Subset Distribution | IR | Gini | Base→CCB Acc(%) | Acc(pp) | Base→CCB F1(%) | F1(pp) | |
|---|---|---|---|---|---|---|---|
| Original OVCamo Test Set | 3,770 | 65.4 | 0.532 | 68.8→73.2 | +4.4 | 61.7→66.6 | +4.8 |
| Exponential-Decay IR100 | 1,312 | 100 | 0.605 | 68.5→73.1 | +4.7 | 57.4→62.7 | +5.3 |
| Compound Extreme Decay20 | 2,255 | 327 | 0.743 | 68.0→74.9 | +7.0 | 48.6→60.0 | +11.4 |
Second, to inspect failure cases on rare classes, we partitioned the 61 OVCamo classes into Many (: 12 classes, 2,211 images), Medium (: 22 classes, 1,116 images), and Rare (: 27 classes, 443 images). Macro-F1 rises substantially across all three frequency bands: Many increases from 74.0% to 75.4%, Medium from 66.5% to 72.5%, and Rare from 52.4% to 57.8% (+5.4 pp). Only Rare accuracy shows a minor −0.7 pp fluctuation (70.2%→69.5%), with 26 corrected vs. 29 harmed across all 443 rare images—nearly breaking even, in contrast to the +164 net gain on Medium and Many. Case-by-case inspection of these few errors reveals a clear cause: under equal-weight voting across clusters averaging ~62 images, an occasional rare-class sample whose initial top-1 prediction happens to be correct—but with a nearly flat probability distribution and very low confidence —can be diluted by equal-weight votes from visually similar frequent-class members when it falls into a mixed cluster. As discussed in Appendix H, introducing confidence-weighted voting within clusters can further mitigate this dilution on low-confidence boundary samples.
Clarifying the Risk of "Misaligned Clustering Encoders" (Q4)
We clarify a key architectural property of multi-view CCB (Offline): feature vectors from different encoders are never mixed or compared in the same metric space. In round , K-means clustering runs independently inside the single frozen encoder's own feature space —clustering solely on DINOv3 features in D rounds, ResNet features in R rounds, and CLIP features in C rounds. The only information passed between rounds is the -dimensional class probability distribution over the shared class vocabulary , avoiding semantic drift caused by over-clustering in a single non-language-aligned encoder.
Narrowing the Gap Between Streaming and Offline Modes (W1 & Q5)
Under causal zero-latency streaming, the model only observes arrived samples rather than all images simultaneously; even so, CCB (Streaming, PC-WOC) trails full-set CCB (Offline) by only 0.8 pp with CLIP-only embeddings (Tables 4, 7; 75.0% vs. 75.9% with DINO feature fusion). Moreover, zero-latency CCB (Streaming) already outperforms offline Batch 512 (71.3%) as well as the online baselines T3A and TDA.
To narrow this small gap further under tight memory and latency budgets, deployment can incorporate either a sliding-window buffer or micro-batch offline aggregation: adding a lightweight Ring Buffer (, occupying only 0.402 MB of state memory) delays final predictions by at most 64 arrivals so early samples query more mature class centroids, raising online accuracy from 72.4% to 72.6% on the exact same cached features; likewise, the micro-batch offline mode in Table 4 (Batch 1024, 72.5% Acc) provides a smooth midpoint between online streaming and full-dataset offline inference.
Polite Follow-up Before Discussion Deadline
Dear Reviewer AwLh,
Thank you again for your detailed review and valuable suggestions. With the author-reviewer discussion period closing tomorrow, we wanted to kindly check in to see if our response has adequately addressed your concerns.
We remain readily available to answer any follow-up questions, clarify any remaining points, or run additional checks before the deadline. We would greatly appreciate your feedback on our rebuttal.
Best regards, The Authors
Response to Reviewer AwLh (Part 1/3)
We thank Reviewer AwLh for acknowledging CCB's practical motivation, retraining-free plug-and-play integration across five base models, and constant-memory streaming mode. Below we address each technical point with theoretical analysis and empirical measurements.
Comparison with TTA Baselines, Computational Overhead, Orthogonal Combinations, and Distinct Contributions (W1 & Q1)
We evaluated gradient-based (TENT, TPT) and gradient-free (T3A, TDA, LAME) baselines under the exact same frozen OVCoser base model and protocol (Table R1). Existing test-time methods hit clear bottlenecks on OV-COS:
By contrast, CCB combines three complementary designs for OV-COS. First, CCB decouples visual clustering from class assignment: it clusters in frozen visual space and computes normalized soft class distributions per cluster rather than single-class clusters, avoiding T3A's hard-label contamination and LAME's degree explosion. Second, instead of TDA's fixed residual weights and negative cache, CCB modulates the cluster prior weight per sample by its prediction confidence, preserving confident predictions while correcting ambiguous ones. Third, CCB unifies full-set offline transductive and online streaming inference within one training-free Bayesian formulation across batch sizes, stream orderings, and encoders.
Under this unified evaluation (Table R1), CCB outperforms baselines within each paradigm with negligible overhead:
More importantly, because CCB operates on output probability distributions, it stacks orthogonally with both gradient-free and gradient-based TTA: adding online CCB after TDA raises accuracy from 69.6% to 72.4% (66.7% Macro-F1); offline CCB after LAME-kNN5 raises accuracy to 73.9%; and offline CCB after spatially adapted TENT preserves mask quality while lifting cIoU to 0.478. These orthogonal gains confirm CCB's population-level structural prior is complementary to existing TTA mechanisms.