- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Cycle-ML, a test-time adaptation (TTA) framework built upon the UniPixel architecture. The method constructs an optimization loss based on the consistency of a mask-language cycle and uses it to adapt the model at test time for reasoning segmentation. To mitigate the issue of self-consistent but incorrect predictions, the authors further introduce a Consistency-Reliability Learning (CRL) mechanism, which jointly leverages cross-modal constraints to estimate prediction reliability and downweight unreliable samples during adaptation. Cycle-ML is evaluated on multiple benchmarks for reasoning segmentation and referring segmentation, covering 15 types of synthetic corruptions. Experimental results show that Cycle-ML generally improves performance over non-adaptive models and several representative TTA baselines.
Critical strengths
-
Cycle-ML leverages UniPixel's ability to generate language descriptions from masks, which innovatively establishing mask-language consistency (MLC) and implementing consistency-reliability learning (CRL) within a bimodal paradigm.
-
Extensive experiments, including various baseline methods and 15 types of synthetic corruptions, demonstrate that Cycle-ML serves as an effective test-time adaptation (TTA) strategy, significantly enhancing performance on reasoning segmentation and referring segmentation tasks.
Critical weaknesses
- Limited in Scalability and Generalizability
Does Cycle-ML require the base model to have the capability to generate text from mask? This paper does not justify why the reconstructed query must be generated by the same base model. If this is not essential, the authors should evaluate alternative query-generation strategies and include experiments on non-UniPixel architecture models to demonstrate the scalability and generalizability of Cycle-ML.
- Method's Defects
When the base model produces mutually consistent but incorrect masks and descriptions, the proposed method may assign an excessively high reliability weight to the adaptive loss. Specifically, it is possible to have while simultaneously obtaining Consequently, the corresponding reliability weight would also be high, causing the model to reinforce incorrect predictions during adaptation. Figure 4 appears to show cases in which both mask and language consistency increase after adaptation, while the IoU with the ground-truth mask decreases substantially. Could this phenomenon be attributed to such self-consistent but incorrect predictions? The authors should provide a detailed analysis of these failure cases to better understand the limitations of the proposed reliability estimation mechanism.
Questions for authors
- Related Works
Could the authors discuss the following closely related TTA methods and clarify how Cycle-ML differs from them?
[1] (Seg-TTO) Test-Time Optimization for Domain Adaptive Open Vocabulary Segmentation
[2] (EATA) Efficient Test-Time Model Adaptation without Forgetting
[3] (ROID) Universal Test-time Adaptation through Weight Ensembling, Diversity Weighting, and Prior Correction
Could the authors clarify how Cycle-ML could be adapted to influential reasoning segmentation methods beyond UniPixel (such as those listed below)? What modifications would be required to apply Cycle-ML to these methods?
[4] (PixelLM) PixelLM: Pixel Reasoning with Large Multimodal Model
[5] (LENS) LENS: Learning to Segment Anything with Unified Reinforced Reasoning
[6] (Seg-Zero) Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement
[7] (SAM-R1) SAM-R1: Leveraging SAM for Reward Feedback in Multimodal Segmentation via Reinforcement Learning
- Ablation Study
a. Could the authors specify the configuration of the CRL cold-start strategy mentioned in Appendix H.2, particularly the minimum buffer size required for “meaningful partitioning”? Could they also provide an ablation study examining how different cold-start configurations affect performance?
b. How sensitive is the final performance to the clipping strategy used to compute Gaussian reliability weights? Could the authors provide an ablation comparing the current strategy with removing clipping and using alternative clipping strategies?
c. How sensitive is Cycle-ML to the order in which test samples are processed? Could the authors evaluate multiple test-sample orderings and report the resulting performance variation to assess the robustness of test-time adaptation?
- Formatting Issues
Could the authors check and correct instances where en dashes are used within words that require hyphens?