- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes BTGNet for few-shot semantic segmentation. It uses an LLM to generate category-specific background words, incorporates them into CLIP-based Grad-CAM generation, and refines the resulting activations through fixed and learnable branches.
Critical strengths
- The motivation is reasonable. Foreground-only activation can include co-occurring background regions, and explicitly representing competing background concepts can help distinguish the target from contextual distractors. This is particularly relevant when only a few annotated support images are available.
- The ablations provide useful evidence for the proposed components. The paper separately examines background-word guidance and activation refinement, and compares random, fixed, and LLM-generated vocabularies. Prompt and cross-LLM analyses further examine the stability of vocabulary generation.
- The counterfactual background evaluation is a valuable addition. Preserving the foreground while replacing the query background directly examines whether the method depends on typical object-context associations. The reported results suggest that the benefit is not explained solely by background co-occurrence shortcuts.
Critical weaknesses
1.The methodological novelty needs clearer positioning relative to CLIP-ES (https://arxiv.org/abs/2212.09506) and WeCLIP (https://arxiv.org/abs/2406.11189). CLIP-ES already incorporates background text into softmax-based Grad-CAM and attention-based refinement. WeCLIP further uses learned feature affinity to select frozen CLIP attention maps through an average-distance threshold, which is closely related to BTGNet’s dynamic refinement. Although the task settings and implementations differ, the paper should explain more precisely what advances beyond category-specific vocabulary generation and the combination of existing refinement principles. 2. The attribution of performance gains across BTGNet-P/H/D remains insufficiently controlled. Evaluating several integrations is useful, and Appendix Table 14 provides additional evidence of applicability. However, the listed pretrained representations differ between the original baselines and some BTGNet variants, making it difficult to attribute the reported gains specifically to background-word guidance. The component ablation on BTGNet-H supports the design within that configuration, but does not establish whether the same components explain the gains in BTGNet-P and BTGNet-D. 3. The published vocabularies contain apparent foreground-background conflicts. Table 12 includes the target itself in several background lists. Since these terms participate in the softmax competition used for activation generation, the authors should clarify whether they were used in the experiments or are errors in the table, and assess their effect.
Questions for authors
- Table 5 compares fixed and LLM-generated vocabularies, while Table 3 examines refinement components. Could the authors cross these factors by evaluating both vocabulary types with and without joint refinement, using the same FSS configuration and controlling vocabulary size? Please also clarify whether the fixed lists correspond to those used in CLIP-ES. This would separate vocabulary improvements from refinement benefits.
- Could the authors clarify the methodological differences from WeCLIP and compare its refinement module with BTGNet’s joint refinement within the same FSS pipeline, using identical initial CAMs and background vocabulary?
- Are the baseline-to-BTGNet comparisons for P/H/D matched in both the segmentation backbone and pretrained representation? Where they are not, could the authors provide matched baselines? A compact component ablation for P and D would also help establish whether the contributions observed on H carry over to the other integrations.
- Were the foreground terms listed as background candidates in Table 12 actually used? Please clarify the filtering procedure and, if they were used, evaluate the effect of removing target names and synonyms.