acceptodds
Under review as a conference paper at ICLR 2027

TIGR-Seg: Text-Initialized Gaussian Refinement with Spatially Aware Visual Adaptation for Medical Image Segmentation

Abstract

Medical image segmentation remains challenging due to the limited availability of pixel-level annotations, heterogeneous target appearance, and variations across imaging modalities. Pretrained vision encoders provide strong representations, but updating all backbone parameters introduces substantial trainable capacity, while textual information often provides semantic and coarse spatial guidance rather than precise pixel-level localization. This work presents TIGR-Seg, a parameter-efficient text-guided segmentation framework comprising Gaussian-Gated Feature Adaptation (GGFA) and a Text-Initialized Gaussian Refinement (TIGR) Decoder. GGFA adapts a frozen pretrained vision encoder through context-conditioned feature modulation and a dynamically predicted correlated Gaussian spatial gate, enabling adaptation across both feature and spatial dimensions while updating only a small set of parameters. The TIGR Decoder incorporates stage-specific textual guidance across multiple decoding resolutions. At each stage, text initializes a Gaussian spatial hypothesis that is subsequently refined using visual evidence through reliability-weighted corrections. The refined spatial priors then guide feature aggregation, reconstruction, and progressive decoder refinement. Experiments across gastrointestinal endoscopy (BKAI), brain MRI (BTMRI), and breast ultrasound (BUSI) datasets demonstrate strong segmentation performance with a low trainable-parameter count. Additional evaluations support the effectiveness and generalizability of TIGR-Seg.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Medical Image Segmentation, Vision-Language Models, Text-Guided Segmentation, Parameter-Efficient Fine-Tuning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iRLrOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

TIGR-Seg combines Gaussian-Gated Feature Adaptation (GGFA) of a frozen Swin encoder with a decoder that refines text-initialized spatial priors using image features. It reports Dice scores of 92.78%, 84.76%, and 86.77% on BKAI, BTMRI, and BUSI, respectively, with 4.6M trainable parameters. The experiments are promising, but the methodological advance and reliability of the text-refinement mechanism remain insufficiently established.

Critical strengths
  1. Coherent architecture and motivation. Combining parameter-efficient visual adaptation with text-guided spatial refinement is sensible. The encoder and decoder components have clear, complementary purposes.
  2. Encouraging adaptation results. Supplementary Table 6 compares GGFA with LoRA, Houlsby Adapter, FiLM, and full fine-tuning while retaining the remaining architecture. GGFA achieves the highest reported Dice/mIoU on both evaluated datasets with 4.6M trainable parameters.
  3. Interesting decoder formulation and useful ablations. Treating text as an initial hypothesis is a reasonable design. The bounded covariance parameterization is valid, and the paper investigates Gaussian gating, correlation, decoder components, prompt sensitivity, and cross-dataset generalization.
Critical weaknesses
  1. Moderate novelty and incomplete attribution. GGFA combines context-conditioned affine modulation with Gaussian gating, while TIGR adds image-conditioned corrections to a text-derived Gaussian prior. The combination is reasonable, but the conceptual advance over existing adaptation and multimodal conditioning is limited. Although the supplementary ablations are useful, there is no targeted comparison with fixed or image-only Gaussian initialization to establish why text initialization itself is necessary, rather than Gaussian conditioning more generally.
  2. Prompt robustness is weaker than the framing suggests. In supplementary Table 5, removing text reduces Dice by 13.30 points on BKAI and 15.51 on BUSI; contradictory locations reduce it by 16.44 and 20.85 points. Even a longer prompt with the same correct information causes noticeable degradation. These results demonstrate strong text dependence, but do not establish that image-guided correction can reliably recover from inaccurate guidance. The paper should distinguish successful refinement of accurate prompts from robustness to unreliable prompts.
  3. The equations do not fully support the interpretation of presence and reliability. Substituting Eq. (26) into Eq. (27) cancels the scalar presence score from the normalized weighted average. Thus, presence cannot directly attenuate that aggregation, although it can still affect reconstruction and gating in Eqs. (29) and (31). Also, the sigmoid correction weight in Eq. (19) is called reliability without calibration or an ablation against a constant weight. The roles attributed to these terms need more precise explanation and targeted validation.
  4. Evaluation independence and text provenance need clarification. Section 3.1 specifies fixed 80/20 splits but does not establish patient-disjoint splitting, particularly important for the brain MRI evaluation. It cites sources for structured descriptions without explaining whether test-time location and shape annotations require ground-truth masks or how equivalent guidance would be obtained in practice. This is not proof of leakage, but it leaves the independence and deployment relevance of the evaluation unclear.
  5. Statistical and ablation evidence is incomplete. Only TIGR-Seg has standard deviations in the main comparison, while competing methods and most ablations are reported as single values. The uncertainty of the comparative gains is therefore unclear. Table 2 omits the configuration with neither GGFA nor TIGR. The PEFT comparison also lacks explicit adapter/rank settings and tuning details, making it difficult to assess whether alternative adaptation methods received equally competitive configurations.
  6. Training efficiency does not establish a compact inference model. Supplementary Table 11 reports 166.9M total parameters, a 667.9 MB model, and 245.4 ms/image latency, despite only 4.6M trainable parameters. Several listed methods are smaller or faster. The authors acknowledge this distinction, but the broader efficiency framing should consistently emphasize parameter-efficient training. Timing details such as batch size, precision, warm-up, and inclusion of text encoding are needed to interpret the latency comparison.
Questions for authors

The architecture is coherent, the reported segmentation performance is strong, and the supplementary experiments provide useful evidence. However, the novelty is moderate, the central correction mechanism remains highly dependent on prompt quality, and important aspects of evaluation independence, comparative uncertainty, and mathematical interpretation need clarification.

Reviewer_C8ndOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TIGR-Seg, a parameter-efficient text-guided framework for medical image segmentation. It contains two main components. GGFA adapts a frozen encoder through context-conditioned feature modulation and a dynamically predicted Gaussian gate. TIGR Decoder uses stage-specific text embeddings to initialize a Gaussian spatial hypothesis and a presence logit, applies bounded reliability-weighted image-evidence correction, and uses the correlated Gaussian prior to guide multi-scale feature aggregation. Experiments show competitive segmentation performance.

Critical strengths

1.The paper introduces the TIGR Decoder, which refines text-initialized Gaussian priors using image evidence to guide multi-scale feature aggregation and updates.

2.The method is parameter-efficient. It requires only 4.6M trainable parameters.

3.The experimental evaluation is extensive and the visualizations are comprehensive.

Critical weaknesses

1.The research boundary and scope of applicability are unclear. Overall, the concept of text initialization with spatial awareness is somewhat innovative. However, the paper’s description of the problem, research motivation, and experiments lacks clinical evidence, making it difficult to assess the significance of these findings within a broader research context. While the experimental results are relatively comprehensive, the discussion of the experiments is limited to metric values and lacks in-depth analysis grounded in clinical scenarios. Most of the cases shown in the visualizations feature lesions with relatively regular shapes, and the single-Gaussian method also has limitations regarding the number of lesions. Consequently, the primary concern regarding this paper is that its scope of applicability appears to be quite limited. The entire paper should undergo substantial revision to improve its logical coherence, explore the study’s limitations, and clearly specify the applicable medical imaging modalities and disease types.

2.The problem statement is unclear. The paper’s problem formulation remains at the level of general domain issues—full fine-tuning is parameter-intensive, and textual spatial precision is coarse—but it does not identify the specific clinical challenges that correspond to these problems.

3.The motivation for the method design is insufficient. The description and explanation of the method are largely confined to architectural design and lack design motivations derived from the identified problems. For example, GGFA uses window-level context to guide adaptation. This context is referred to in the paper as a basis for spatial structure, but the rationale is not explained in relation to the specific characteristics of medical images. Why do GGFA and the TIGR Decoder use Gaussian modulation for features?

4.The data setup is unclear. It remains unknown whether the multimodal datasets are trained jointly or separately, which makes the scope of the work’s contribution hard to assess.

5.The text generation method is unclear. The text generation strategy and its accuracy are not described, and the generation methods for the different text granularities in Table 5 are also unspecified, making it difficult to assess the applicable text conditions.

6.The ablation study lacks the case in which neither component is added. (Table 2)

Questions for authors

Please refer to the Critical Weaknesses. The authors are encouraged to carefully consider and address these points, as doing so will strengthen the logical coherence of the paper and help clarify its clinical contribution.

Reviewer_UndfOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

To address the challenge of converting text semantics into spatial priors with minimal parameter adaptation, this paper proposes TIGR Seg. This method utilizes text to initialize interpretable Gaussian space priors, and then gradually refines the concepts through image features, making them attractive. Experiments on three publicly available medical image datasets, as well as ablation studies on GGFA and TIGR Decoder components, have demonstrated the effectiveness of their combination.

Critical strengths
  1. Transform the text into a continuous and relevant Gaussian prior space, refine it with visual evidence, and design an interpretable and research worthy solution.
  2. The experiment is comprehensive and the ablation research system can further support the claims of the paper.
Critical weaknesses
  1. In Equation (17), GAP/GMP compresses spatial features into global statistics, weakening the explicit spatial localization ability; It is still unclear how the calibration head can reliably adjust the Gaussian center based on this.
  2. Table 1 only provides the mean and standard deviation of the proposed methods, with a single baseline, limiting variability and gain stability evaluation; The total parameters of the entire model are 166.9M, and the deployment efficiency is questionable.
Questions for authors
  1. What is the basis for predicting Gaussian center shift using GAP/GMP features? How does the localization capability compare to introducing local spatial feature refinement?
  2. Is the baseline in Table 1 also averaged over the same three random seeds? Please explain and report its variability as much as possible.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.