acceptodds
Under review as a conference paper at ICLR 2027

BiReS: Bidirectional Semantic Recalibration for Text-Guided Multi-Scale Medical Image Segmentation

Abstract

Medical image segmentation remains challenging due to substantial variations in target scale, complex and poorly defined boundaries, and the limited semantic awareness of vision-only models. Although multimodal approaches introduce complementary semantic priors, the granularity gap between global textual semantics and dense visual representations complicates effective cross-modal fusion. Meanwhile, many existing methods treat language primarily as auxiliary guidance for visual features, with limited image-conditioned feedback to textual representations, resulting in asymmetric cross-modal interaction. To address these limitations, we propose **BiReS**, a multimodal segmentation framework that combines bidirectional semantic recalibration with scale-aware feature refinement. Specifically, we introduce a **Semantic Recalibrator (SR)** that leverages global textual semantics to modulate visual responses along both spatial and channel dimensions, while attention-weighted visual pooling reciprocally updates textual representations, forming a closed-loop semantic recalibration process. To handle target scale heterogeneity and hierarchical feature mismatch, we introduce a **Multi-Scale Feature Enhancement (MSFE)** module in each decoder stage to refine fused hierarchical features with local, multi-scale, and global context. A **Tri-Attention (TA)** mechanism further refines decoder features, suppressing irrelevant responses while enhancing salient structures and fine-grained boundary cues. Extensive experiments on multiple medical image segmentation benchmarks demonstrate that BiReS consistently outperforms state-of-the-art vision-only and vision-language methods, with notable gains on challenging targets exhibiting substantial scale variation and complex boundary structures.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.