- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how representations from different multimodal Transformer layers should be composed into segmentation prompts for medical visual grounding. The proposed LaSE follows a map-first strategy: Layer-wise Token Matching (LTM) constructs candidate spatial maps at each layer; Medical Evidence-Guided Fusion (MEGF) uses MedSAM2 features pooled from the regions indicated by these maps to adaptively fuse both the maps and their corresponding segmentation-token states; and Lightweight Mask Refinement (LMR) refines the decoded mask. Evaluation covers four source and ten out-of-domain datasets. The paper reports a 5.12-point improvement in average OOD DSC over MedCLIPSeg, alongside component ablations and analyses of pre-decoder spatial maps.
Critical strengths
- Well-motivated design. LaSE lets layer fusion consider actual spatial responses, addressing the gap between semantic relevance and localization accuracy.
- MEGF uses medical image features from candidate regions to predict input-dependent layer weights. Shared fusion weights keep spatial maps and segmentation-token states aligned.
- Experiments cover four source and ten OOD datasets, with gains across all ten targets.
Critical weaknesses
-
The main ablation does not sufficiently isolate the contribution of spatial-evidence-guided fusion. The layer-first control uses hidden states and REINFORCE to select a single layer, whereas the full model uses regional features for soft multi-layer fusion. This changes the information source, fusion mechanism, and optimization procedure simultaneously. Better pre-decoder maps are informative, but do not distinguish the benefit of regional evidence from that of multi-layer aggregation or more direct optimization. Since map-first fusion is the central methodological contribution, the absence of a hidden-state-only control with the same soft-fusion mechanism, and simple baselines such as uniform fusion, limits the strength of the attribution.
-
The benchmark provides limited support for general medical visual grounding. The released descriptions sometimes contain annotation-derived attributes, including location, shape, and component count. This is not inherently inappropriate, but performance in this setting does not directly establish generalization to independently authored clinical requests. The paper also provides limited evidence that the model can select different targets in the same image when the text changes, or reject mismatched requests. Consequently, the evidence more strongly supports cross-domain segmentation under the evaluated descriptions than text-dependent target selection or complex referring-expression resolution.
-
Although LaSE requires only one MedSAM2 decoder call, Table 6 reports approximately 4.35B total parameters and 0.773 seconds per image, compared with 0.049 seconds for MedCLIPSeg. Thus, reduced decoder overhead does not establish full-system efficiency. The efficiency claim should be limited to the specific decoding design and supported by matched timing comparisons.
Questions for authors
- A hidden-state-only weighting control using the same soft multi-layer fusion mechanism, together with a uniform-fusion baseline, is needed to distinguish the contribution of regional evidence from that of the fusion strategy.
- Performance after removing annotation-derived spatial and shape attributes, together with generic-prompt or mismatched-prompt controls, would clarify how strongly the results depend on the target descriptions.
- The authors should clarify whether the test sets contain different target requests for the same image and report separate results for such cases or empty-target cases to substantiate target-selection capabilities.
- The authors should clarify the scope of the efficiency claim and provide a matched runtime breakdown for the multimodal backbone, layer-wise fusion, and mask decoding, including comparison with the layer-first variant.