- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper primarily investigates target localization and pixel-level mask prediction for reasoning segmentation with frozen large multimodal models. For reasoning queries, the model first generates a target description R, and then performs spatial grounding using Q+R as the textual context. The authors propose Instance-Conditioned Latent Steering (ICLS), which learns instance-specific steering from layer-wise representation differences between the original image and its target-erased counterpart, and injects the resulting steering into the hidden states of the frozen LMM. They further introduce a Steering-Conditioned Grounding Decoder (SCGD), which reuses the same representation to guide progressive multi-scale mask decoding, followed by SAM refinement. Experiments are conducted on ReasonSeg and the RefCOCO series.
Critical strengths
The paper presents an approach to adapting target-related representations for individual instances during conditional spatial grounding with frozen large multimodal models. ICLS learns instance-specific steering from layer-wise representation differences between the original image and its target-erased counterpart, while SCGD reuses this signal to guide multi-scale mask decoding. With the LMM backbone kept frozen, the authors report substantial improvements on ReasonSeg over their reproduced F-LMM baseline, and the ablation studies identify ICLS as the main contributor to these gains. This design establishes a concrete connection between internal representation adaptation in a frozen LMM and downstream spatial grounding.
Critical weaknesses
1.The scope of the paper’s contribution to reasoning segmentation is insufficiently clear. Section 3.1 indicates that the model first generates a target description R, after which ICLS and SCGD operate only during the grounding pass using Q+R as textual context. Thus, the proposed method primarily improves conditional spatial grounding given the generated description, rather than target-description generation or the reasoning process itself.
2.Several key methodological details are insufficiently specified, and the model architecture is overly complex. These include layer/head selection, the definitions of and , text–visual representation alignment, SAM prompt construction, and the modules active during QA evaluation. Moreover, despite freezing the LMM backbone, the combination of U-Net, SCGD, and SAM refinement makes the architecture overly complex.
Questions for authors
1.The method primarily improves referring segmentation rather than the full reasoning segmentation process. Both ICLS and SCGD operate only during the subsequent grounding stage, after Q+R is available. Thus, the proposed modules optimize grounding rather than the preceding reasoning process that generates R.
2.The correctness of the target description R is not independently evaluated. Figure 4 provides qualitative examples, but whether ICLS and SCGD can recover from incorrect descriptions remains unclear. It would be valuable to include the following analyses: (1)The target-identification accuracy of the generated description R. (2)The impact of incorrect descriptions on the final mask. (3)A comparison between using Q alone and using Q+R.
3.The experimental comparison with recent related methods is insufficient. In Table 1, Qwen2.5-SAM-7B (Hao et al., 2026) is the only comparison method explicitly cited as a 2026 work, while most other methods are from 2023–2025. The authors should include additional relevant methods from 2026 to strengthen the experimental evidence.
4.The method does not consistently outperform LENS-7B. Table 1 shows higher gIoU only on ReasonSeg Test-Long and Test-All, but lower cIoU across all ReasonSeg splits. Section 4.2 highlights the gIoU gains without discussing the cIoU deficits. Please explain this discrepancy and provide a balanced assessment of the method’s strengths and limitations.
5.SAM refinement may confound the contributions of ICLS and SCGD. Tables 3–6 do not separately evaluate SAM refinement. It will be helpful to clarify how SCGD mask logits are converted into SAM prompts, report performance before and after refinement, and compare F-LMM and ICLS under the same SAM configuration.
6.The reference baseline for the 7.3M additional parameter claim should be made explicit. The abstract emphasizes that SteerGrounding introduces only 7.3M additional trainable parameters, but the 7.3M parameters are additional to the F-LMM baseline, whereas the complete model has 20.1M trainable parameters. The abstract should clearly distinguish additional from total trainable parameters, and the paper should also report the total model parameter count, including frozen modules.
7.The QA evaluation protocol requires clarification. Section 4.2 reports performance comparable to F-LMM, but does not specify which modules are active during QA evaluation or whether prompts and decoding settings match those of the original model. These details are needed to assess the claim of maintained multimodal capabilities.