Evidence Is Not Extent: Separating Referential Evidence, Object Extent, and Boundary Trust for Referring Image Segmentation

Parameter-efficient referring image segmentation asks one dense score to name the referent, cover its extent, and vouch for its boundary. However, evidence that identifies the referent is often sparse, such as a color patch or an anchor object, while the final mask must cover the full object extent. Therefore, sharpening the score may truncate the object, whereas a shape prior that completes it can spill onto same-category distractors. We introduce E3-RIS, separating these decisions by authority: each source may change only the pixels it is entitled to change. Prompt-Anchored Referential Evidence (PARE) pools prompt-initialized queries into one posterior that names the referent. Evidence-Constrained Bidirectional Projection (ECBP) lets frozen SAM add or remove pixels only inside a boundary band where the posterior agrees. Boundary Trust-Region Arbitration (BTRA) admits a direction-fixed edit when a learned score clears its threshold, and otherwise keeps the projected mask. On RefCOCO val, E3-RIS lifts mIoU over TALENT from 77.8 to 78.5 and [email protected] from 40.1 to 44.2. It also exceeds TALENT in mIoU across all nine official RefCOCO, RefCOCO+, and RefCOCOg splits.