Negative Memory Suppression: Rejecting Distractors in Coronary Angiography Vessel Segmentation using an Anchor Frame

Accurate vessel segmentation in X-ray coronary angiography supports quantitative analysis and procedural guidance, but is complicated by vessel-like non-vascular structures, such as catheters and ribs, that resemble coronary vessels in projection and can move differently from them. A pre-contrast frame provides a reference for identifying nonvascular structures already present before contrast injection. We introduce NeMeS, a promptless dual-branch memory model for coronary vessel segmentation in X-ray angiography that uses a pre-contrast anchor frame, typically available at the beginning of an angiographic acquisition, as annotation-free negative memory. A proposer branch segments vessel-like candidates from the video clip, while an anchor-conditioned suppressor attenuates these responses through a multiplicative gate that can suppress, but cannot increase, proposer predictions. Trained exclusively on the in-house training split, NeMeS outperforms the strongest baseline on both the full test set and a device-cluttered hard-case subset. It also transfers zero-shot to XACV and MOSXAV, achieving comparable Dice coefficient to the reported finetuned VasoMIM result on XACV and improving MOSXAV Dice coefficient by 5.0 points over TMANet. Anchor perturbation and synthetic distractor experiments support anchor-dependent rejection, while probing trained and randomly initialized, frozen networks provides evidence consistent with an architectural inductive bias for vessel-presence discrimination.