acceptodds
Withdrawn from ICLR 2027

Negative Memory Suppression: Rejecting Distractors in Coronary Angiography Vessel Segmentation using an Anchor Frame

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurate vessel segmentation in X-ray coronary angiography supports quantitative analysis and procedural guidance, but is complicated by vessel-like non-vascular structures, such as catheters and ribs, that resemble coronary vessels in projection and can move differently from them. A pre-contrast frame provides a reference for identifying nonvascular structures already present before contrast injection. We introduce NeMeS, a promptless dual-branch memory model for coronary vessel segmentation in X-ray angiography that uses a pre-contrast anchor frame, typically available at the beginning of an angiographic acquisition, as annotation-free negative memory. A proposer branch segments vessel-like candidates from the video clip, while an anchor-conditioned suppressor attenuates these responses through a multiplicative gate that can suppress, but cannot increase, proposer predictions. Trained exclusively on the in-house training split, NeMeS outperforms the strongest baseline on both the full test set and a device-cluttered hard-case subset. It also transfers zero-shot to XACV and MOSXAV, achieving comparable Dice coefficient to the reported finetuned VasoMIM result on XACV and improving MOSXAV Dice coefficient by 5.0 points over TMANet. Anchor perturbation and synthetic distractor experiments support anchor-dependent rejection, while probing trained and randomly initialized, frozen networks provides evidence consistent with an architectural inductive bias for vessel-presence discrimination.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
X-ray Coronary Angiography, Medical Video Segmentation, Negative Memory

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_RMrQOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes NeMeS, a promptless dual-branch memory architecture for coronary angiography vessel segmentation. The key idea is to exploit a pre-contrast frame as annotation-free negative memory: a proposer branch performs causal vessel segmentation using temporal memory, while an anchor-conditioned suppressor branch identifies structures that are already present before contrast injection. Their predictions are combined through the multiplicative gate $ P = \sigma(L_A)\left(1-\sigma(L_B)\right), $ so that the suppressor can attenuate vessel-like distractors but cannot introduce new positive predictions.

The method is trained on a large in-house coronary angiography dataset and evaluated on the full in-domain test set, a device-cluttered hard-case subset, and two external datasets, XACV and MOSXAV, in a zero-shot setting. The authors additionally provide anchor ablations, a parameter-matched random-anchor control, device-specific false-positive analysis, synthetic distractor experiments, temporal-change evaluation, and feature probing. The reported results show improvements over CNN-, ViT-, temporal-, and SAM2-based baselines, particularly for device-heavy cases and cross-domain evaluation.

Critical strengths
  1. Using the pre-contrast frame as negative memory is well motivated by the acquisition process itself: nonvascular anatomy and interventional devices may already be present before coronary opacification, whereas the target vessels emerge after contrast injection. This provides a natural mechanism for distinguishing true vessels from vessel-like distractors and directly targets an important failure mode of existing segmentation systems.
  2. Beyond standard Dice-based comparisons, the paper evaluates hard cases, two external datasets without target-domain fine-tuning, device-specific false positives, anchor perturbations, a parameter-matched random-anchor model, synthetic distractor rejection, temporal-change accuracy, and representation probing. In particular, the substantial reduction of device false positives on MOSXAV provides convincing evidence that the anchor is useful for the failure mode targeted by the paper.
Critical weaknesses
  1. The methodological contribution appears highly specific to the coronary angiography acquisition protocol, and the current experiments do not establish that ``negative memory'' is a sufficiently general learning principle. NeMeS relies on the availability of a pre-contrast frame that contains approximately the same anatomical scene and devices but lacks the target contrast-filled vessels. This is a strong task-specific source of supervision. Once such a reference is available, the proposed architecture mainly consists of a second SAM2 branch conditioned on this reference and a multiplicative suppression gate. It is therefore unclear how much of the contribution represents a general architectural principle rather than an effective exploitation of a domain-specific acquisition cue. This is particularly important for ICLR, where broader methodological implications would substantially strengthen the contribution.
  2. The experiments do not sufficiently establish that the proposed dual-branch memory architecture is necessary compared with simpler ways of exploiting the same pre-contrast information.
Questions for authors
  1. Can the authors compare NeMeS against simpler methods that have access to exactly the same pre-contrast anchor, for example $ x_t-x_a, \qquad [F(x_t);F(x_a)], $ a registration/subtraction-based baseline, or a single-branch SAM2 model with anchor-conditioned cross-attention? Such comparisons would materially clarify whether the gain arises from the proposed negative-memory decomposition or simply from making the pre-contrast reference available to the model.
  2. How generally is the required pre-contrast anchor available in real clinical acquisitions, and what happens when the reference is substantially mismatched because of table motion, catheter manipulation, projection changes, patient motion, or residual contrast from a previous injection? The current contrast-level perturbation experiment addresses vessel opacification, but does not fully characterize these clinically plausible anchor failures.
  3. The paper reports two-sided paired tt-tests with P<0.01P<0.01. What is the statistical unit used in these tests---frame, sequence, or patient? Since multiple annotated frames can belong to the same sequence or patient, treating individual frames as independent observations could substantially underestimate uncertainty. Please clarify the pairing strategy and, preferably, report sequence- or patient-level confidence intervals.
  4. How was the in-house ``hard-case'' subset of 190 sequences determined? Was the criterion defined prospectively and independently of model predictions, and was the selection performed without inspecting the comparative performance of NeMeS and the baselines? A reproducible definition is important because the hard-case analysis constitutes a central part of the paper's motivation and claimed advantage.
  5. The random-initialization probing experiment is interpreted as evidence for an architectural inductive bias. However, the dual-branch representation has explicit access to both the target and anchor, whereas the single-branch SAM2 representation does not contain the same information. Can the authors provide an input-matched control in which both architectures receive the same anchor and target information but differ only in the proposed negative-memory organization? This would more directly support the architectural-inductive-bias claim.
Reviewer_iLcjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a promptless dual-branch memory model (NeMeS) for coronary vessel segmentation in X-ray angiography. NeMeS uses an annotation-free pre-contrast anchor frame (available at the beginning of the procedure) as negative memory to learn a suppressor that gates a vessel proposer to reject vessel-like distractors. The authors showed that NeMeS outperforms the strongest baselines on an in-house dataset and that it can transfer zero-shot to XACV and MOSXAV with comparable performance compared to fine-tuned baseline methods.

Critical strengths
  1. The issue is well motivated, and the authors tried to resolve an issue that is clinically relevant. Figure 1 shows that the SAM2 model could be influenced by vessel-like distractors and make false positive predictions.

  2. The proposed method requires no additional data collection/calibration steps in the existing work flow. NeMeS takes advantage of the pre-contrast anchor frame as a distractor prior, which is already in the clinical workflow and requires no annotation and no additional acquisition.

  3. The method can show some promising zero-shot results with NeMeS transfers to XACV and MOSXAV without any target-domain training or fine-tuning, which is important in cross-setting/cross-hospital generalization and transfer.

Critical weaknesses
  1. The contribution of the anchor is unclear, more experiments are needed to highlight its effectiveness. Section 4.4 shows suppressor and capacity control results (in dice, the full test set): 0.785 (SAM2) < 0.797 (parameter-matched random-anchor variant) < 0.805 (pre-contrast-anchor model). It seems that the random anchor is contributing most of the performance improvement, and the pre-contrast-anchor model’s contribution is less prominent (+0.8 dice out of +2.0 dice in total). In Figure 4, the suppressor response covers almost the entire image except the area of the vessels, which is hard to tell apart from a non-vessel detector that ignores the anchor completely. More experiments should be conducted to clearly support the strength of the anchor.

  2. The baselines in Table 1 need expanding. Besides SAM2, results in medical image segmentation foundation models such as MedSAM2 [1,2] as well as other newer video foundation models like sam3 [3] should be included as baselines. Both branches currently only use the SAM2 Hiera-T, more powerful SAM2 backbones should be used to test where the observed FP issue persists.

  3. The claim of motion-tolerant, registration-free suppression is not directly validated. The authors used examples (catheters, guidewires, etc) as a motivation since these elements exhibit independent or partially correlated motion. But I couldn’t find direct evaluation, or visualization of the scenarios in which the distractor is displaced or deformed. One close experiment is the paste-in-a-rib in Table, but only the thickness is varied. Including more experiments with moving and deforming objects would benefit the claim.

  4. Minor typos like “Appendix ??” and “described in Sections 4 and 4.4”.

[1] Ma, J., Yang, Z., Kim, S., Chen, B., Baharoon, M., Fallahpour, A., ... & Wang, B. (2025). Medsam2: Segment anything in 3d medical images and videos. arXiv preprint arXiv:2504.03600.

[2] Zhu, J., Hamdi, A., Qi, Y., Jin, Y., & Wu, J. (2024). Medical sam 2: Segment medical images as video via segment anything model 2. arXiv preprint arXiv:2408.00874.

[3] Carion, N., Gustafson, L., Hu, Y. T., Debnath, S., Hu, R., Suris Coll-Vinent, D., ... & Feichtenhofer, C. (2026, April). Sam 3: Segment anything with concepts. In International conference on learning representations (Vol. 2026, pp. 138846-138923).

Questions for authors
  1. The comparison with VasoMIM on XACV is unclear. In Table 1, NeMeS achieves a Dice of 77.5 on XACV, identical to the reported VasoMIM result, whereas in Table 2, NeMeS achieves 77.9. What causes this change, and are the two settings the same?
  2. The information about the in-house database is limited, more information should be provided.
Reviewer_gDDbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents a strategy to segment coronary vessels from X-ray angiography videos. The problem they want to address is the similarity between interventional tools (mainly catheters) and vessels. For that purpose, they propose using a negative anchor frame as a reference, under the assumption that such a frame contains all non-vascular structures, whereas vessels are not visible due to the lack of contrast. On those grounds, they build a dual-branch architecture where a proposer branch segments vessel candidates and an anchor-conditioned suppressor branch attenuates the proposer's responses.

The method is trained on an in-house dataset and evaluated in a zero-shot setting on two public databases.

Critical strengths

The paper has some strength, but I would not call them critical.

Critical weaknesses
  • The negative-memory mechanism cannot handle distractors absent from the anchor, which undermines the central claim. The method assumes that the pre-contrast anchor contains all nonvascular structures. This is often not the case. Tools will appear along the sequence (the guidewire, catheter, etc.). Since the suppressor can only attenuate responses matched in the anchor, no mechanism prevents vessel-like structures that appear after the anchor from being segmented. The paper’s own experiments reflect this. In the paste-in-a-rib experiment, the model performs well when the distractor is present in both anchor and clip, but false positives increase when it is present in the clip only. Similarly, the anchor-frame choice experiments show a drop in performance when the anchor already contains contrast-filled vessels. Moreover, temporal novelty relative to the anchor is not specific to contrast, since any structure introduced after the anchor is equally novel. For these structures, the model must still rely on appearance, the feature that the paper describes as a shortcut.
Questions for authors
  1. Does suppression hold when the distractor moves between anchor and clip? For instance, the distractor experiment could be repeated with the distractor displaced according to motion estimated from real structures in the same sequence (e.g., the catheter or diaphragm), or results could be reported on real sequences with visible motion of nonvascular structures between anchor and frame.
  2. Please report results stratified by whether each distractor is present in the anchor frame, particularly devices introduced afterwards. Does NeMeS improve over its proposer branch alone, and over the strongest baseline, on distractors absent from the anchor?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.