acceptodds
Under review as a conference paper at ICLR 2027

REVIVE: Counterfactual On-Policy Diagnosis And Recovery for Interactive Ultrasound Segmentation Agent

Abstract

Recent MLLM-based segmentation agents automate multi-round interactive segmentation by generating corrective prompts for SAM-based models. However, they primarily emphasize action imitation while lacking explicit diagnosis of the current error state before action selection. Moreover, trajectory-level reinforcement learning optimizes sampled actions through sparse reward signals but does not explicitly compare alternative corrections at the same on-policy state, limiting recovery from suboptimal decisions and allowing errors to accumulate across interaction rounds. In clinical settings, human annotators follow a "diagnose-act-verify" self-correction loop: they need to diagnose the current mask errors before acting and then verify their effectiveness from the updated mask. We propose REVIVE, a self-improving segmentation agent that explicitly diagnoses the residual-error state before action, which further translates residual error evidence into action-specific representations. Then, the Counterfactual On-Policy Action Ranking (COAR) is proposed to execute alternative corrective actions at on-policy states and align the action likelihoods with the resulting segmentation outcomes, providing dense state-wise supervision for corrective action-type selection, while its integration with trajectory-level GRPO improves multi-round interaction behavior. Experiments demonstrate that our REVIVE consistently improves segmentation quality and interaction reliability, while reducing error-action misalignment and improving recovery from self-induced errors.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Interactive Segmentation, Medical Image Segmentation, Multimodal Large Language Models, Agentic Reinforcement Learning, Counterfactual Policy Learning, Ultrasound Imaging

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_s69FOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies MLLM-based agents that refine a segmentation mask over several rounds by sending a positive click, a negative click or a stop signal to a frozen MedSAM, with a focus on ultrasound images. It argues that existing agents such as MedSAM-Agent and IBISAgent choose actions without an explicit estimate of the current error, and that trajectory-level RL never compares alternative actions at the same state. REVIVE adds two components. A small Residual-Error Verifier predicts false-negative and false-positive maps from MedSAM outputs, and these maps are turned into action-specific tokens for the MLLM. During training, COAR evaluates all three action types at each state visited by the policy, scores them against the ground truth, and applies a pairwise ranking loss to the action-type logits, on top of the GRPO objective of MedSAM-Agent. Against ten MLLM-based methods, REVIVE has the highest Dice on six of seven ultrasound datasets and on one of four polyp datasets. An ablation on BUS-BRA raises Dice from 0.766 (GRPO only) to 0.834 with both components. A further table reports that REVIVE recovers from 88% of its erroneous actions, against 25% and 41% for the two agent baselines.

Critical strengths
  1. The paper starts from a reasonable observation. The frozen segmentation tool can be queried cheaply during training, so the outcome of every action type can be computed at the states the policy actually visits and used as supervision at each step. The resulting ranking loss is simple and fits into the existing training recipe of MedSAM-Agent.
  2. The diagnosis module is designed so that it can be used at test time. The error maps are predicted from signals of the frozen tool, and each action gets its own evidence: false negatives outside the mask for a positive click, false positives inside the mask for a negative click, and both for stop.
  3. The evaluation covers seven ultrasound and four polyp datasets with ten MLLM-based baselines and four metrics. It also looks at what the agent does after a wrong action (Table 4), not only at the final Dice.
Critical weaknesses
  1. The comparison with other methods does not show that the gains come from the proposed components. The paper does not say whether any baseline was trained or fine-tuned on the data used for REVIVE, and for the seven ultrasound datasets no training set or test split is given. The tables suggest that the baselines were run as released: LISA++ and SAM4MLLM have recall close to 1 and precision about equal to IoU on most datasets, so they return almost the whole image. The authors' own GRPO-only model M1 reaches 0.77 Dice on BUS-BRA (Table 3), while MedSAM-Agent, whose GRPO training and reward design M1 retains, is listed at 0.54 (Table 1). About three quarters of the gap between MedSAM-Agent and REVIVE is therefore present before either component is added. The text also overstates the results: REVIVE is behind Citrus-V on Kvasir-SEG and CVC-ClinicDB and behind MedSAM-Agent on FH-PS-AoP and CVC-300, yet Citrus-V is said to be "still outperformed" (L413-415).
  2. The evidence for the two components and for the recovery claim is thin. The ablation uses one dataset (BUS-BRA) and reports no variance over runs, and the diagnosis module alone adds 0.014 Dice. The accuracy of the Residual-Error Verifier is never reported, although the diagnosis step depends on it and the paper itself stresses that errors in ultrasound are hard to identify. Simpler uses of the same signals are not tested, such as applying the rule of Eq. 7 to the predicted maps without the MLLM, or relabeling on-policy states with Eq. 7 in the style of DAgger (Ross et al., 2011, cited in the paper). The columns of Table 4 are not defined (what counts as an error or a recovery, how many cases, which datasets), and the table includes only external agents, not M1 to M3, so the higher recovery rate cannot be attributed to COAR. Its Error column reads 22.5% for REVIVE and 22.3% for IBISAgent, although the abstract claims reduced error-action misalignment.
  3. The description of the method is incomplete and partly inconsistent with the experiments. The MLLM backbone, the stopping threshold, the quality measure S, the verifier architecture and the maximum number of rounds are not given. The action space is defined as positive click, negative click and stop (L206-208), but Table 4 has a BBox column that reads 100% for REVIVE and a Point column that reads 29.1%; how the initial mask is obtained is never explained. In Eq. 13 the STOP score is a margin to a threshold while the click scores are one-step gains, so for a mask below the threshold a click that lowers Dice can still be ranked above STOP. Fig. 1 shows the agent comparing the outcomes of the alternative actions and selecting the best one, which needs the ground truth and can therefore happen only during training. It is unclear what the "verify" step consists of at test time.
Questions for authors
  1. Were the baselines fine-tuned on the same training data or run with the released checkpoints? IBISAgent, whose protocol is followed (L376), fine-tuned the general-purpose baselines on its own training data. Could MedSAM-Agent and IBISAgent be retrained on the same split, and MedSAM with the initial prompt alone be added?
  2. What text prompt and target did each baseline receive? On CAMUS, MedSAM-Agent and IBISAgent have precision above 0.9 but recall of about one third, and they score below LISA++ and SAM4MLLM, which segment almost the whole image.
  3. What are the training sets and test splits for the seven ultrasound datasets, which MLLM is used, and what are the stopping threshold, the measure S and the maximum number of rounds? Is the tool the box-prompted MedSAM of Ma et al. (2024) or MedSAM2, as in MedSAM-Agent and IBISAgent, and how are clicks passed to it?
  4. How accurate are the predicted FN/FP maps on test images, and what is the result when they are used directly (clicking at the largest predicted component) without the MLLM policy? In COAR, how are the click positions of the non-selected action types obtained (mentioned only in the conclusion, L474-475), and how often does the outcome-based ranking differ from the rule of Eq. 7?
Reviewer_nmFmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes REVIVE, an interactive ultrasound segmentation agent that combines explicit error diagnosis with multiround on-policy recovery learning. A residual-error predictor estimates false-negative and false-positive regions to guide action selection, while a ranking objective compares alternative actions at the same state and complements trajectory-level reinforcement learning. Experiments on ultrasound and polyp segmentation datasets evaluate segmentation performance and recovery from erroneous actions.

Critical strengths
  1. The concept of counter-factual and false-positive or fals-negative region residual error is good and important for medical image segmentation.
  2. The results of proposed method is significantly better than the MedSAM agents in the target datasets.
Critical weaknesses
  1. Limited methodological novelty. The multi-turn RL framework and reward design largely follow MedSAM-Agent. The additional contributions mainly consist of explicitly modeling false-negative and false-positive residual regions, conditioning action selection on these predictions, and introducing a pairwise ranking objective. The ranking loss itself is standard rather than a novel optimization mechanism. The paper does not sufficiently establish why this particular combination is preferable to simpler alternatives, such as directly supervising the best candidate action under the same training conditions.

  2. Action-type preferences are confounded by spatial proposal quality. Although the action space contains three categories—POS, NEG, and STOP—the effectiveness of POS and NEG also depends critically on their specific click coordinates. COAR evaluates instantiated actions but applies its ranking supervision to action-type logits. Consequently, an unfavorable outcome may reflect a poorly chosen coordinate rather than an inappropriate action type. The current experiments do not disentangle improvements in spatial proposals from improvements in action selection through RL. Further controlled analyses, such as fixing the coordinate proposal mechanism or evaluating multiple coordinates per action type, are needed to establish what COAR actually learns.

  3. The large discrepancy between the GRPO baseline and MedSAM-Agent is insufficiently explained. On BUS-BRA, the GRPO-only baseline in Table 3 achieves a Dice score of 0.7657, substantially exceeding the 0.5431 reported for MedSAM-Agent in Table 1, even before either proposed component is introduced. This 22.26-percentage-point difference is considerably larger than the 6.85-point improvement obtained by adding both components. The paper should clarify differences in backbone, training data, initialization, and interaction budget, and provide a matched comparison. Without this clarification, the large gains in the main comparison cannot be confidently attributed to the proposed method.

Questions for authors
  1. The proposed method seems quite general and can ben applied to other domain of medical segementation. Why limited on Ultra-Sound modality?
Reviewer_ENbYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies action selection and recovery in multi-round interactive ultrasound segmentation. REVIVE freezes MedSAM and trains a Residual-Error Verifier to predict FN/FP residuals in the current mask. Attention extracts evidence for each action type, which the MLLM uses to choose a positive click, a negative click, or STOP. During training, COAR executes alternative actions from the same policy-induced state, ranks them using segmentation outcomes and a stopping threshold, and trains jointly with multi-turn GRPO. The experiments cover ultrasound and polyp segmentation, BUS-BRA component ablations, and error recovery. They show gains in these settings, but leave open how much REVIVE adds to existing error prediction, automated clicking, and policy improvement methods, and whether the added complexity is justified.

Critical strengths
  1. Table 3 provides a convincing ablation on a common GRPO baseline. BUS-BRA Dice increases from 0.7657 to 0.7801 with diagnosis, 0.8086 with COAR, and 0.8342 with both. This supports individual and combined gains more directly than comparisons between different systems.
  2. Recovery after poor clicks matters for automated segmentation. REVIVE achieves the highest Dice on six of seven ultrasound datasets and also tests polyp segmentation (Table 2). Overall and next-interaction recovery reach 88.0% and 85.8% (Table 4). These results support the systems contribution.
Critical weaknesses
  1. The methodological advance remains unclear. PseudoClick already predicts FP/FN maps and generates corrective clicks; IBISAgent already uses FN/FP-aware click and progressive-improvement rewards. Neither implements the full REVIVE pipeline, but its clearest distinction is comparing candidate actions at the same state. Component ablations on one dataset do not establish the broader contribution or scope of this combination.
  2. The added complexity lacks adequate justification. Simpler controls, such as direct residual-map clicking or best-action supervision under identical localization, data, and tools, are missing. Extra tool calls and training/inference costs are unreported. The verifier depends on MedSAM internals, yet transfer across tools is untested.
  3. Table 4 underspecifies errors, recovery criteria, denominators, and recovery horizons. Whether each policy's error states are comparable in difficulty is also unclear.
Questions for authors
  1. What capability is new relative to PseudoClick and existing medical agents? Under identical initial localization, data, and a frozen tool, have you compared direct residual-map clicking or best-action supervision from the same candidate outcomes?
  2. Please specify the base MLLM, tool version, ultrasound splits, sample counts, and baseline retraining. Were prompts and interaction limits matched? Report extra candidates per training state, training time, latency, and average tool calls. Do gains persist under matched conditions?
  3. How are POS/NEG coordinates generated, how many are sampled per type, and how are Eq. 14's categorical scores computed? How does coordinate quality affect type supervision? Eq. 7 stops at the threshold; Eq. 13 may still favor clicking. What threshold is used, and how are these rules reconciled?
  4. How does Table 4 define errors, recovery, denominators, sample counts, and maximum recovery rounds? Are comparisons on separate trajectories or common error states? Are verifier FP/FN prediction results and variation across repeated runs available?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.