- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies whether pretrained multimodal models can perform medical image segmentation without task-specific training. It proposes SegBanana, which combines a frozen image generation model with retrieval of annotated examples, candidate comparison, and a controller that tracks previous attempts. Based on feedback, the controller decides whether to generate a new mask, refine an existing one, or stop. Experiments on eight datasets report an average mDice of 77.45%, compared with 57.69% for direct generation, exceeding the evaluated baselines on average.
Critical strengths
-
Clear and informative visualizations. The framework overview and step-by-step case study make the inference process easy to follow. The qualitative comparisons also illustrate how the method handles different anatomical structures and segmentation errors.
-
Well-motivated method design. The preliminary experiments identify limitations of direct generation and examine how retrieved examples, diverse sampling, and targeted editing can improve results. These observations provide a clear rationale for the proposed components.
Critical weaknesses
-
Unclear source of improvement. Without a fixed workflow matched for sampling budget, the gains from adaptive control cannot be separated from those of additional computation.
-
Insufficient baseline details. Near-zero results for some baselines warrant closer validation of prompts, preprocessing, and output handling.
-
Inconclusive ablations. The critic ablation retains evaluation within the controller, and the state ablation retains full history, limiting conclusions about their necessity.
-
Unquantified critic reliability. Incorrect candidate selection and premature stopping are not measured, despite the gap between selected and oracle results.
-
Incomplete reproducibility and cost reporting. Sample counts, support-pool construction, repeated-run variability, and total inference cost are insufficiently documented.
Questions for authors
The framework currently relies on Nano Banana 2 as its UMM backbone. Can it be adapted to other UMMs, such as GPT-image2?
4 comments
Weakness 4: Candidate Selection and Premature Stopping
Reviewer Question: "Unquantified critic reliability. Incorrect candidate selection and premature stopping are not measured, despite the gap between selected and oracle results."
Response: Candidate selection: In approximately 62% of cases, the selected mask’s Dice score was within 0.5 points of the oracle score. In only 7% of cases did the gap exceed 5 points. Averaged equally across the eight datasets, selected and oracle mDice were 77.45% and 81.06%, respectively, a selection loss of 3.61 points.
Premature stopping: The controller stopped at the first iteration in 46.0% of cases, at the second in 14.0%, and at the third in 40.1%. To assess whether continuing would improve the result, we forced one additional EDIT after stopping on the premature samples. The additional EDIT step produced only small changes in dataset-level mean mDice on these cases.
| Metric | TNBC | RAVIR | Drishti-GS | ISIC | BUS-UCLM | Kvasir | ACDC | BraTS |
|---|---|---|---|---|---|---|---|---|
| Selected mDice (%) | 74.75 | 80.93 | 87.09 | 82.06 | 78.07 | 89.53 | 61.68 | 65.52 |
| Oracle mDice (%) | 77.99 | 81.83 | 92.79 | 87.52 | 82.51 | 89.74 | 65.94 | 70.17 |
| Selection loss (pp) | 3.24 | 0.90 | 5.70 | 5.46 | 4.44 | 0.21 | 4.26 | 4.65 |
| mDice at stopping (%) | 73.89 | 81.89 | 63.73 | 88.85 | 83.76 | 89.62 | 76.69 | 77.24 |
| mDice after extra EDIT (%) | 73.84 | 81.92 | 64.56 | 89.60 | 84.20 | 89.68 | 77.02 | 76.40 |
| Δ mDice (pp) | −0.05 | +0.03 | +0.83 | +0.75 | +0.44 | +0.06 | +0.33 | −0.84 |
Weakness 5: Reproducibility and Inference Cost
Reviewer Question: "Incomplete reproducibility and cost reporting. Sample counts, support-pool construction, repeated-run variability, and total inference cost are insufficiently documented."
Response: Repeated-run variability. Across three independent runs on the same test cases, the mean macro-averaged mDice was 76.79%, with a sample standard deviation of 0.63 percentage points and a sample variance of 0.40 squared percentage points.
| Method | Setting | TNBC | RAVIR | Drishti-GS | BUS-UCLM | Kvasir | ACDC | ISIC | BraTS | Avg. |
|---|---|---|---|---|---|---|---|---|---|---|
| BiomedParse | - | 11.81 | 5.67 | 78.19 | 63.81 | 88.95 | 89.03 | 87.59 | 75.11 | 62.52 |
| MedSAM3 | - | 68.21 | 4.21 | 34.13 | 57.68 | 91.17 | 16.74 | 89.80 | 55.42 | 52.17 |
| SegBanana | Run 1 | 74.75 | 80.93 | 87.09 | 78.07 | 89.53 | 61.68 | 82.06 | 65.52 | 77.45 |
| SegBanana | Run 2 | 72.61 | 77.35 | 87.95 | 76.55 | 88.28 | 60.67 | 82.69 | 63.45 | 76.19 |
| SegBanana | Run 3 | 69.05 | 80.45 | 86.59 | 76.64 | 89.06 | 60.86 | 85.40 | 65.83 | 76.74 |
| SegBanana | Mean ± SD | 72.14 ± 2.88 | 79.58 ± 1.94 | 87.21 ± 0.69 | 77.09 ± 0.85 | 88.96 ± 0.63 | 61.07 ± 0.54 | 83.38 ± 1.77 | 64.93 ± 1.29 | 76.79 ± 0.63 |
Evaluation splits, sample counts, support-pool construction, and inference cost: We used the original splits when test annotations were public. Otherwise, we held out at least 20% of labeled data, splitting by patient when identifiers were available. For 3D MRI, we randomly sampled two slices per patient. In total, we evaluated 1,945 images or slices across the eight datasets. At each retrieval step, we searched all annotated image–mask pairs in the target training set (in-domain) or a training set matched by modality and task (OOD), selecting two pairs based on query similarity and diversity. The supplement lists the support dataset for each target. For cost estimation, we rounded up the average numbers of generator calls and iterations per image to 4 and 2, respectively. Assuming approximately five Gemini calls per iteration for the controller and critic combined, this gives an estimated 10 Gemini calls per image. At USD 0.08 per generator call and USD 0.02 per Gemini call, the estimated cost was USD 0.52 per image.
Question 6: Alternative UMM Backbone
Reviewer Question: "The framework currently relies on Nano Banana 2 as its UMM backbone. Can it be adapted to other UMMs, such as GPT-image2?"
Response: Yes. We evaluated SegBanana with GPT-image-2 as the generator across all eight datasets. Mean Dice increased from 49.13 for the standalone generator to 68.87 with SegBanana, a gain of 19.74 Dice points.
| Method | TNBC | RAVIR | Drishti-GS | ISIC | BUS-UCLM | Kvasir | ACDC | BraTS | Avg. |
|---|---|---|---|---|---|---|---|---|---|
| BiomedParse | 11.81 | 5.67 | 78.19 | 87.59 | 63.81 | 88.95 | 89.03 | 75.11 | 62.52 |
| MedSAM3 | 68.21 | 4.21 | 34.13 | 89.80 | 57.68 | 91.17 | 16.74 | 55.42 | 52.17 |
| GPT-image-2 Dice | 37.04 | 32.73 | 45.83 | 63.60 | 44.59 | 80.50 | 38.08 | 50.64 | 49.13 |
| SegBanana (GPT-image-2) Dice | 53.67 | 73.49 | 76.40 | 77.97 | 67.32 | 87.59 | 62.76 | 51.78 | 68.87 |
Thank the authors for the response. Good paper! I keep my score unchanged.
Thank you for your thoughtful assessment and detailed feedback. Your suggestions prompted us to add a fixed-budget baseline and examine how well SegBanana selects masks and decides when to stop. We will include these results in the revised manuscript and describe the evaluation protocol more clearly.
Again, we sincerely appreciate the time you devoted to reviewing our work.
We sincerely thank the reviewer for the thoughtful review and for highlighting the clarity of our visualizations and the role of the preliminary experiments in motivating SegBanana's design. We also appreciate the suggestions for isolating the contribution of adaptive control under a matched sampling budget, strengthening the baseline and ablation studies, and quantifying the critic's reliability.
Below, we address the comments and question point by point.
Weakness 1: Source of Improvement Under a Fixed Sampling Budget
Reviewer Question: "Unclear source of improvement. Without a fixed workflow matched for sampling budget, the gains from adaptive control cannot be separated from those of additional computation."
Response: We add a fixed-budget sampling baseline. For each dataset, we rounded up SegBanana's average number of generator calls and used that integer as the baseline's budget for every image. We applied the same oracle criterion to all methods, scoring the best candidate mask for each image against the ground truth. SegBanana reached 81.06, outperforming the fixed-budget baseline (66.86) on all eight datasets by an average of 14.20 Dice points while using fewer generator calls on average (3.93 vs. 4.50 per image). Thus, SegBanana achieved higher Dice with fewer generator calls, highlighting the benefit of adapting refinement to each sample's difficulty.
Weakness 2: Validation of Near-Zero Baseline Scores
Reviewer Question: "Insufficient baseline details. Near-zero results for some baselines warrant closer validation of prompts, preprocessing, and output handling."
Response: We checked the prompts, preprocessing, and conversion of model outputs to evaluation masks for all baselines with near-zero scores and found no implementation errors in these steps. Among the medical-specific methods, low scores were task-specific. MedSAMAgent scored 0.30 mDice on RAVIR and 83.47 on ISIC, while IBISAgent scored 1.00 on TNBC and 87.47 on ISIC. This variation suggests difficulty generalizing across imaging modalities and target structures. Among the general-purpose segmentation models, SAM 3 scored 56.65 mDice on ACDC and 0.00 on TNBC, RAVIR, Kvasir, and BraTS, where it returned no mask for nearly all target-positive images we inspected. We suspect it has difficulty linking specialized anatomical terms to the corresponding image structures. Kong et al. [1] similarly found that text prompts alone often failed to identify nuclei in pathology images.
[1] Kong Q, Sharifi S, Wang Y, et al. Is SAM3 ready for pathology segmentation?[J]. arXiv preprint arXiv:2604.18225, 2026.
Weakness 3: Contribution of the Independent Critic and Explicit State
Reviewer Question: "Inconclusive ablations. The critic ablation retains evaluation within the controller, and the state ablation retains full history, limiting conclusions about their necessity."
Response: Without evaluation from either the critic or the controller, SegBanana has no feedback to guide refinement and can only sample repeatedly. As reported in Weakness 1, fixed-budget repeated sampling achieved 66.86 mean oracle Dice, compared with 81.06 for SegBanana, despite using more generator calls per image (4.50 vs. 3.93). To isolate the contribution of the independent critic, we therefore retained refinement and assigned candidate evaluation to the controller. Under this ablation, mean Dice fell from 77.45 to 72.33. Similarly, the explicit-state ablation retained the full interaction history so the agent could still use prior feedback. We will clarify in the main text what the independent critic adds beyond controller self-evaluation and what the explicit state adds beyond access to the full history.