- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents ExpARSeg for medical image segmentation with limited source annotations and transfer to unseen datasets. It uses local pixel–text compatibility to control bidirectional feature updates in frozen vision and text encoders. A two-stage refinement module selects locations with low prediction margins and applies text-conditioned corrections using local image details. The model is trained with segmentation losses and a target-conditioned KL regularizer. Experiments span four imaging modalities and ten target datasets. ExpARSeg achieves the highest DSC in 11 of 12 source-data settings and the highest average target DSC in three of four modality groups.
Critical strengths
- The EPT adapter provides a concrete way to make cross-modal updates depend on local pixel–text compatibility. It uses foreground support and commitment to route the exchanged features, giving adaptation an explicit task-related bias. This is a useful design contribution for learning from limited mask supervision.
- The evaluation spans four imaging modalities and ten held-out target datasets, with a shared pretrained backbone for CLIP-based methods. Table 1 lists improvements across all four modalities at the 10% annotation budget. This provides useful evidence that the approach can benefit segmentation with limited annotations across several tasks.
Critical weaknesses
- Appendix A.1 states that prompt construction uses attributes from images, masks, and labels. All text-based methods share these prompts, so this does not invalidate their comparison. My concern is whether ExpARSeg still performs better when such descriptions are unavailable. The generic-prompt experiment helps, but it reports recall at a controlled background false-positive rate rather than DSC. It does not show whether the reported segmentation gains remain with generic prompts.
- Table 2 lists transfer results only at the 100% source annotation budget. It remains unclear whether ExpARSeg also outperforms the main baselines on target datasets when trained with 10% or 50% of the source annotations. Figure 5 provides useful feature analyses at these budgets, but does not answer this segmentation question.
Questions for authors
- With generic task prompts replacing the supplied descriptions at test time, what are the target DSC scores for ExpARSeg and MedCLIPSeg under the same evaluation protocol as Table 2? The recall analysis in Figure 11 does not show whether the segmentation advantage remains.
- Does the mean DSC in Figure 6 include target datasets? Please clarify which datasets it averages over. Are target DSC comparisons with the main baselines available for models trained with 10% or 50% of the source annotations?
- How well does SSR recover lesions that receive confidently negative coarse margins on target datasets? Its point selection favors small absolute margins, which may leave these missed regions unselected.
6 comments
Response to Reviewer UYdk [Q2]: Segmentation Performance under Different Annotation Budgets
Q2. Provide comparisons for models trained with 10% or 50% of the source annotations.
Figure 6 reports the mean DSC on the source test sets of BUSI, Kvasir, BTMRI, and ISIC, separately for each modality using 10% of the source training annotations. Mean DSC refers to that each value is averaged over the test images within the corresponding dataset. Target datasets are not included.
Here we provide DSC comparisons with the strongest baseline MedCLIPSeg under different annotation budgets. Results are as follows:
10% Data:
| Method | MedCLIPSeg | ExpARSeg |
|---|---|---|
| Overall | 70.42±0.71 | 73.07±0.66 |
| BUSI | 57.73±2.15 | 69.76±1.26 |
| BTMRI | 83.24±0.16 | 82.51±1.90 |
| Kvasir-SEG | 75.37±2.41 | 80.67±1.30 |
| ISIC | 90.45±0.17 | 90.56±0.29 |
| BUSUC | 77.68±1.77 | 76.09±3.85 |
| BUSBRA | 58.03±2.93 | 58.99±6.14 |
| BUID | 69.98±2.35 | 66.26±3.15 |
| UDIAT | 67.13±0.73 | 70.19±2.52 |
| CVC-ColonDB | 53.17±2.33 | 58.46±2.61 |
| CVC-ClinicDB | 64.19±0.88 | 68.89±1.55 |
| CVC-300 | 73.09±4.78 | 76.61±0.86 |
| BKAI | 63.12±0.56 | 65.49±1.98 |
| BRISC | 68.51±9.22 | 75.74±1.52 |
| UWaterlooSkinCancer | 84.18±0.42 | 82.80±2.11 |
50% Data:
| Method | MedCLIPSeg | ExpARSeg |
|---|---|---|
| Overall | 78.89±1.23 | 80.28±0.35 |
| BUSI | 80.37±0.66 | 80.49±0.60 |
| BTMRI | 87.01±0.19 | 87.90±0.24 |
| Kvasir-SEG | 87.97±0.54 | 87.97±0.33 |
| ISIC | 91.76±0.42 | 92.19±0.27 |
| BUSUC | 81.54±1.35 | 82.01±2.16 |
| BUSBRA | 66.38±6.64 | 74.46±1.79 |
| BUID | 75.74±3.43 | 79.88±1.74 |
| UDIAT | 72.32±3.84 | 73.57±3.38 |
| CVC-ColonDB | 68.43±0.61 | 68.10±0.64 |
| CVC-ClinicDB | 76.67±1.25 | 76.53±1.18 |
| CVC-300 | 79.80±0.79 | 80.64±1.36 |
| BKAI | 74.42±1.37 | 74.65±2.56 |
| BRISC | 77.69±1.71 | 80.50±1.52 |
| UWaterlooSkinCancer | 84.34±0.61 | 85.09±2.43 |
100% Data:
| Method | MedCLIPSeg | ExpARSeg |
|---|---|---|
| Overall | 80.11±1.32 | 81.98±0.42 |
| BUSI | 84.09±0.92 | 85.08±1.10 |
| BTMRI | 88.69±0.04 | 88.98±0.27 |
| Kvasir-SEG | 89.95±0.59 | 89.81±0.15 |
| ISIC | 92.33±0.02 | 92.59±0.04 |
| BUSUC | 79.71±2.60 | 83.60±1.88 |
| BUSBRA | 67.04±3.31 | 74.87±1.18 |
| BUID | 76.80±4.71 | 80.12±1.49 |
| UDIAT | 72.88±2.68 | 75.29±4.11 |
| CVC-ColonDB | 72.83±0.58 | 73.17±0.59 |
| CVC-ClinicDB | 80.21±0.42 | 80.85±0.15 |
| CVC-300 | 82.96±0.56 | 83.39±0.67 |
| BKAI | 77.89±1.49 | 79.48±1.20 |
| BRISC | 77.75±4.62 | 78.48±4.74 |
| UWaterlooSkinCancer | 78.49±4.24 | 82.03±2.41 |
With 10% source annotations, ExpARSeg averages 69.95 DSC across the ten target datasets, compared with 67.91 for MedCLIPSeg. This is a gain of 2.04 percentage points (pp), with ExpARSeg ahead on 7 of 10 targets. All four endoscopy targets improve: CVC-ColonDB by 5.29 pp, CVC-ClinicDB by 4.70 pp, CVC-300 by 3.52 pp, and BKAI by 2.37 pp. The largest gain is on BRISC (+7.23 pp), while UDIAT and BUSBRA improve by 3.06 pp and 0.96 pp. The gains therefore extend across several target datasets and modalities at this extremely limited annotation budget.
With 50% source annotations, the target mean DSC is 77.54 for ExpARSeg and 75.73 for MedCLIPSeg, giving an advantage of 1.81 pp. The largest improvements are on BUSBRA (+8.08 pp) and BUID (+4.14 pp), followed by BRISC (+2.81 pp). Smaller gains occur on UDIAT, BUSUC, CVC-300, BKAI, and UWaterlooSkinCancer. The two datasets where MedCLIPSeg remains ahead are CVC-ColonDB and CVC-ClinicDB, with differences of just 0.33 pp and 0.14 pp. Thus, the higher target average is accompanied by gains on most individual datasets.
With 100% source annotations, ExpARSeg reaches 79.13 mean target DSC versus 76.66 for MedCLIPSeg, with an improvement of 2.47 pp. It leads on all ten target datasets, with gains ranging from 0.34 to 7.83 pp. These include 7.83 pp on BUSBRA, 3.89 pp on BUSUC, 3.54 pp on UWaterlooSkinCancer, and 3.32 pp on BUID.
To sum up, ExpARSeg achieves higher average target DSC at all three annotation budgets and leads on 25 of the 30 target comparisons settings. Its advantage extends from scarce source annotations to full supervision, with gains on most targets at 10% and 50% and on all targets at 100%.
Response to Reviewer UYdk [Q3]: Effect of SSR on Confidently Negative Regions
Q3. Whether the point selection of SSR only favors small absolute margins?
SSR can update a missed region through corrections applied at neighboring locations. Half-resolution corrections are interpolated onto the full-resolution grid, allowing their effects to reach pixels beyond the directly selected points. The updated margins then guide the next round of point selection. An initially confident false negative within this spatial support can therefore receive a correction and become a candidate for further refinement. At both stages, fine visual details and text conditioning provide information for revising the coarse prediction.
The BRISC and UWaterloo target cases in Figure 8 show how this refinement improves lesion coverage. Parts of the annotated lesions are absent from the coarse masks but recovered in the final predictions, with the cyan regions showing where SSR changes the predicted labels. The final masks extend into the missed foreground and follow the GT contours more closely. These examples support SSR’s ability to recover missed lesion regions through its two-stage spatial refinement.
Response to Reviewer UYdk [Q1]: Segmentation Performance using Generic Task Prompts
We thank the reviewer for the precise and well-argued review, and we hope our responses can deal with the main concerns.
Q1. Whether the reported segmentation gains remain with generic prompts?
The original benchmark in our main submission allows us to compare adaptation methods under a common level of textual assistance. Now we extend the generic-prompt evaluation protocol in Figure 11 to further assess final segmentation quality using DSC. Specifically, we predefine six templates using only the imaging modality and segmentation target for each modality, such as “Segment the polyp regions in this endoscopic image.” All methods share the same pool across annotation budgets, seeds, and source and target datasets. Template construction, assignment, and weighting are independent of patient annotations and validation results. At deployment, the task and modality determine the pool, so the same instructions can be reused for every patient. We have also included results from two extra random seeds for a more comprehensive evaluation. Comparisons between our method and the strongest baseline MedCLIPSeg are as follows:
10% Data:
50% Data:
100% Data:
Averaged over the ten target datasets, ExpARSeg obtains DSC scores of 68.29±0.38, 74.44±0.48, and 76.65±0.46 at the 10%, 50%, and 100% annotation budgets, respectively, while MedCLIPSeg obtains 66.10±1.52, 73.49±0.22, and 75.98±0.32. ExpARSeg has the higher three-seed mean on 9/10, 9/10, and 8/10 target datasets, with the largest average gain at 10% annotations. Examples at this budget include BUID, where DSC rises from 64.48 to 69.32, CVC-300, from 71.10 to 74.17, and BRISC, from 70.82 to 73.46. We also report performance on the four source datasets, where ExpARSeg continues to lead across the board. In summary, the results of this supplementary experiment show that ExpARSeg achieves higher DSC than MedCLIPSeg on most target datasets when both methods are trained and tested with generic task prompts. Its advantage therefore extends to a practical setting where prompts specify the segmentation task and require no patient-specific annotations.