acceptodds
Under review as a conference paper at ICLR 2027

Knowing When Not to Segment: Label-Free Adaptation in Free-Hand Ultrasound

Abstract

Self-training and other label-free adaptation methods for segmentation usually assume that every image contains the target structure, but in videos and sweeps the structure is often out of view. Wireless handheld probes bring ultrasound out of the hospital, but many of their users are not medical professionals and cannot judge whether a model's prediction is correct. Segmentation models trained on public data from cart-based scanners segment structures in handheld images well when they are in view, but they do not know when not to segment. We study how to teach such a model, without target labels, to predict nothing when a structure is not in the frame. We built a dataset of free-hand thyroid sweeps with a wireless handheld probe, which we will release. It contains 20,014 frames from 29 adults, 232 of which are annotated for thyroid, carotid artery and trachea. Of these, 47% show no identifiable target structure, and two clinicians annotated the 58 held-out frames of its hidden test set. Our experiments reveal two problems. First, current evaluation measures only how accurately a structure is drawn and ignores frames without it, so it cannot show whether a model chooses not to draw when the target structure is absent. Second, because existing models are trained on datasets in which nearly every image contains the target structure, they also draw it on frames where it is absent. We therefore propose an evaluation that also scores predicting nothing. Frames with a structure are scored by how accurately it is drawn and frames without it by whether nothing is drawn, with equal weight, so predicting nothing scores 0.50. We also add anatomical constraints to self-training. Predicted structures are checked against rules on their position, shape, echogenicity and relations to the other structures, and implausible predictions are not used for training. The model thereby learns to predict nothing. On the evaluation and held-out frames, false thyroid detections fall from 99% and 90% to 40% and 29%, and the score reaches 0.59 and 0.61, significantly above predicting nothing and above the seven other adaptation methods that start from the same source models, while mean segmentation accuracy on frames with the structure does not drop.

Primary area:
datasets and benchmarks
Keywords:
self-training, source-free domain adaptation, test-time adaptation, selective prediction, medical image segmentation, ultrasound, dataset

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2WFKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a dataset and benchmark for label-free adaptation in free-hand ultrasound. Balanced Dice gives equal weight to segmentation accuracy on frames with a structure and predicting nothing on frames without it. Persist & fit uses agreement between adjacent frames, anatomical rules, and carotid completion to prepare pseudo-labels for self-training. At inference, the trained network needs no rules, plane classifier, or neighbouring frames.

Critical strengths
  1. The paper shows that evaluation limited to frames with a structure can hide frequent false detections on frames without it.
  2. It provides a systematic analysis supported by ablations, failure analysis, and robustness checks. Code, predictions, and annotations are also provided.
Critical weaknesses
  1. The evaluation limits the benchmark’s conclusions. Section 4 distinguishes absent from uncertain structures, but the main score in Table 3 combines them. Results excluding uncertain cases preserve the main trends, but claims about true absence should be based on those results. The held-out frames are also concentrated near the start and end of sweeps (Appendix P), leaving limited evidence about performance throughout complete sweeps.

  2. The main comparisons need clearer interpretation and cost evidence. Table 3 shows no consistent balanced Dice advantage for Persist & fit over self-training with rules at test time. Section 5 acknowledges this. The stated advantage is deployment without rules or a plane classifier, but its cost benefit over those baselines is not directly measured. The supervised reference also uses different source training (Appendices E and I), and the stronger source models already have higher carotid Dice before self-training (Table 3).

Questions for authors

Clear answers supported by evidence could change my recommendation.

  1. Do the main conclusions hold for test frames sampled throughout complete sweeps, rather than mainly near their start and end?
  2. Is the intended task to identify true absence or to avoid segmentation when visibility is uncertain? Why is the score that combines these cases used as the main score?
  3. The supervised reference and the stronger self-training baselines use different training data. How should we interpret their results in Table 3?
  4. What is the accuracy and cost trade-off between Persist & fit and self-training with rules at test time? Please compare total inference time and adaptation cost under the same hardware and input settings.
Reviewer_HbNXOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This authors try to address an interesting problem in medical image segmentation, that is to find when the targeted anatomical structure is absent and the model should produce an empty segmentation mask. The authors proposed persist&fit, a label-free method that reduces false-positive segmentations in thyroid ultrasound. Experiments on a self-collected ultrasound dataset demonstrate reductions in false detections compared with conventional adaptation methods.

Critical strengths
  1. The paper targeted a meaningful limitation of current domain adaptation methods in segmentation, reducing false-positive predictions is particularly important for ultrasound AI deployment in real-world settings.

  2. The overall paper is easy to follow and the authors compared propsed persist&fit with multiple test-time adaptation, source-free domain adaptation, and foundation model baselines.

  3. The proposed persist&fit requires no target-domain annotations and produces a single network for inference without additional anatomy.

  4. The newly collected handheld ultrasound dataset will be released

Critical weaknesses
  1. The proposed method is based on existing approaches, including temporal consistency, and self-training of teacher and student, and the anatomical constraints are manually specified. The methodological contribution is somehow limited.

  2. Although additional experiments are conducted on SegThy, the self-collected dataset contains 29 participants acquired using a single ultrasound probe, with 232 frames, which is small.

Questions for authors
  1. In Table 3, persist&fit performs worse than self-training+ post-processingn, or self-training (augmentation) + rules, the overall results do not showed a consistent advantage of the proposed persist&fit. How should we understand this phenomenon.
Reviewer_fxe6OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The manuscript motivates AI-guided thyroid ultrasound with wireless handheld probes in community screening, where users are often not experienced in the medical field to judge whether a segmentation is correct. Therefore, the manuscript argues that it is important that the model knows when not to segment. The manuscript claims that segmentation models in literature are mostly trained on data where nearly every image contains the target, and due to this segments the target structure on frames where it is not present.

The first public dataset, which was acquired with a wireless handheld-probe thyroid sweeps, is introduced in the manuscript. It contains 232 annotated frames of three target anatomical structures: thyroid, carotid artery and trachea, of these frames 47% of the annotated frames show none of the target structures.

The presented method persist & fit is proposed in the manuscript, which is a label-free adaptation method with a teacher-student setup. Persist and fit filters teacher’s pseudo labels in 2 steps, first it uses temporal consistency across neigbouring frames and then it makes use of hand-crafted anatomical rules based on connected-component properties.

The manuscript also provides a experimental analysis of common label-free adaptation methods and promptable foundation models and compares these to the persist and fit method by evaluating them with the Balanced DICE score.

Critical strengths

Critical strength: Relevant & Underexplored problem. Why it is consequential to my overall assessment: The identification of failure modes in this area is important: models that are trained on imbalanced data containing relevant structures but also frames where they are absent is an often occurring property in real-world data. Thus, handling this scenario is clearly motivated and practical, as motivated with the application example of guiding users without any medical expertise to use handheld wireless ultrasound correctly.

Critical weaknesses

Critical weakness: The contributions are very limited due to the very specific application area and missing possibility for generalization. Why it is consequential to my overall assessment: The manuscript presents hand-crafted anatomical rules which are only designed for the neck anatomy in hand-held ultrasound and transfer is only tested to SegThy. Thus, it is not shown that the presented methodology can be used on a different anatomy or in other medical contexts, which narrows the potential impact to the domain of hand-held ultrasound of the neck area, which itself is rather limited. Coupled with the results of persist and fit beating baselines with only a very slim margin and a non-significant gain when it comes to the comparison of source models, the contribution is limited further. A very related baseline [A] (only mentioned in appendix, not in realted work) is a source-free, backprop-free TTA method for ultrasound segmentation, which was also evaluated on thyroid data, which would match protocol for the systematic analysis in main result Tables 3 and 5. Although cited, it is still not compared against. Minor, but related work such as [B] should be cited in related work (in context of anatomical-priors, line 141).

Critical weakness: Limited reliability of some claims and suboptimal structure of the manuscript. Why it is consequential to my overall assessment: The experiments have been tested on a small unseen held-out test set of 58 samples, which is extremely small to reliably assess the efficacy of the presented method and compare it to baselines. What limits the contributions further is the very limited overall dataset size of 232 annotated ultrasound frames. The manuscript further claims an improved evaluation metric as contribution (i.e., evaluation scheme where standard evaluation fails), yet, a clear comparison is only provided in the supplemental material (Table 5, 6), and is not experimentally substantiated in the main paper. Similarly, experimental analysis on the SegThy dataset, which is claimed as main contribution (line 98, experiments on "second target domain") are merely supplied in the appendix, which for substantiating a contribution should be discussed prominently in the main paper. In related work (section 2), the manuscript references future sections including results from the evaluation section (section 5) which interrupts the reading flow, showcasing that the overall structure of the manuscript could be improved (cf. lines 138, 141 and 149).

References:

[A] Wang, W., Zhou, J., Zhang, C., Xing, W., Fan, S., & Qu, X. (2026). Prototype bank‐driven test‐time adaptation for medical ultrasound image segmentation. Medical Physics, 53(1), e70280.

[B] Zhang, R., Lu, W., Guan, C., Gao, J., Wei, X., & Li, X. (2024, October). Shan: Shape guided network for thyroid nodule ultrasound cross-domain segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention (pp. 732-741). Cham: Springer Nature Switzerland.

Questions for authors

Q1 Missing baselines Prototype-bank TTA for ultrasound [A] is cited but listed as not run. Could you explain why these are excluded from the experiments?

Q2 Effort for possible generalization How hard would it be to define similar anatomical constraints for anatomical areas, in which target structures are less rigidly arranged than in the neck?

Q3 Exact target use case The manuscript motivates the work with bringing AI-guided ultrascan to users without any medical expertise in community settings. I am interested in what exact clinical use case is addressed? And how well do sweeps acquired by the research team represent scans performed by inexperienced users?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.