- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents a dataset and benchmark for label-free adaptation in free-hand ultrasound. Balanced Dice gives equal weight to segmentation accuracy on frames with a structure and predicting nothing on frames without it. Persist & fit uses agreement between adjacent frames, anatomical rules, and carotid completion to prepare pseudo-labels for self-training. At inference, the trained network needs no rules, plane classifier, or neighbouring frames.
Critical strengths
- The paper shows that evaluation limited to frames with a structure can hide frequent false detections on frames without it.
- It provides a systematic analysis supported by ablations, failure analysis, and robustness checks. Code, predictions, and annotations are also provided.
Critical weaknesses
-
The evaluation limits the benchmark’s conclusions. Section 4 distinguishes absent from uncertain structures, but the main score in Table 3 combines them. Results excluding uncertain cases preserve the main trends, but claims about true absence should be based on those results. The held-out frames are also concentrated near the start and end of sweeps (Appendix P), leaving limited evidence about performance throughout complete sweeps.
-
The main comparisons need clearer interpretation and cost evidence. Table 3 shows no consistent balanced Dice advantage for Persist & fit over self-training with rules at test time. Section 5 acknowledges this. The stated advantage is deployment without rules or a plane classifier, but its cost benefit over those baselines is not directly measured. The supervised reference also uses different source training (Appendices E and I), and the stronger source models already have higher carotid Dice before self-training (Table 3).
Questions for authors
Clear answers supported by evidence could change my recommendation.
- Do the main conclusions hold for test frames sampled throughout complete sweeps, rather than mainly near their start and end?
- Is the intended task to identify true absence or to avoid segmentation when visibility is uncertain? Why is the score that combines these cases used as the main score?
- The supervised reference and the stronger self-training baselines use different training data. How should we interpret their results in Table 3?
- What is the accuracy and cost trade-off between Persist & fit and self-training with rules at test time? Please compare total inference time and adaptation cost under the same hardware and input settings.