- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
In this paper, the authors question whether dense segmentation is necessary for object length measurement and propose MEASURENET, a framework that directly predicts object polylines using measurement-aware losses. The authors also introduce CeDeM, a new duodenal biopsy dataset with polyline annotations for villus and crypt measurement. Across four public datasets and the proposed CeDeM dataset, MEASURENET consistently achieves strong performance in both polyline localization and length measurement. The experiments further suggest that direct polyline-based measurement can be more annotation-efficient and achieve lower measurement error than conventional pipelines that rely on dense segmentation masks as an intermediate representation.
Critical strengths
- The research question is meaningful. Length measurement is important across biological and clinical domains. Conventional methods usually obtain polylines through intermediate segmentation rather than directly predicting them, while segmentation mask annotation is expensive.
- The CeDeM dataset is a valuable contribution. It contains expert pathologist annotations and addresses a clinically meaningful measurement task.
- The empirical coverage is strong. The evaluation includes both the proposed dataset and multiple public datasets, spanning CT, histopathology, brightfield/fluorescence microscopy, and optical coherence microscopy.
Critical weaknesses
- My biggest concern is the novelty. First, prior work has explored direct geometric prediction for object measurement, so this paper is not the first to perform direct measurement. The paper should clarify its differences from existing direct measurement methods. Second, the proposed measurement-aware losses are mainly combinations of existing losses, and the main architectural modification is just replacing the bounding-box head with a polyline head.
- The evaluation metric is problematic because it does not enforce one-to-one instance matching. For example, if the ground truth contains two polylines of length 10 but the model predicts only one polyline of length 10, the current metric can still report zero error, which is not appropriate.
- There is a mismatch between the annotation-efficiency claim and the training supervision on the public datasets. Across the four public datasets, the polyline labels are derived from segmentation masks rather than manually annotated. Therefore, the experiments do not fully validate the claimed annotation efficiency.
- Architectural validation is limited. The proposed framework is evaluated only on DINO-DETR with a Swin-L backbone. It remains unclear whether the proposed polyline representation and measurement-aware losses are architecture-agnostic.
Questions for authors
- Were the values of K selected exclusively using the training and validation sets, without using any information from the test set?
- Did the authors consider evaluating MEASURENET on existing public datasets with manually traced centerline annotations?
7 comments
Q2: Evaluation on public datasets with manually traced centerlines.
The closest public dataset we could identify is DeepTangle, which provides manually traced centerlines of C. elegans for evaluation, but no manually annotated training split. We evaluated MeasureNet and LETR (our closest baseline), both trained on BBBC010, directly on DeepTangle without fine-tuning. MeasureNet achieves an mAP of 57.3 versus 18.9 for LETR, and an M-MAE of 9.5 versus 13.3. If the reviewer has another relevant dataset in mind, we would be happy to evaluate it and provide additional comparisons.
Thanks for the authors' responses, which have addressed most of my concerns. I appreciate the clarification that the key novelty of this work lies in the core hypothesis of formulating length measurement as an instance-wise polyline prediction task, rather than in introducing a new backbone or loss function. I suggest that the authors further highlight this core hypothesis in the revised manuscript.
However, I still have one remaining concern regarding the methodological novelty. Specifically, I am curious about the distinction between the proposed MEASURENET and existing geometric detection methods adapted for length measurement by incorporating additional measurement-aware loss terms. Could the authors further clarify what unique advantages arise from the combination of the instance-wise polyline prediction formulation and the proposed measurement-aware losses, which may not be readily achieved by adapting existing geometric detection approaches?
We thank the reviewer for finding the research question meaningful, CeDeM a valuable contribution, and the empirical coverage across five datasets and imaging modalities strong. Changes in the revised manuscript are marked in red.
W1: Novelty relative to direct measurement methods, losses and architecture.
We agree that direct geometric prediction and the individual loss functions are not new. Our key novelty lies in the core hypothesis of formulating length measurement as an instance-wise polyline prediction task, rather than in introducing a new backbone or loss function.
Unlike direct scalar regression, our approach predicts the underlying geometry, allowing clinicians to visually verify and correct individual points when necessary. This is critical in applications such as healthcare, where the cost of errors can be high. Unlike existing line detectors (LETR, RANK-LETR) and general polyline detectors (YOLinO, MapTR), MeasureNet explicitly optimizes measurement accuracy through joint supervision of point localization, total length, and individual segment lengths. Although the architectural modification is subtle, the measurement-aware training objective provides consistent improvements (Table 3). We further demonstrate that sparse polyline annotations achieve competitive measurement accuracy with substantially lower annotation effort across five datasets.
W2: M-MAE does not match instances one-to-one: if the ground truth has two polylines of length 10 and the model predicts only one of length 10, the error is zero.
We agree that M-MAE alone does not penalize missing instances when their lengths match. However, M-MAE evaluates measurement accuracy, while precision, recall and mAP evaluate detection performance. In the given example, M-MAE can be zero, but recall is only 50%. We therefore report both measurement and detection metrics (Tables 1 and 2, and Appendix E). MeasureNet consistently achieves strong performance on both.
W3: On the four public datasets the polyline labels are derived from segmentation masks rather than annotated manually, so the annotation-efficiency claim is not fully validated.
We agree that for the existing datasets, polylines are derived by post-processing the existing segmentation masks, because no polyline-annotated datasets exist in the literature. To support our hypothesis about their effectiveness, we took several steps.
First, annotation efficiency was evaluated independently: two medical experts manually annotated polylines and dense masks for a subset of images across all five datasets, obtaining a 2.54–5.83× reduction in annotation time (Fig. 3(a), Table 5). Second, we visually inspected the polyline annotations and found no apparent errors in the inspected cases; Figures 4 and 5 further illustrate that the polylines adequately capture the underlying structures. Finally, and importantly, we release CeDeM, a dataset natively annotated with polylines by expert pathologists, which provides further validation of their effectiveness.
W4: Architecture-agnostic validation.
Our approach can be adapted to other object detectors by replacing the bounding-box head with a K-point polyline head and adding the measurement-aware losses. To investigate this, we extended RT-DETR [1] with a ResNet-50 backbone and evaluated it with and without the proposed measurement-aware losses.
The measurement-aware losses consistently reduce measurement error on all four datasets while also improving detection performance. We have added this to Appendix F.2.
[1] DETRs Beat YOLOs on Real-time Object Detection, CVPR 2024.
Q1: Were the values of K selected using only the training and validation sets, without any information from the test set?
Yes. All values of K were selected using only the training and validation sets. For CeDeM, we evaluated K∈{2,3,4} on the validation set and selected K=3 based on measurement accuracy. For the public datasets, K was selected based on the polyline approximation error relative to the full skeleton, using validation data. The test sets were used only for the final evaluation after fixing K.