- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes RPGNet for three-class classification of laryngopharyngeal endoscopic images. It combines SAM2-guided dual-view learning, a shared GreedyViG encoder, and coarse-to-fine feature routing with a dynamic selection budget. It reports improved macro-F1 on one internal and three held-out external cohorts.
Critical strengths
- The paper evaluates RPGNet on one internal dataset and three external datasets, showing consistent improvements in macro-F1 over the compared methods. The multi-center evaluation provides encouraging evidence of the model's generalization ability.
- The proposed coarse-to-fine routing strategy is well motivated. By restricting feature selection through parent-child spatial relationships, the method offers a reasonable way to preserve lesion-related information across different scales.
- RPGNet achieves competitive performance with relatively low computational complexity. The reported parameter count and FLOPs are substantially lower than those of SAM-Swin, suggesting potential benefits for practical applications.
Critical weaknesses
Top-k selection. This makes it difficult to isolate the contribution of each component and justify the adaptive budget design. 2. The method largely builds on existing components, including SAM2, GreedyViG, and multiscale feature selection. Its distinction from existing hierarchical token selection and pruning methods is not sufficiently established. 3. The model still performs poorly on certain clinically important classes. Benign recall on FAHSU is only 27.29%, while malignant recall on NHSMU drops from 97.87% with SAM-Swin to 94.41% with RPGNet. 4. The claimed reduction in background interference lacks quantitative evidence. Since RPN operates on features already extracted by the backbone, it is unclear to what extent the proposed routing actually prevents background contamination. 5. RPGNet and the main competing methods use different backbones, making it difficult to attribute the performance gains to the proposed modules. Comparisons under matched backbone architectures and training settings would be more convincing.
Questions for authors
- Can the authors compare Dynamic-k with fixed-k and unrestricted Top-k selection under matched token budgets to better isolate the contribution of each component?
- How does RPN differ from existing hierarchical token selection methods? Can the authors provide quantitative evidence that it reduces background interference?
- Can the authors provide comparisons using the same backbone and training settings to better establish the contribution of the proposed modules?
- Could the authors further analyze the low benign recall on FAHSU and the reduced malignant recall on NHSMU compared with SAM-Swin?
10 comments
Distinguishing RPN from Existing Token Selection and Pruning Methods
Weakness 2: The method largely builds on existing components, including SAM2, GreedyViG, and multiscale feature selection. Its distinction from existing hierarchical token selection and pruning methods is not sufficiently established.
Response:
We thank the reviewer for highlighting the need to clarify RPN's relationship to existing token selection and pruning methods. To address this concern, we explain the differences in where selection occurs, how candidate tokens are defined, and how retention budgets are assigned. RPGNet incorporates established components, including SAM2 and GreedyViG. The contribution we emphasize for RPN is the combination of spatially constrained parent–child selection and adaptive retention budgets predicted for individual parents within recursive multiscale routing.
For selection along network depth, DynamicViT [1] estimates token importance and progressively prunes less informative tokens at selected layers. A-ViT [2] uses token-wise halting to determine when individual tokens stop participating in subsequent layers. These methods regulate token participation during network computation. RPN operates on an already extracted feature pyramid and recursively selects tokens across spatial scales to construct multiscale descriptors for classification.
For multiscale token selection, MST [3] computes a reference feature by averaging base-token features associated with positive user clicks. It ranks multiscale tokens by their cosine similarity to this reference and applies Top-k selection with a specified retention count at each scale. The selected tokens then participate in feature interaction with the base tokens. RPN defines its candidate sets through explicit parent–child spatial correspondence and predicts the retention count separately for each retained parent.
Specifically, at each coarse-to-fine transition, a retained parent can select only from its four directly corresponding children at the next finer scale. The budget head uses the parent token to predict a retention count kₚ ∈ {0, 1, 2, 3, 4}, while the scoring head evaluates the child tokens and retains the top kₚ candidates within that parent's spatial footprint. The retained children become the parents for the next transition, and this procedure continues recursively across the feature pyramid.
RPN therefore couples where selection can occur, through the parent-constrained candidate set, with how many tokens to retain locally, through the parent-specific budget prediction. This formulation allows different parent regions within the same image to receive different branching budgets while preserving spatial correspondence across scales. The controlled comparisons reported in our response to Weakness 1 further evaluate the contributions of these two routing choices.
References
[1] Rao et al. “DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification.” NeurIPS, 2021.
[2] Yin et al. “A-ViT: Adaptive Tokens for Efficient Vision Transformer.” CVPR, 2022.
[3] L. Xu, Y. Chen, S. Li, and J. Luo, “MST: Adaptive Multi-Scale Tokens Guided Interactive Segmentation,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 8, pp. 8148–8160, 2025, doi: 10.1109/TCSVT.2025.3546634.
Class-wise Recall Limitations on External Cohorts
Weakness 3: The model still performs poorly on certain clinically important classes. Benign recall on FAHSU is only 27.29%, while malignant recall on NHSMU drops from 97.87% with SAM-Swin to 94.41% with RPGNet.
Response:
We thank the reviewer for drawing attention to these clinically important class-wise results. To better interpret them, we evaluate the class composition of the test cohorts and assess the reported recall values alongside the aggregate classification results.
Table 3. Class composition of the internal and external test cohorts.
| Dataset | Normal, n (%) | Benign, n (%) | Malignant, n (%) | Total |
|---|---|---|---|---|
| FAHSYSU | 2,280 (31.54%) | 1,199 (16.59%) | 3,750 (51.87%) | 7,229 |
| NHSMU | 5,716 (85.52%) | 592 (8.86%) | 376 (5.63%) | 6,684 |
| FAHSU | 4,374 (89.96%) | 447 (9.19%) | 41 (0.84%) | 4,862 |
The external cohorts contain predominantly normal images. Benign recall on FAHSU is based on 447 images, while malignant recall on NHSMU is based on 376 images, compared with 1,199 benign and 3,750 malignant images in FAHSYSU. Each misclassification therefore has a larger effect on the external-cohort recall estimates. These counts provide context for interpreting the estimates and their sampling sensitivity; attributing the observed errors to class imbalance would require additional evidence.
We acknowledge that the benign recall of 27.29% on FAHSU indicates limited sensitivity for this class. On NHSMU, malignant recall decreases from 97.87% with SAM-Swin to 94.41% with RPGNet, a reduction of 3.46 percentage points. Both models are evaluated on the same cohort with the same class composition. Explaining this gap requires further analysis of model behavior, and the lower malignant recall remains a limitation of RPGNet in this comparison.
At the same time, RPGNet improves Macro-F1 over SAM-Swin by 3.38 percentage points on NHSMU and 9.58 percentage points on FAHSU. Since Macro-F1 assigns equal weight to each class, these gains provide complementary evidence of improved aggregate recognition across classes. The results therefore show stronger overall Macro-F1 on both external cohorts, together with persistent weaknesses in FAHSU benign recognition and lower NHSMU malignant sensitivity relative to SAM-Swin.
Evaluating Adaptive Budgets and Spatial Constraints
Weakness 1: Top-k selection. This makes it difficult to isolate the contribution of each component and justify the adaptive budget design.
Response:
We thank the reviewer for highlighting the need to clarify the contributions of the routing components. We provide two additional comparisons: fixed-k routing to examine the benefit of adaptive budgets, and unrestricted Top-k selection to evaluate the role of parent–child spatial constraints. All variants share the same encoder, supervision, Stage-2 training settings, and multiscale readout, allowing the comparisons to focus on the routing choices.
First, we retain the Scoring Module and parent–child spatial constraint while assigning a constant branching budget of k ∈ {0, 1, 2, 3, 4}. Each selected parent retains exactly k of its four candidate children, with their identities determined by the Scoring Module. These controls allow us to compare input-dependent allocation with constant branching within the same recursive routing framework.
Table 1. Comparison of fixed-k and Dynamic-k routing on FAHSYSU.
Among the fixed-k controls, k = 2 gives the highest performance. Dynamic-k improves ACC from 94.20% to 94.85% and Macro-F1 from 91.98% to 92.55%, corresponding to gains of 0.65 and 0.57 percentage points, respectively. Its advantage over all five constant branching settings supports the use of adaptive allocation within the parent-constrained routing framework.
Second, we evaluate unrestricted Top-k selection, which allows child tokens to be selected regardless of whether their parent regions were selected. The multiscale readout remains unchanged, enabling us to evaluate the effect of the parent–child constraint while preserving the feature aggregation formulation.
Table 2. Unrestricted Top-k selection compared with full RPGNet across four datasets.
Full RPGNet obtains higher ACC and Macro-F1 on every dataset. Relative to unrestricted selection, its Macro-F1 increases by 0.66 percentage points on FAHSYSU, 1.23 on SAHSYSU, 2.16 on NHSMU, and 0.54 on FAHSU. The improvements across both internal and external cohorts support the utility of preserving parent–child spatial correspondence during token selection.
Together, these experiments provide separate evidence for adaptive budget allocation and parent-constrained selection, with the multiscale readout held constant, and clarify the routing contributions beyond the original combined ablation.