- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ROOTROUTE, a two-stage method for multi-label text routing. A Core first forms an initial score for each concept by pooling the top-2 window matches. A verifier then attends to the top windows and adds a gated, bounded correction to that score. The method uses frozen BioMedCLIP features and routes radiology reports to 16 anatomical concepts. It is validated on seen-source reports, two unseen-source collections, an expert-annotated Stanford set and a controlled language suite. The verifier improves F1 and exact-set agreement over the Core on all five evaluations.
Critical strengths
S1: The model is lightweight, with about 124K trained parameters on frozen features. This makes the method cheap to train and deploy compared with LLM-based alternatives.
S2: The decision-level analysis (Table 2, Table 7) is detailed and reports negative results honestly. The evaluation also includes unseen sources and an expert-annotated set. This makes the paper's strengths and limits easy to judge.
Critical weaknesses
W1: The motivation lacks empirical support. The paper claims that pooled scores compress evidence and cannot tell whether a decision should be revised, but no experiment verifies this. The Core's errors may simply come from its low capacity, since it uses one direction per concept and top-2 pooling. The authors should test whether window-level evidence separates correct and incorrect decisions that have similar pooled scores, and whether the errors remain under a stronger pooling.
W2: The two-stage design shows no clear advantage, and its generality is not shown. All consistent gains are measured against the authors' own weak Core. The matched attention head, with the same features and similar size, beats Full on seen-source reports, IU-Xray F1 and X-Raydar Exact (Table 1). The method is also tested on only one domain with 16 labels. The authors should apply the verifier on top of a stronger first stage and test on standard multi-label benchmarks, or narrow the claims to radiology reports.
W3: The method modules are weakly linked to the stated problem. The verifier only sees the top-8 windows chosen by the Core (Eq. 4), so it cannot recover evidence the Core missed; recall on abbreviations and typos stays around 0.26. The low flip rate follows from the gate and small budgets by design, not from learning. Among reachable but uncorrected errors, 82–87% are pushed in the wrong direction. The authors should explain how each module addresses the claimed problem and support this with evidence.
W4: Statistical significance is not analyzed. All models are trained with a single seed, and the bootstrap intervals cover only test sampling. Many reported gains are small, such as +1.1 F1 on the language suite and most ablation differences within one point. The authors should report mean and standard deviation over 3–5 seeds.
W5: The ablation study is insufficient. The method has many components and hyperparameters, but only initialization, the update rule and the verifier inputs are ablated. The auxiliary losses, section and position features, fold-excluded training, eligibility cutoff and budget choices are not tested. Most importantly, there is no single-stage baseline that uses the same top-8 attention, so the gain from revision cannot be separated from better aggregation.
Questions for authors
Q1: Could the authors provide evidence that Core errors come from evidence compression rather than the limited capacity of the Core?
Q2: Could the authors explain how the verifier recovers evidence the Core missed, given that it only sees the Core's top-8 windows?
Q3: Do the authors have results over multiple seeds for the main comparisons?
Q4: Could the authors report whether the verifier still helps when applied on top of the matched attention head?
Q5: Could the authors compare Full with a single-stage model that uses the same top-8 attention?