acceptodds
Withdrawn from ICLR 2027

ROOTROUTE: Hypothesis Formation and Selective Revision for Multi-Label Text Routing

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-label text routing must combine evidence distributed across local spans into document-level concept decisions. Aggregating the strongest local matches provides an initial estimate of concept support, but the resulting score compresses the evidence that produced it and offers limited information about whether the initial judgment should be retained or revised. We propose ROOTROUTE, which explicitly separates hypothesis formation from evidence-guided revision. The Core learns reusable concept directions from representative training evidence and sparsely aggregates local matches into an initial support score. A verifier then revisits the current document with target-conditioned attention and predicts a constrained correction from the retrieved evidence and the initial support state. We instantiate ROOTROUTE for anatomical routing from clinical reports using frozen BioMedCLIP representations and 128,627 training reports from five sources. Across four medical-report evaluations and a controlled-language suite of 20,480 examples, ROOTROUTE consistently improves both F1 and exact-set agreement over the Core. On seen-source reports, macro-F1 increases from 0.919 to 0.963 and exact-set agreement from 0.767 to 0.876. On 2,000 independently expert-mapped RadGraph-XL-Stanford reports, it reaches 0.928 macro-F1 and 0.683 exact-set agreement. Decision-level analysis further shows that revision changes only a small fraction of Core decisions and that these changes are predominantly corrective, while controlled diagnostics show that revisiting document evidence can recover concept support missed by the initial aggregation. These results support explicit separation of initial support formation from selective evidence-guided revision for multi-label text routing, while making the resulting decision changes directly inspectable. Code will be released after the review process.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-label Classification, Interpretability, Clinical Report Routing

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3U9xOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ROOTROUTE, a two-stage method for multi-label text routing. A Core first forms an initial score for each concept by pooling the top-2 window matches. A verifier then attends to the top windows and adds a gated, bounded correction to that score. The method uses frozen BioMedCLIP features and routes radiology reports to 16 anatomical concepts. It is validated on seen-source reports, two unseen-source collections, an expert-annotated Stanford set and a controlled language suite. The verifier improves F1 and exact-set agreement over the Core on all five evaluations.

Critical strengths

S1: The model is lightweight, with about 124K trained parameters on frozen features. This makes the method cheap to train and deploy compared with LLM-based alternatives.

S2: The decision-level analysis (Table 2, Table 7) is detailed and reports negative results honestly. The evaluation also includes unseen sources and an expert-annotated set. This makes the paper's strengths and limits easy to judge.

Critical weaknesses

W1: The motivation lacks empirical support. The paper claims that pooled scores compress evidence and cannot tell whether a decision should be revised, but no experiment verifies this. The Core's errors may simply come from its low capacity, since it uses one direction per concept and top-2 pooling. The authors should test whether window-level evidence separates correct and incorrect decisions that have similar pooled scores, and whether the errors remain under a stronger pooling.

W2: The two-stage design shows no clear advantage, and its generality is not shown. All consistent gains are measured against the authors' own weak Core. The matched attention head, with the same features and similar size, beats Full on seen-source reports, IU-Xray F1 and X-Raydar Exact (Table 1). The method is also tested on only one domain with 16 labels. The authors should apply the verifier on top of a stronger first stage and test on standard multi-label benchmarks, or narrow the claims to radiology reports.

W3: The method modules are weakly linked to the stated problem. The verifier only sees the top-8 windows chosen by the Core (Eq. 4), so it cannot recover evidence the Core missed; recall on abbreviations and typos stays around 0.26. The low flip rate follows from the gate and small budgets by design, not from learning. Among reachable but uncorrected errors, 82–87% are pushed in the wrong direction. The authors should explain how each module addresses the claimed problem and support this with evidence.

W4: Statistical significance is not analyzed. All models are trained with a single seed, and the bootstrap intervals cover only test sampling. Many reported gains are small, such as +1.1 F1 on the language suite and most ablation differences within one point. The authors should report mean and standard deviation over 3–5 seeds.

W5: The ablation study is insufficient. The method has many components and hyperparameters, but only initialization, the update rule and the verifier inputs are ablated. The auxiliary losses, section and position features, fold-excluded training, eligibility cutoff and budget choices are not tested. Most importantly, there is no single-stage baseline that uses the same top-8 attention, so the gain from revision cannot be separated from better aggregation.

Questions for authors

Q1: Could the authors provide evidence that Core errors come from evidence compression rather than the limited capacity of the Core?

Q2: Could the authors explain how the verifier recovers evidence the Core missed, given that it only sees the Core's top-8 windows?

Q3: Do the authors have results over multiple seeds for the main comparisons?

Q4: Could the authors report whether the verifier still helps when applied on top of the matched attention head?

Q5: Could the authors compare Full with a single-stage model that uses the same top-8 attention?

Reviewer_8453OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the multi-label text routing problem in the context of clinical-report application. The target is to identify the anatomy discussed in the reports. The proposed method is a two-stage framework for multi-label text routing. A Core module learns concept representations and aggregates local evidence to form initial predictions, while a target-conditioned verifier selectively revises these predictions using document evidence and bounded residual corrections. Evaluated on anatomical concept identification from radiology reports, the method consistently improves F1 and exact-set accuracy over the Core across multiple datasets, including unseen-source and controlled-language evaluations.

Critical strengths

The paper evaluates the proposed method across multiple datasets, including unseen-source and expert-annotated evaluations. The authors also provide extensive ablation studies and decision-level analyses to investigate the effectiveness of individual components. This strengthens the empirical credibility of the work and provides useful insights into when and how the proposed revision mechanism improves predictions.

Critical weaknesses
  1. The task of identifying anatomical concepts from medical reports is relatively straightforward for modern LLMs. However, the paper only evaluates LLaMA-3-8B with a basic zero-shot prompt that does not explicitly specify the 16 target labels, and uses restrictive output parsing. This may substantially underestimate LLM performance. Without comparisons against stronger LLMs or properly designed prompts, it is difficult to assess the practical significance of the proposed method.
  2. The proposed framework primarily combines established techniques, including prototype learning, label-conditioned attention, and residual correction. More importantly, the capacity-matched attention classifier achieves comparable or better performance on several benchmarks, raising questions about whether the proposed two-stage formulation offers a meaningful advantage over simpler alternatives.
Questions for authors

Q1: Could the authors investigate why LLaMA-3-8B performs relatively poorly on this seemingly straightforward anatomical concept identification task? In particular, to what extent are its errors caused by incorrect semantic predictions versus output formatting or parsing failures? For example, does the LLM frequently identify the correct anatomical concepts but express them using synonyms or alternative terminology that the parser fails to recognize? An error breakdown and evaluation using more flexible parsing would help determine whether the reported results accurately reflect the LLM's underlying capabilities.

Q2: What is the performance if we use stronger open-source model, say QWEN 3.5 9B? or other cheap close-source models, say, gpt-5-mini?

Reviewer_GyaYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces ROOTROUTE, a two-stage approach to multi-label text routing. The Core learns concept directions and pools the strongest local evidence, while a target-conditioned verifier makes gated, bounded corrections to its scores. Using frozen BioMedCLIP representations, the method routes reports to 16 anatomical labels. Evaluation covers seen-source reports, two unseen sources, expert-mapped Stanford reports, and controlled language probes. The full system improves over the Core across all evaluations, and the analysis tracks both corrected and newly introduced errors.

Critical strengths
  1. Separating initial support from subsequent revision makes the design easy to follow and individual decision changes straightforward to inspect.

  2. The experiments examine the main design choices, including initialization, alignment, pooling, revision rules, and verifier inputs. A classifier using the same representations and similar trainable capacity provides a useful comparison.

  3. The expert-mapped Stanford evaluation complements the automatically labeled datasets. Group-aware splitting and paired bootstrap intervals strengthen the evidence that revision improves over the Core.

Critical weaknesses
  1. The benefit over direct classification is less clear than the improvement over the Core.

  2. Evidence for broader applicability remains limited to one encoder and 16 anatomical labels.

  3. These experiments support the anatomical application, but leave performance on larger or more varied label vocabularies largely open.

  4. The efficiency argument relies mainly on parameter counts, which do not capture the cost of the full pipeline (Time, Memory Cost, etc.).

Questions for authors
  1. Have disagreements with the lexical labeler been manually checked, especially when the model predicts additional anatomy?

  2. Please report available training costs, end-to-end latency, and memory use for Core, Full, and the matched head, stating whether embeddings are cached.

  3. Are results available for another label vocabulary or encoder?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.