acceptodds
Under review as a conference paper at ICLR 2027

Retrieve, Reason, Recover: Evidence-Grounded ICD Clinical Coding under Long Tails and Missing Labels

Abstract

Automated International Classification of Diseases (ICD) coding from clinical narratives faces three interrelated challenges; severe long-tail code distributions with sparse training examples, systematically incomplete labels from clinical undercoding, and a lack of interpretable evidence linking predictions to documentation. We propose HERA-ICD, a two-stage framework that reframes ICD coding as retrieval-augmented classification. Stage 1 performs hierarchical retrieval at document, code, and span granularities, enabling prototype-based few-shot and description-based zero-shot prediction for rare codes. In Stage 2, we construct per-instance graphs, whose topologies dynamically adapt to the retrieved evidence. A Graph Attention Network then reranks the candidates using document-specific reasoning. To handle missing labels, we integrate positive-unlabeled learning with graph-based label completion, propagating soft positives through the ICD hierarchy. We theoretically prove three key properties of our framework: the non-negative PU objective is unbiased and consistent, the generalization gap of the retrieval-then-rerank architecture is independent of vocabulary size, and prototype scoring generalizes from fewer than ten examples. Consequently, experiments on MIMIC-III and MIMIC-IV demonstrate consistent improvements over state-of-the-art methods, particularly on rare codes, while providing traceable evidence linking predictions to specific clinical spans.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
ICD Coding, Retrieval-Augmented Classification, Graph Neural Networks, Positive-Unlabeled Learning, Long-Tail Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZxeEOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper frames ICD coding as retrieval-augmented classification with 2 stages. Stage 1 retrieves at document, code and span levels with a Clinical-Longformer bi-encoder and FAISS. Stage 2 builds a per-note evidence graph and reranks candidates with an edge-weighted GAT and a learned fusion gate.

Critical strengths

The revewer can feel this paper is well-motivated option and treatment of long-tail and missing labels with traceable evidence, and the results show consistent gains on rare codes across MIMIC-III and MIMIC-IV.

Critical weaknesses

From the method discription part, the label completion and the PU guarantees are internally inconsistent? As described, label propagation uses a row-normalized graph matrix and starts unobserved codes at 0. Then under that setup, no unobserved code can receive a soft label above the propagation strength, am I understand it correctly? Table 6 reports good results for propagation strengths of 0.1 and 0.2 but the default propagation strength and soft-positive threshold both set to 0.3, which means no code can pass it?

Questions for authors

How can l exceed alpha in Fig. 5? What W/alpha/theta_soft are actually used? Is retrieval leave-one-out in training? Are Table 4 MIMIC-IV statistics and split protocols the same as those used for the copied baselines?

1 comment

Thank you. Your reading of Equation 4 is correct and it exposed a typo.

W1. Propagation and the thresholds.

As printed, no unobserved code can exceed α\alpha, so with α=θsoft=0.3\alpha = \theta_{\mathrm{soft}} = 0.3 nothing would be selected. The printed equation has a typo in one superscript. The second term should use the current value of the label, not the starting value, which is what our code does:

ℓi,c(t+1)=α∑c′Wcc′,ℓi,c′(t)+(1−α),ℓi,c(t)(c∉Pi)\ell^{(t+1)}_{i,c} = \alpha \sum_{c'} W_{cc'}\\,\ell^{(t)}_{i,c'} + (1-\alpha)\\,\ell^{(t)}_{i,c} \quad (c \notin \mathcal{P}_i)

Under this rule soft labels can rise above α\alpha. On the 47,723 MIMIC-III training notes at the default settings, the average note receives 33.5 soft positives, and 99.7 percent of notes receive at least one.

This also explains Table 6. Under the implemented rule the sweep is meaningful:

α\alpha0.10.20.30.40.5
Soft positives per note3.417.133.549.465.0

Theorem 4 is restated for the implemented iteration in our answer to Reviewer qkWg, part 1.

Q. Figure 5, settings, leave-one-out and splits.

Figure 5 shows values above α\alpha because that is what the implemented rule produces. The settings are α=0.3\alpha = 0.3, θsoft=0.3\theta_{\mathrm{soft}} = 0.3, T=20T = 20, with a fixed row-normalised weight matrix. None of these are learned.

Retrieval in training is not leave-one-out. A training note can appear in its own document index. This creates a training-time shortcut that does not exist at test time, where the test note is never in the index, so it cannot inflate the reported numbers. We disclose it and will fix it. During training all gold codes are also added to the candidate set so the reranker always sees them. At test time candidates come from retrieval alone, with recall at 50 of 94.1 percent.

On MIMIC-IV, MSMN and CoRelation (marked with a diamond) are copied from Nguyen et al. (2023) and use their splits.

Ethics flag.

MIMIC-III and MIMIC-IV are de-identified and released under the PhysioNet credentialed data-use agreement. We collected no new data and had no contact with patients. Figure 1 reproduces short excerpts with the surrogate identifiers as released.

Reviewer_L8UQOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

HERA-ICD integrates multigranular document/code/span retrieval, per-note graph reranking, few-shot transfer, and positive-unlabeled learning for automated ICD coding. HERA-ICD has the following three steps:

  1. Retrieve a small set of relevant evidence

  2. Build a note-specific graph from that evidence

  3. Rerank the candidate codes

In the first step, it retrieves evidence at three levels: Similar patient notes: “Which past notes resemble this one?”, candidate codes: “Which codes are plausible for this note?”, and relevant spans: “Which exact sentences support each possible code?” For a rare code with a few known examples, it makes a “prototype”: the average representation of prior notes with that code. It asks whether the new note resembles that prototype. For a code never seen during training, it cannot build a prototype. Instead, it compares the note with the code’s official textual description.

In the second step, it builds a custom graph for this specific note. The graph contains the retrieved codes and evidence spans as nodes. It adds relationships based on:

  • ICD hierarchy;
  • Which spans appear to support which codes;
  • A retrieved similar historical note connects to the ICD codes that were assigned to it.

In the third step, it reranks the codes using the graph. A Graph Attention Network passes information among related candidate codes and evidence.

Critical strengths

The paper tackles a failure mode that standard ICD benchmarks largely ignore: incomplete coding labels. Rather than assuming every unassigned code is a true negative, HERA-ICD explicitly models unobserved codes as potentially positive and combines PU learning with hierarchy/co-occurrence-based soft-label completion.

The result is strong. HERA-ICD reports gains over strong baselines (including TraceCoder) on MIMIC-III and MIMIC-IV, with especially large macro-F1 gains (e.g., 15.8 vs. 12.5 for TraceCoder on MIMIC-III-Full).

Critical weaknesses

The paper combines multiple existing solution together as one. Hierarchical retrieval, adaptive hierarchy reasoning, prototype and description-based transfer all have close precedents, including HieNet, TraceCoder, AGM-HT, and MedCodER, so the paper is better positioned as a broad synthesis than a new method, as proposed in the contribution section in line 087. Specifically, TraceCoder dynamically matches UMLS/Wikipedia/LLM-generated knowledge to each note and attributes predictions to both clinical text and external evidence. MedCodER and LLM-codex also explicitly extract or produce sentence-level evidence; LLM-codex uses segment-level proposals plus a verifier and targets limited/rare-label settings.

The experiments do not validate the central claim of recovering real undocumented diagnoses. MIMIC billing labels are used both as incomplete training supervision and as the test ground truth. Therefore, standard F1 cannot determine whether newly predicted/recovered codes are genuine missed diagnoses or false positives. The synthetic label-dropping experiment only tests an imposed MCAR deletion process, not real coding omissions, and there is no clinician adjudication or independently complete-label evaluation of recovered codes. Consequently, the paper’s main practical claim is unverified.

Questions for authors

How good is the HERA-ICD on Delirium? Delirium is known to be positive underlabeled. Among 857 CAM-ICU-positive patients in MIMIC-III, only 152 (17.7%) had a delirium ICD diagnosis code. https://pmc.ncbi.nlm.nih.gov/articles/PMC9152701 The ICD-9 set they used included 293.0, 293.1, 290.11, 290.3, 290.41, 291.0, 292.81, and 780.97. How much HERA-ICD outperforms baseline without non-negative PU objective on Delirium evaluation subset?

What dataset characteristics are necessary for applying HERA-ICD effectively? In particular, would an ICD-coding dataset with repeatedly human or LLM-verified labels (rather than potentially incomplete MIMIC labels) be an appropriate benchmark for HERA-ICD?

1 comment

Thank you. Your two main points are correct.

W1. Novelty.

We agree that each component has precedents. HieNet, AGM-HT, MedCodER and LLM-codex are close prior work and will be cited and discussed. We describe HERA-ICD as an integration. What we believe is new is the combination: a per-instance graph whose attention is weighted by retrieval scores, PU learning with graph-based label completion, and an analysis with its scope stated. The contribution list will be softened to match.

W2. The recovery claim.

You are right that MIMIC F1 cannot show that a predicted code missing from the record is a real missed diagnosis, because the labels are both the training signal and the test reference. We narrow the claim to: the PU term lowers the penalty on unobserved codes. The delirium study below is our first test against an independent reference.

Q1. Delirium.

We used the eight ICD-9 codes you listed as one target and CAM-ICU status as the reference.

ModelAUROC vs CAM-ICUUncoded CAM-ICU-positive patients flagged
No-PU variant0.67125 of 705
Full HERA-ICD0.74190 of 705

A patient-level bootstrap gives a 95 percent interval of 0.04 to 0.10 for the AUROC gap, so it excludes zero. The absolute AUROC is capped because discharge summaries often do not mention ICU delirium at all.

The gap is modest and we think we know why. Delirium is recorded for about 18 percent of true cases, so the prior estimated from frequency is about five times too small, which weakens the PU correction (see our answer to Reviewer qkWg on the prior). The gain comes mainly from label propagation.

Q2. Which datasets suit HERA-ICD.

The method needs code descriptions and preferably a hierarchy, a training set large enough to build document, code and span indices, and a long tail of rare codes. It helps most when labels are incomplete. On a dataset with verified labels the PU term reduces to ordinary cross-entropy and we expect no gain from it, as on MIMIC-III-50. Such a dataset is the right place to test the recovery claim: train on incomplete labels, evaluate on verified ones.

Reviewer_qkWgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

HERA-ICD addresses automated ICD coding with rare codes, incomplete labels, and evidence traceability. It combines document-, code-, and span-level retrieval with a per-instance graph attention reranker, and incorporates positive-unlabeled learning with graph-propagated soft positives (\S\S 3.2–3.5). The paper reports macro-F1 improvements on MIMIC-III-Full and both MIMIC-IV subsets, supported by component ablations, retrieval measurements, synthetic label deletion, and explanation evaluations (Tables 1–3; \S\S 5.3–5.7).

Critical strengths
  1. The integrated approach addresses a meaningful problem. Retrieval, relational modeling, and missing-label handling form a coherent response to the challenges established in \S\S 1–3. Reported macro-F1 gains over TraceCoder are 3.3 points on MIMIC-III-Full and 3.7/4.6 points on the MIMIC-IV ICD-9/ICD-10 subsets, encouraging improvements beyond the close frequent-code comparisons (Tables 1–2).

  2. The evaluation examines several aspects of the proposed mechanism. Component removals and a static-graph comparison are complemented by retrieval quality, computational costs, and explanation measurements (Tables 3, 7–9). These analyses provide a useful basis for understanding the approach, subject to the concerns below.

Critical weaknesses
  1. \textbf{The label-completion mechanism and PU guarantees are inconsistent as written.} For an unobserved code, Eq. (4) initializes its soft label to zero. Since WW is row-stochastic and propagated values remain within [0,1][0,1], its updated value cannot exceed the propagation mixing coefficient. However, the reported settings use a mixing coefficient of 0.30.3 and require a value strictly greater than τsoft=0.3\tau_{\mathrm{soft}}=0.3 to select a soft positive (\S\S 3.4, 5.1). Consequently, no previously unobserved code can qualify. This conflicts with the interpretation of the label-completion ablation and the values above 0.30.3 illustrated in Figure 5 (Table 3; Appendix B.2). Moreover, Theorem 4 gives the fixed point for an unclamped iteration, whereas Eq. (4) clamps observed positives to one. The PU analysis raises a separate concern. Eq. (11) assumes an unlabeled marginal mixture with prior cc, but Eq. (5) uses the residual pool after removing observed and propagated positives. Appendix B.2 acknowledges a residual-prior bias without establishing why it vanishes, yet Theorem 2 claims consistency. Proposition 1 also targets true risk plus a soft-positive penalty, whereas Theorem 2 claims convergence to true risk alone. Finally, observed training frequencies estimate ecce_c c under Assumption 1, rather than cc, unless corrected (Appendix C.1). Required clarification: reconcile the actual propagation update and settings, report how many new soft positives they generate, and correct or narrow the PU guarantees. Identify whether these corrections affect the reported experiments.

  2. \textbf{The reliability of smaller improvements remains uncertain.} Tables 1–3 and 5–6 provide point estimates without specifying run counts, variability, confidence intervals, or significance tests. This particularly affects the 69.5 versus 69.1 macro-F1 comparison on MIMIC-III-50 and the 0.6–0.7-point component improvements in Table 3. The larger full-vocabulary gains remain encouraging, but the claim that every component contributes reliably needs stronger support. Minimal fix: provide repeated-seed variability for the key comparisons and paired confidence intervals or suitable paired tests using common test examples, respecting patient grouping where applicable. The absence of testing does not establish that these differences are insignificant.

Questions for authors
  1. \textbf{How do code-level examples affect prediction?} Section 3.2 describes code-example nodes and prototype scores, but Algorithm 2 instantiates code, span, and document-context nodes, while Eq. (3) does not explicitly incorporate the prototype score. Where do these examples and scores enter the implemented model? Does the static-graph comparison preserve the same evidence inputs, candidates, and model capacity (\S\S 3.2–3.3; Table 3)?

  2. \textbf{What exactly is the sufficiency metric?} Section 5.7 defines it as retained probability but treats lower values as better. Lower retained probability would indicate weaker sufficiency; perhaps the intended quantity is a probability drop. Please give the formula, clinician-rationale sample and annotation procedure, and explain whether retrieval and graph construction are recomputed during interventions (\S 5.7; Table 9).

  3. \textbf{What is the scope of Theorem 1?} Appendix B.3 treats retrieval scores as fixed, while \S 3.5 fine-tunes the query encoder. Please distinguish a guarantee conditional on a fixed retriever over retrieved candidates from one covering the trained pipeline and full vocabulary.

1 comment

Thank you. You found a real error.

W1. Label completion. You are right: as printed, Equation 4 caps unobserved codes at α\alpha. The printed equation has a one-superscript typo. The implemented update for c∉Pic \notin \mathcal{P}_i is ℓi,c(t+1)=α∑c′Wcc′ℓi,c′(t)+(1−α)ℓi,c(t)\ell^{(t+1)}_{i,c} = \alpha \sum_{c'} W_{cc'}\ell^{(t)}_{i,c'} + (1-\alpha)\ell^{(t)}_{i,c} (code: alpha * propagated + (1 - alpha) * labels[node]). The second term uses the current value, not ℓ(0)\ell^{(0)}, so labels can exceed α\alpha. Measured on the 47,723 MIMIC-III-Full training notes at the defaults (α=θsoft=0.3\alpha=\theta_{\mathrm{soft}}=0.3, T=20T=20), graph built as in the released code:

Printed Eq. 4Implemented
Max soft label, unobserved codes0.30000.9992
Soft positives per note0.0033.5
Notes with at least one0.3%99.7%

Notes have 11.4 recorded codes on average. Of the 33.5 soft positives, 29.4 lie in [0.3,0.5)[0.3,0.5), 3.3 in [0.5,0.7)[0.5,0.7) and 0.8 above 0.7; each is weighted by its label in the loss, so weak ones contribute little. Figure 5 is consistent with this rule.

Theorem 4, corrected. Let M=αW+(1−α)IM=\alpha W+(1-\alpha)I and U\mathcal{U} the unobserved codes, observed ones clamped to 1. Then ℓU(t+1)=MUUℓU(t)+MUP1\ell^{(t+1)}_{\mathcal{U}}=M_{\mathcal{UU}}\ell^{(t)}_{\mathcal{U}}+M_{\mathcal{UP}}\mathbf{1}. If every unobserved code has a path to an observed one, ρ(MUU)<1\rho(M_{\mathcal{UU}})<1, the iteration converges geometrically to 1\mathbf{1}, and ∣ℓU(T)−1∣∞≤ρ(MUU)T\\|\ell^{(T)}_{\mathcal{U}}-\mathbf{1}\\|_\infty\le\rho(M_{\mathcal{UU}})^T (row-stochasticity of MM gives (I−MUU)−1MUP1=1(I-M_{\mathcal{UU}})^{-1}M_{\mathcal{UP}}\mathbf{1}=\mathbf{1}). The limit is uninformative, so the training signal is the finite-TT iterate and TT is a method parameter. We measured ρ=0.9996\rho=0.9996; after 20 steps the iterates are 0.987 below the limit. The 3.5×10−113.5\times10^{-11} truncation claim is withdrawn.

PU guarantees. We agree on all four. (i) Prior: under MCAR recording with rate ece_c, fc=ecπcf_c=e_c\pi_c; we use π^c=fc\hat\pi_c=f_c, a lower bound. The bias of the unbiased risk is (π^c−πc)EPc[ℓ+−ℓ−](\hat\pi_c-\pi_c)\mathbb{E}_{P_c}[\ell^+-\ell^-], at most B∣π^c−πc∣B|\hat\pi_c-\pi_c|; a small prior subtracts less in the negative term, so the negative risk is over-estimated and the loss errs toward negatives, closer to ordinary cross-entropy. Consistency to the true risk needs ece_c, which PU data cannot identify. (ii) Prop. 1 vs Thm. 2: Theorem 2 now reads E[R^c]→Rc+Rcsoft\mathbb{E}[\hat R_c]\to R_c+R_c^{\mathrm{soft}}, matching Proposition 1. (iii) Residual pool: using Uc′\mathcal{U}'_c with prior πc′≤πc\pi'_c\le\pi_c under-estimates the negative risk by (πc−πc′)EPc[ℓ−](\pi_c-\pi'_c)\mathbb{E}_{P_c}[\ell^-], a bounded conservative offset that does not vanish; Theorem 2 holds up to it. (iv) Theorem 1: conditional on a fixed retriever, it bounds the candidate-set gap only. Corollary: errfull=K∣C∣errcand+\mathrm{err}_{\mathrm{full}}=\frac{K}{|\mathcal{C}|}\mathrm{err}_{\mathrm{cand}}+ miss rate, with miss rate =1−recall@50=5.9=1-\mathrm{recall@50}=5.9\\%. No reported number changes; the code is unchanged.

W2. Reliability. Patient-level cluster bootstrap on saved test predictions:

DatasetGain over TraceCoder95% interval
MIMIC-III-Full+3.31.8 to 4.8
MIMIC-III-50+0.4−0.4 to 1.2

We withdraw the claim of leading on MIMIC-III-50 and the individual 0.6- and 0.7-point ablation claims. Seed variance will be in the final version.

Q1. The prototype score does not enter Eq. 3; it is blended with the reranker output at inference for rare codes. Node types are code, span and context (Algorithm 2); the Section 3.2 sentence saying otherwise will be corrected. The static-graph control keeps candidates and parameter count but drops span and context nodes as well as the per-instance edges, so its 1.6-point drop measures both; an edges-only control will be in the final version.

Q2. Sufficiency. You are right: we measured retained probability and mislabelled the direction. With the standard definition (baseline minus retained; lower is better):

Comp. (higher better)Suff. (lower better)Overlap (higher better)
PLM-ICD attention0.180.4534.2
HERA-ICD spans0.410.6052.7

Our spans are worse on sufficiency; the other two are unchanged. Retrieval and graph construction are recomputed on the perturbed text.

Q3. See (iv). Also, training retrieval is not leave-one-out: a note can retrieve itself. This shortcut is absent at test time, where the test note is never indexed, so it cannot inflate results; we will exclude the query note in the final version.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.