acceptodds
Under review as a conference paper at ICLR 2027

KLEAR Enables Pre-Transplant Forecasting of Acute Rejection in Deceased-Donor Kidney Transplant Recipients

Abstract

Kidney transplantation remains the gold standard for treating end-stage renal disease, yet acute T cell-mediated rejection (TCMR) and antibody-mediated rejection (ABMR) significantly compromise early graft outcomes and long-term survival. Current risk assessment relies on low-resolution Human Leukocyte Antigen (HLA) matching, which fails to capture the molecular nuances of the donor-recipient immune interface. This submission introduces KLEAR (Kidney Likelihood Estimator for Acute Rejection), a machine learning framework designed for pre-transplant risk stratification. KLEAR leverages high-resolution HLA typing to derive novel molecular features (E3-MoFs) capturing physiochemical mismatches. To address the challenges of high-dimensional tabular clinical data and severe class imbalance, we propose a two-stage methodology: first, a generative oversampling strategy using a Variational Autoencoder integrated with a Bayesian Gaussian Mixture Model (VAE-GMM) to align the latent manifolds of rejection phenotypes; and second, a hybrid architecture that feeds tree-based leaf embeddings from eXtreme Gradient Boosting (XGBoost) into a dense neural network (DNN). Using a multi-site cohort of 2,013 patients from Mayo Clinic, the largest integrated transplant provider in the United States, KLEAR was trained and validated in a pilot prospective study. KLEAR demonstrates a 5.0-fold increase in AUPRC for TCMR and a 7.5-fold increase for ABMR compared to random classifiers. A small prospective pilot cohort provides feasibility evidence, but its event count is too limited to support deployment claims. Overall, this study highlights how molecular features of HLA mismatches, combined with clinical variables, can improve prediction of TCMR and ABMR and proposes a novel modeling framework for pre-transplant risk stratification. Once further validated in a large-scale, multi-center, prospective study, KLEAR may potentially inform clinical decision-making and support trial enrichment in transplantation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Kidney Transplantation, Machine Learning, Predictive Modeling, Antibody-Mediated Rejection, T Cell-Mediated Rejection

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_5zooOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents KLEAR, a model for predicting one-year T cell-mediated rejection (TCMR) and antibody-mediated rejection (ABMR) before kidney transplantation. The inputs are donor and recipient clinical data, HLA mismatch counts, and molecular mismatch features (PCA). The model has four stages: a VAE and a GMM generate extra rejection cases until the classes are balanced, XGBoost leaf indices are added as features, and a small neural network is trained with focal loss. The paper describes a 2,013-patient Mayo Clinic cohort, but every reported model uses the high-resolution HLA subset of 319 patients, with 22 TCMR and 12 ABMR cases. KLEAR reports a cross-validated AUPRC of 0.35 (TCMR) and 0.30 (ABMR), against prevalences of 0.07 and 0.04. A prospective cohort of 82 patients, with 4 TCMR and 3 ABMR cases, is also reported.

Critical strengths
  • The clinical question is important. Predicting rejection risk before transplant could help with organ acceptance and treatment choices, and most existing scores (e.g., iBox) use data from after the transplant.
  • The paper builds separate feature sets for TCMR and ABMR based on the biology of each (Section 3.2, Tables 7-8), and the appendix documents the features in detail.
  • The authors include a later prospective cohort (Section 7) rather than stopping at cross-validation, and they report the confusion matrix (Table 4).
  • AUPRC is reported next to the random baseline at each prevalence (Tables 1, 3, 4), which makes the numbers easy to read. Calibration (Figure 7) and decision curves (Figure 2) are also included.
  • The abstract and ethics statement admit that the prospective cohort is too small to support deployment, and that race-based predictors need subgroup evaluation.
Critical weaknesses
  • There are too few rejection cases to support the claims. Development has 22 TCMR and 12 ABMR cases, and the prospective cohort has 4 and 3. The confidence intervals in Table 1 overlap heavily across methods (TCMR: KLEAR 0.35 [0.18-0.54] vs. logistic regression 0.26 [0.11-0.45]), and both ROC-AUC intervals for KLEAR include 0.5. There are no paired statistical tests, and Section 5.1 itself says these should be added in future revisions. The data do not show that KLEAR beats logistic regression, or that the E3 features beat clinical data alone.

  • The main hypothesis is never tested. The paper argues that high-resolution HLA typing adds value over standard typing, but the imputed (n = 1,263) and low-resolution (n = 1,340) cohorts appear in no result. Section 5.2 is titled "Resolution-dependent performance gain", yet Table 2 compares only feature subsets. Has the 2,013-patient cohort described in the paper used in any of the

  • The evaluation may leak information. Algorithm 1 fits PCA on the full dataset before cross-validation (step 3), which contradicts the reproducibility statement that feature construction is done within each fold. As described, Optuna picks hyperparameters on the same 5-fold cross-validation, with no nested loop, and early stopping uses the evaluation fold (step 20). The data used to fit the calibration is not stated. With about 20 positive cases, each of these can make the results look better than they are. If the reported numbers come from a separate held-out set, the paper should say so.

  • The prospective results are weak. AUPRC is 0.075 for TCMR and 0.077 for ABMR, only slightly above the random baseline (0.049 and 0.037, Table 4). The model catches 3 of 4 TCMR cases and all 3 ABMR cases, but it does so by also flagging 32 of 78 patients without TCMR and 48 of 79 without ABMR. The introduction still claims the model could "revolutionize donor-recipient matching", and Section 7.2 calls it "clinically superior".

  • The four stages are never ablated. The paper does not test KLEAR without oversampling, with SMOTE or class weights instead, without leaf features, or with cross-entropy instead of focal loss. Using focal loss on already balanced data corrects for imbalance twice. It is unclear which stages matter.

  • The ML novelty is limited. Feeding tree leaf indices into another model is the well-known GBDT+LR approach (He et al., 2014), which is not cited, and VAE or GMM oversampling is standard. The baselines are also weak for small tabular data (LightGBM, CatBoost or TabPFN).

  • Some results look unstable. The best TCMR setup in Table 2 has the highest AUPRC but a below-chance ROC-AUC of 0.46. XGBoost in Table 1 shows the same pattern (AUPRC 0.26, ROC-AUC 0.43). This suggests that a few positive cases rank at the top while the rest rank low, so the AUPRC may not hold on a different split. The same KLEAR setup also drops from AUPRC 0.35 on 319 patients (Table 1) to 0.264 on 285 (Table 2), and the paper does not explain why.

  • I am not convinced by the interpretation of the site analysis (Table 3). AUPRC rises with prevalence, so Florida's higher AUPRC (prevalence 0.12 vs. 0.04) is expected and does not show model robustness. Rochester contributes 443 patients to the 2,013 total but appears in none of the high-resolution analyses.

  • The top two TCMR predictors are recipient race and donor race (Figure 8a), larger than any E3 feature. This weakens the claim that molecular features are what drive the predictions. Oversampling does not address racial fairness, and there is no subgroup analysis.

  • Several method details are missing or inconsistent. The E3 framework, which produces the core features, is never cited, and how it computes them is not explained. The abstract mentions a Bayesian GMM, but Section 4.3 uses an ordinary GMM. The focal loss in Figure 1 has a (1-α) weight that Eq. (6) lacks.

Questions for authors
  1. Was hyperparameter tuning nested inside the cross-validation? Was PCA refit inside each fold, and what data was the calibration fitted on?
  2. What are the results on the imputed and low-resolution cohorts? Can you compare high-resolution, imputed and low-resolution typing on the same patients, as Section 5.2 suggests?
  3. Can you report paired comparisons of KLEAR against logistic regression and against clinical data alone, and show how stable the rankings are over repeated cross-validation with different seeds?
  4. How does performance change without oversampling, without leaf features, or without focal loss? How do LightGBM, CatBoost and TabPFN compare?
  5. What is the prospective ROC-AUC, and how did the bootstrap handle resamples with zero or one positive case?
  6. How does the TCMR model perform without race as an input, and within racial subgroups?
  7. What is the E3 framework, and is it publicly available?
  8. Why does the best TCMR setup in Table 2 have ROC-AUC 0.46 but the highest AUPRC?
2 comments

Response to Reviewer 5zoo - Part I

Q1: "Was hyperparameter tuning nested inside the cross-validation? Was PCA refit inside each fold, and what data was the calibration fitted on?"

Answer: The submitted pseudocode placed PCA before cross-validation. The corrected evaluation separates tuning and calibration from outer evaluation: Optuna selection uses inner folds, and calibration and thresholds use development-only cross-fitted predictions. Imputation, standardization and categorical encoders are fitted within the corresponding training partitions; however, the E3 principal components were precomputed rather than refit within each fold. The submitted manuscript names isotonic regression, while a companion figure script uses sigmoid calibration; we have documented this discrepancy in the Figure 7 caption. Figure 7 shows the submitted calibration plots, and calibration plots for the corrected models are not included.

Manuscript Sections 4.2 and 4.6, Appendix C and Appendix F, Figure 7.


Q2: "What are the results on the imputed and low-resolution cohorts? Can you compare high-resolution, imputed and low-resolution typing on the same patients, as Section 5.2 suggests?"

Answer: We also describe the full imputed (n = 1,263) and low-resolution (n = 1,340) HLA cohorts. Table 2 compares models in 285 recipients matched by clinical variables across all three HLA datasets, using identical folds and outcomes (21 TCMR and 12 ABMR events).

Table 2 Panel A reports the input feature-set analysis within a 285-recipient subset with shared patient folds. It includes full E3 features from high-resolution typing and partial E3 features from imputed typing; for TCMR with clinical predictors, AUPRC was 0.358 versus 0.257. Panel B compares high resolution (DRCs with full E3-MoFs) with low resolution (DRCs without HLA-derived inputs) on the same 285 recipients, endpoints and folds. Event counts now appear in the Outcome column.

Panel B now reports AUPRC increases from low to high resolution of 12.2% for TCMR and 6.7% for ABMR. Section 5.2 retains the paired absolute AUPRC differences and 95% intervals: 0.039 (−0.184 to 0.262) for TCMR and 0.022 (−0.097 to 0.140) for ABMR. The prospective results in Table 4 are for the high-resolution KLEAR model in Table 1 (AUPRC 0.065 for TCMR and 0.074 for ABMR); they are not an input-resolution comparison. These estimates are exploratory because there were four TCMR and three ABMR events. The low-resolution setting contains no HLA-derived inputs, so the development comparison estimates the added value of high-resolution molecular features over clinical characteristics and does not isolate typing resolution. A paired imputed arm and a comparison with resolution-specific HLA inputs are not included in this revision.

Manuscript Section 3.1 and Section 5.2, “this comparison estimates the added value of high-resolution molecular features over clinical characteristics and does not isolate typing resolution.” Tables 2,4.


Q3: "Can you report paired comparisons of KLEAR against logistic regression and against clinical data alone, and show how stable the rankings are over repeated cross-validation with different seeds?"

Answer: We agree that the submitted results and overlapping single-model intervals leave comparative performance unresolved. In the corrected Table 1, KLEAR’s development intervals overlap those of logistic regression and the other baselines: TCMR AUPRC was 0.143 versus 0.080, and ABMR AUPRC was 0.069 versus 0.058. The paired comparison with clinical inputs alone is Table 2 Panel B (Q2); Table 4 instead reports the high-resolution KLEAR prospective operating-point results from Table 1. This revision does not include a paired comparison of KLEAR with logistic regression or repeated-seed development results. Section 7.1 reports prospective-cohort AUPRC for four additional seeds, which does not show a stable advantage. We therefore make no claim that KLEAR outperforms simpler models.

Manuscript Sections 5.1, 5.2 and 7.1, and Tables 1, 2 and 4.


Q4: "How does performance change without oversampling, without leaf features, or without focal loss? How do LightGBM, CatBoost and TabPFN compare?"

Answer: The revision cites He et al. (2014) and frames the tree-leaf representation as prior work applied with a neural classifier in this setting. The archived companion implementation applies the same focal-loss multiplier to both classes and weights examples by prediction difficulty, and Equation 6 now matches this implementation. The abstract now describes a Gaussian mixture model rather than a Bayesian Gaussian mixture. We agree that ablations of oversampling, leaf features and focal loss, and benchmarks against LightGBM, CatBoost and TabPFN, are needed to establish the value of the combined design. These are not included in this revision.

Manuscript Abstract, Section 2, Section 4.5 and Equation 6.

Response to Reviewer 5zoo - Part II

Q5: "What is the prospective ROC-AUC, and how did the bootstrap handle resamples with zero or one positive case?"

Answer: There were four TCMR and three ABMR events in the prospective validation cohort. Table 1 now reports prospective-cohort ROC-AUC for the corrected seed-42 models: 0.554 for TCMR and 0.679 for ABMR for KLEAR, and 0.346–0.420 and 0.430–0.620 for the other classifiers. Table 4 gives the corresponding high-resolution KLEAR AUPRC, sensitivity, specificity and confusion counts in the same cohort. Given the event counts, these are exploratory point estimates.

The original submitted AUPRC values, summarized as mean ± SD over 1,000 bootstrap replicates, remain described in Section 7.1; Table 4 now shows corrected high-resolution KLEAR point estimates and operating-point counts instead. The manuscript does not describe how replicates with zero or one positive case were handled. With so few events, the submitted operating points and decision curves support only an exploratory interpretation. The revised text reports the false-positive burden, and Section 7.2 states that clinical net benefit remains undetermined. We have removed the “revolutionize” and clinical-superiority language.

Manuscript Sections 7.1–7.2, Tables 1 and 4 and Figure 2.


Q6: "How does the TCMR model perform without race as an input, and within racial subgroups?"

Answer: The revision states that donor and recipient race were prominent model inputs and that outcome oversampling provides no evidence of demographic fairness. We agree that a matched refit removing both variables is the appropriate sensitivity analysis. This analysis and subgroup results are not included in this revision.

Manuscript Sections 6.1 and 8, and Figure 8.


Q7: "What is the E3 framework, and is it publicly available?"

Answer: Section 3.2 now explains how E3 uses typed HLA alleles and aligned donor–recipient sequences to derive residue and physicochemical mismatch features. In the high-resolution analysis, E3 generated 95 features from two-field typing across nine HLA loci. Section 3.2 distinguishes class I α1/α2-restricted summaries from full-molecule features and describes the epitope-related inputs. We cite Jucaud et al. (2020, ATC abstract 103) for earlier use of E3, and Appendix E.2 lists the model-specific inputs. An availability statement for E3 is not included in this revision.

Manuscript Section 3.2, beginning “E3 is a computational framework,” Appendix E.2 and References.


Q8: "Why does the best TCMR setup in Table 2 have ROC-AUC 0.46 but the highest AUPRC?"

Answer: A small number of high-ranked cases can raise precision at some recall levels even when the overall positive–negative ranking is weak; Section 5.2 now notes this sensitivity. In the rerun Table 2, this setting has AUPRC 0.358 and ROC-AUC 0.701. Table 2 differs from Table 1 because it uses the 285-recipient subset (21 TCMR events) without nested cross-validation, whereas Table 1 uses all 319 recipients with nested cross-validation. The similar XGBoost pattern in the submitted Table 1 (AUPRC 0.26, ROC-AUC 0.43) does not appear in the corrected Table 1 (0.083 and 0.511). We do not infer a reliable improvement from these estimates.

Manuscript Section 5.2, paragraph beginning “For TCMR, the DRC plus full E3 feature set.” Tables 1 and 2.


Additional comment: Site analysis and cohort coverage

Reviewer comment: Florida has higher TCMR prevalence than Arizona, and Rochester is absent from the reported high-resolution site analysis.

Answer: We revised the text to show the event counts alongside the AUPRC estimates and to describe Table 3 as a site-stratified development analysis, now with 95% confidence intervals. The TCMR prevalence was 13/106 in Florida and 9/213 in Arizona, with AUPRC of 0.480 (0.21–0.74) and 0.083 (0.02–0.27); the Florida ABMR estimate rests on three events. Site membership was inferred from patient identifiers and requires confirmation against source records. The AUPRC difference cannot establish transportability, and a site-held-out analysis is not included. Rochester contributes to the broader source population and was absent from this high-resolution comparison.

Manuscript Sections 3.1 and 5.3, and Table 3.

Reviewer_ogGAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper aims to predict whether a deceased-donor kidney transplant recipient will develop T cell-mediated rejection or antibody-mediated rejection within one year before transplantation. Thus the authors introduces KLEAR, a machine learning framework which combines high-resolution HLA molecular mismatch features with donor-recipient clinical variables, VAE-GMM oversampling, XGBoost leaf representations, and a neural network. The results suggest that molecular-level HLA features provide more predictive information than conventional HLA mismatch counts.

Critical strengths
  1. The paper addresses a clinically meaningful problem of predicting acute rejection before kidney transplantation. Many established risk models rely on post-transplant information, such as biopsy findings, donor-specific antibodies, proteinuria, or graft function, which limits their usefulness when making decisions before transplantation. Therefore, a key strength of this paper is that it provides a valuable preliminary framework for pre-transplant risk stratification, with potential relevance to donor-recipient assessment, individualized surveillance, and immunosuppression planning.

  2. Rather than relying only on conventional antigen-level HLA mismatch counts, the paper uses high-resolution HLA typing to construct molecular features describing amino-acid, electrostatic, hydrophobic, eplet, and indirect T-cell recognition differences between donors and recipients. It also develops separate feature sets for TCMR and ABMR, reflecting their different immunological mechanisms. The comparison on a fixed patient cohort suggests that these richer molecular features contain additional predictive information beyond conventional HLA mismatch counts, HLAMatchmaker scores, or PIRCHE-II alone.

Critical weaknesses
  1. The released code appears to use the test set during model training and evaluation, including for early stopping, threshold selection, and probability calibration.

  2. The VAE-GMM approach is a reasonable attempt to mitigate class imbalance during optimization. However, oversampling cannot fully compensate for the small number of independent rejection events, particularly for ABMR. The primary models are developed with only 22 TCMR and 12 ABMR events while using a highly adaptive multi-stage pipeline. The paper does not clearly demonstrate nested evaluation for Optuna tuning, model selection, isotonic calibration, and operating-threshold selection, nor does it provide paired statistical comparisons against primary baselines. Development confidence intervals are extremely wide and often overlap weak or random performance. In the prospective cohort, AUPRC is only 0.075/0.077, with 32 and 48 false positives and specificity of 59.0%/39.2% for TCMR/ABMR. With only four and three positive events, bootstrap dispersion around AUPRC does not establish robust discrimination, calibration, or decision benefit.

Questions for authors
  1. The released code appears to use the test set for VAE and XGBoost early stopping, threshold selection, and probability calibration. If this test set corresponds to the 82-patient prospective cohort, the reported performance may not represent a fully independent evaluation. Please clarify the exact data flow and identify which data were used for training, hyperparameter selection, calibration, threshold determination, and final evaluation.

  2. How do the authors justify the use of a highly flexible VAE-GMM–XGBoost–DNN pipeline given the limited number of independent rejection events? In particular, each ABMR training fold contains only approximately 9–10 observed positive cases. Synthetic oversampling cannot add independent clinical information and may amplify sample-specific patterns. Please provide evidence that the results are stable across random seeds and that VAE-GMM offers a consistent advantage over no oversampling and simpler resampling methods.

  3. Were the biopsies protocol-based or performed because of clinical suspicion? Excluding patients without a biopsy within one year may introduce verification and selection bias, particularly if biopsies were indication-driven. Please clarify the biopsy criteria and discuss how this selection affects the model’s generalizability.

  4. Please provide component-wise ablation experiments for KLEAR. Although standalone XGBoost and DNN baselines are reported, the individual contributions of VAE-GMM oversampling, XGBoost leaf features, focal loss, and their combination remain unclear. Comparisons with no oversampling, simpler balancing methods, and versions removing each component would help establish which parts of the pipeline drive the reported improvement.

  5. How were hyperparameter optimization and performance estimation separated? Section 4.6 states that Optuna optimization was performed with five-fold cross-validation. Please clarify whether the performance reported in Tables 1 and 2 was estimated using the same cross-validation folds used for hyperparameter selection.

  6. Recipient and donor race are the two most influential predictors in the TCMR model, yet performance and calibration are not reported across racial groups. Please evaluate subgroup performance and report whether model accuracy changes when race variables are removed.

3 comments

Response to Reviewer ogGA - Part I

Q1: "The released code appears to use the test set for VAE and XGBoost early stopping, threshold selection, and probability calibration. If this test set corresponds to the 82-patient prospective cohort, the reported performance may not represent a fully independent evaluation. Please clarify the exact data flow and identify which data were used for training, hyperparameter selection, calibration, threshold determination, and final evaluation."

Answer: We thank the reviewer for examining the released code. The reviewer is correct: the submitted code used the outcomes of the 82 recipients in the prospective validation cohort for early stopping, threshold selection and calibration. Section 7.1 describes the original result as a temporal-cohort analysis. Table 4 reports the corrected high-resolution KLEAR evaluation using development-selected thresholds; it does not reproduce the submitted bootstrap rows.

In the corrected protocol, the prospective validation cohort is held aside. Models are fitted, tuned and thresholded on development data only, and each frozen outer-fold pipeline is applied to the cohort once. Corrected seed-42 AUPRC was 0.065 for TCMR and 0.074 for ABMR (prevalences 0.049 and 0.037), and no classifier identified a rejection event at its development-selected threshold. Table 4 gives the corresponding sensitivity, specificity and confusion counts for the Table 1 KLEAR rows. Because this cohort was examined in earlier analyses, these results are exploratory, and independent validation will require a fresh untouched cohort.

Manuscript Section 7 opening paragraph, Section 8, Tables 1 and 4, Figure 4 caption and Appendix C, Algorithm 1, steps 2 and 7.


Q2: "How do the authors justify the use of a highly flexible VAE-GMM–XGBoost–DNN pipeline given the limited number of independent rejection events? In particular, each ABMR training fold contains only approximately 9–10 observed positive cases. Synthetic oversampling cannot add independent clinical information and may amplify sample-specific patterns. Please provide evidence that the results are stable across random seeds and that VAE-GMM offers a consistent advantage over no oversampling and simpler resampling methods."

Answer: We agree that the number of observed rejection events limits what a flexible pipeline can learn. Synthetic rows contribute to model fitting and add no independent rejection events, and the revised manuscript reports all estimates as exploratory. The corrected development results in Table 1 are from a single training seed. Section 7.1 reports prospective-cohort AUPRC for four additional seeds: 0.040–0.051 for TCMR and 0.038–0.058 for ABMR. Three of the four TCMR values fall below prevalence, so a stable advantage is not established. This revision does not include a multi-seed development analysis or comparisons of VAE–GMM with no oversampling or simpler balancing.

Manuscript Sections 4.3, 7.1 and 8, and Table 1.


Q3: "Were the biopsies protocol-based or performed because of clinical suspicion? Excluding patients without a biopsy within one year may introduce verification and selection bias, particularly if biopsies were indication-driven. Please clarify the biopsy criteria and discuss how this selection affects the model’s generalizability."

Answer: We clarified that the observed outcomes are biopsy-ascertained and that the eligibility criteria select a subset of transplant recipients. Differences in biopsy indication and surveillance practice may alter the likelihood of detecting rejection, and Section 8 notes the resulting limits on applicability. We therefore limit interpretation to the biopsy-selected population. The revision does not distinguish protocol from indication biopsies by site.

Manuscript Section 3, “The biopsy and follow-up requirements may select recipients with different risks or opportunities for diagnosis.” Appendix D.1 and Section 8.

Response to Reviewer ogGA - Part II

Q4: "Please provide component-wise ablation experiments for KLEAR. Although standalone XGBoost and DNN baselines are reported, the individual contributions of VAE-GMM oversampling, XGBoost leaf features, focal loss, and their combination remain unclear. Comparisons with no oversampling, simpler balancing methods, and versions removing each component would help establish which parts of the pipeline drive the reported improvement."

Answer: We agree that the standalone XGBoost and DNN comparisons do not isolate the pipeline components. We appreciate the recommended design, which compares VAE–GMM with no oversampling and with simpler balancing, removes the leaf features, and replaces focal loss with binary cross-entropy under the same folds and tuning budget. These ablations are not included in this revision. The existing results cannot establish which components contribute, and the Conclusion states that the contribution of individual components remains undetermined.

Manuscript Table 1 and Conclusion.


Q5: "How were hyperparameter optimization and performance estimation separated? Section 4.6 states that Optuna optimization was performed with five-fold cross-validation. Please clarify whether the performance reported in Tables 1 and 2 was estimated using the same cross-validation folds used for hyperparameter selection."

Answer: The submitted report did not establish whether the Optuna tuning folds were also used for the reported estimates. The corrected Table 1 uses nested cross-validation: inner folds support hyperparameter selection (1,000 Optuna trials), and outer holdouts support performance estimation. Table 2 is not nested; its KLEAR estimates follow trial selection on the same folds and are reported as descriptive.

Manuscript Sections 4.6 and 5.2, and Appendix C, Algorithm 1, steps 2–5.


Q6: "Recipient and donor race are the two most influential predictors in the TCMR model, yet performance and calibration are not reported across racial groups. Please evaluate subgroup performance and report whether model accuracy changes when race variables are removed."

Answer: We revised the interpretation of Figure 8. The SHAP rankings describe how the fitted TCMR model uses the race inputs. They cannot establish why those variables contribute or how accurately the model performs across groups. Section 8 states that oversampling provides no evidence of demographic fairness. We agree that a race-removal analysis and subgroup evaluation are warranted; they are not included in this revision.

Manuscript Sections 6.1 and 8. Figure 8 caption.

Thank you for the candid response and the corrections to the evaluation protocol. However, my main concerns remain unresolved. The corrected prospective-cohort results do not establish a stable advantage over prevalence baselines, and no rejection events were detected at the development-selected thresholds. The requested component ablations, multi-seed development analysis, and subgroup/race-removal evaluations remain absent. Although the clarifications and more cautious claims are welcome, they do not provide the evidence needed to establish the method’s effectiveness. I therefore maintain my original recommendation.

Reviewer_gMLVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

KLEAR predicts first-year T cell–mediated rejection (TCMR) and antibody-mediated rejection (ABMR) using donor–recipient characteristics and phenotype-specific molecular HLA mismatch features. The pipeline combines PCA, VAE–GMM minority oversampling, XGBoost leaf encodings, and a dense neural network. Although the source population comprises 2,013 recipients, the principal model-development analysis uses 319 patients with 22 TCMR and 12 ABMR events. Development AUPRCs are 0.35 and 0.30; a temporally subsequent, single-site cohort of 82 patients yields AUPRCs of 0.075 and 0.077. Same-patient feature comparisons suggest potential value from molecular HLA information, but the evidence remains preliminary.

Critical strengths

The application is consequential, and distinguishing TCMR from ABMR through biologically motivated molecular feature sets is useful. The same-patient comparisons in Table 2 provide a more informative assessment of molecular features than comparisons across different HLA cohorts would.

The temporally subsequent validation cohort provides a valuable check beyond development cross-validation. Reporting its small event count, modest AUPRCs, and confusion matrices makes the limits of generalization visible.

Critical weaknesses

The validation does not establish reliable predictive or clinical benefit. Development relies on only 22 TCMR and 12 ABMR events, with wide confidence intervals and no paired uncertainty estimates for improvements over competing models or feature sets. Prospective validation contains only four and three events, with AUPRCs falling to 0.075 and 0.077. At the reported operating points, the positive predictive values are only 3/35 (8.6%) and 3/51 (5.9%). Figure 2 provides no uncertainty estimates, and these data do not substantiate Section 7.2’s claim of clinical superiority. Biopsy and one-year follow-up requirements also restrict the population to which the results apply.

The evaluation protocol and component contributions remain insufficiently established. Although the text states that preprocessing is fold-specific, Algorithm 1 places PCA before cross-validation. The manuscript does not clearly separate hyperparameter selection, early stopping, isotonic calibration, and final performance estimation. These ambiguities matter particularly with so few positive cases. Moreover, standalone XGBoost and DNN comparisons do not isolate the value of VAE–GMM oversampling, leaf-feature fusion, or focal loss; matched comparisons with class weighting and simpler oversampling are missing. Consequently, the results do not yet establish that the proposed pipeline is responsible for the reported gains.

Questions for authors

Evaluation clarification: Were Tables 1–3 calculated from untouched outer-fold predictions, with Optuna tuning and early stopping confined to inner splits? Specify the samples used to fit PCA, target encoders, and isotonic calibration, and reconcile Algorithm 1 with the fold-specific preprocessing statement. How were confidence intervals constructed?

Evidence needed to assess the contribution: Can you provide paired AUPRC differences with uncertainty for KLEAR versus the strongest baselines and for molecular versus clinical-only inputs? Under identical splits and tuning budgets, how does performance change when replacing VAE–GMM with class weighting or simple oversampling, and when removing leaf features or focal loss?

Prospective clarification: Was the complete model, including calibration and thresholds, frozen before prospective outcomes were examined? Report threshold-selection procedures and uncertainty for prospective performance and net benefit. Which predictors were available at the intended decision time, particularly final cold ischemia time and induction therapy if these were model inputs?

Cohort and governance clarification: Please specify the Banff version, handling of borderline and mixed rejection, biopsy practices, and exclusions for death or graft loss before one year. Also state ethics approval and consent/waiver arrangements. What evidence supports the claim that outcome oversampling mitigates demographic bias, given the prominent contribution of race?

2 comments

Response to Reviewer gMLV - Part I

Q1: "Evaluation clarification: Were Tables 1–3 calculated from untouched outer-fold predictions, with Optuna tuning and early stopping confined to inner splits? Specify the samples used to fit PCA, target encoders, and isotonic calibration, and reconcile Algorithm 1 with the fold-specific preprocessing statement. How were confidence intervals constructed?"

Answer: The submitted analysis did not separate these steps. The submitted pseudocode placed PCA before the cross-validation loop, we did not identify separate data for stopping, tuning, calibration and scoring, and the submitted code used the prospective-cohort outcomes for early stopping, threshold selection and calibration. The corrected benchmark in Table 1 uses patient-level nested cross-validation with five outer and five inner folds, with outer folds used only for evaluation. Imputation, standardization, categorical encoding, oversampling, early stopping and Optuna selection (1,000 trials) are confined to the appropriate training partitions, and calibration and threshold selection use development-only predictions, with the outer evaluation outcomes held out.

Table 1 reports bootstrap 95% intervals, and Table 2 reports bounded t intervals across the five validation folds. Corrected development AUPRC is 0.143 (95% CI 0.05–0.29) for TCMR and 0.069 (0.03–0.17) for ABMR, against prevalences of 0.07 and 0.04. Because the cohort reflects routine clinical care, rejection is rare and the outcome classes are highly imbalanced, so the expected AUPRC of a random classifier equals the prevalence. KLEAR’s development AUPRC is about twice the prevalence for TCMR (0.143 versus 0.07) and about 1.7 times the prevalence for ABMR (0.069 versus 0.04). The nested design, with outer folds reserved for evaluation, guards against the optimistic estimates that tuning on evaluation data would produce. However, the lower bound of each interval falls below prevalence, so improvement over a random classifier is not statistically established. The two limitations remain, the E3 principal components were precomputed rather than refit within each training fold, and Table 2 is not nested and is therefore reported as descriptive, were also listed into the limitation section of the manuscript.

Table 3 uses separate within-site development fits in Florida (106 recipients) and Arizona (213). It does not use site-held-out predictions or reuse Table 1’s pooled outer-fold predictions. The Florida ABMR analysis used three folds because only three recipients had ABMR; its AUPRC was 0.359 (95% CI 0.01–1.00). The site assignments were inferred from patient identifiers. The report gives site-specific 95% intervals but does not establish whether each site’s scoring folds were held out from all tuning and early stopping, or specify how those intervals were constructed. We therefore interpret Table 3 descriptively.

Manuscript Sections 4.2, 4.6, 5.2, 5.3 and 7, and Tables 1–3. Appendix C, Algorithm 1.


Q2: "Evidence needed to assess the contribution: Can you provide paired AUPRC differences with uncertainty for KLEAR versus the strongest baselines and for molecular versus clinical-only inputs? Under identical splits and tuning budgets, how does performance change when replacing VAE–GMM with class weighting or simple oversampling, and when removing leaf features or focal loss?"

Answer: We agree that the submitted single-model intervals do not quantify paired differences. The paired comparison in this revision is Table 2 Panel B, which compares clinical predictors with and without high-resolution E3 features on the same 285 recipients and patient-level folds. Panel B now reports AUPRC increases from low to high resolution of 12.2% for TCMR and 6.7% for ABMR, calculated from the development cross-validation means before three-decimal table rounding. The paired absolute AUPRC differences remain in Section 5.2: 0.039 (95% CI −0.184 to 0.262) for TCMR and 0.022 (−0.097 to 0.140) for ABMR. Neither interval excludes zero, so the incremental contribution of E3 is not established. This revision does not include a paired comparison of KLEAR with the strongest baseline, or component-wise comparisons that remove VAE–GMM oversampling or leaf features or replace focal loss with binary cross-entropy. In the corrected Table 1, KLEAR’s intervals overlap those of all baselines, and the DNN and elastic net have higher ABMR point estimates. We therefore make no claim of superiority, and the Conclusion states that the contribution of individual components remains undetermined.

Manuscript Sections 5.1, 5.2 and 7.1, Tables 1, 2 and 4, and Conclusion.

Response to Reviewer gMLV - Part II

Q3: "Prospective clarification: Was the complete model, including calibration and thresholds, frozen before prospective outcomes were examined? Report threshold-selection procedures and uncertainty for prospective performance and net benefit. Which predictors were available at the intended decision time, particularly final cold ischemia time and induction therapy if these were model inputs?"

Answer: The audit confirmed that the submitted code used outcomes from the 82 recipients in the prospective validation cohort for early stopping, threshold selection and calibration. The submitted model and thresholds were therefore not frozen before these outcomes were used, and we now describe this as a temporal-cohort analysis rather than independent validation. At the submitted operating points, positive predictive value was 3/35 (8.6%) for TCMR and 3/51 (5.9%) for ABMR.

The archived implementation maximizes rejection-class F1 over precision–recall thresholds when the positive fraction is below 10%; otherwise it maximizes Youden’s J (sensitivity + specificity − 1). Both prospective endpoints met the F1 criterion in the submitted evaluation. For the corrected seed-42 evaluation, the thresholds were selected on development predictions and frozen before prospective scoring. We can verify that separation, but cannot confirm the optimization criterion for the corrected run from the available record.

In the corrected protocol, the prospective validation cohort is held aside, and each outer-fold pipeline is frozen and applied once. Corrected seed-42 AUPRC was 0.065 for TCMR and 0.074 for ABMR (prevalences 0.049 and 0.037), ROC-AUC was 0.554 and 0.679, and no classifier identified a rejection event at its development-selected threshold. Four additional seeds gave AUPRC of 0.040–0.051 for TCMR and 0.038–0.058 for ABMR. The AUPRC exceeds the random-classifier baseline (prevalence) in both endpoints, by about 1.3 times for TCMR and 2 times for ABMR. Given four and three events, these are point estimates and remain exploratory.

The decision curves in Figure 2 are from the submitted analysis and were not recomputed with uncertainty bands; Section 7.2 states that clinical net benefit remains undetermined. Section 4.1 now specifies a pre-transplant prediction time and notes that actual induction treatment and final cold ischemia time may become known only after that point. This revision does not include a model refitted without these inputs.

Manuscript Section 4.1 and Tables 6 and 9 for predictor timing. Sections 7.1–7.2, Tables 1 and 4, Figure 2 and Appendix C, Algorithm 1, for the prospective validation cohort.


Q4: "Cohort and governance clarification: Please specify the Banff version, handling of borderline and mixed rejection, biopsy practices, and exclusions for death or graft loss before one year. Also state ethics approval and consent/waiver arrangements. What evidence supports the claim that outcome oversampling mitigates demographic bias, given the prominent contribution of race?"

Answer: Eligibility required at least one recorded biopsy and one year of follow-up. Recipients whose follow-up ended before one year, including those who died before that point, did not qualify. For graft loss before one year, the follow-up criterion alone does not establish exclusion if observation continued. We cannot give a separate graft-loss exclusion rule or counts by reason from the reported cohort flow.

This study was approved by the Mayo Clinic Institutional Review Board (IRB). De-identified Electronic Health Record data were used; patient consent was obtained per the IRB-approved protocol. All procedures complied with the Declaration of Helsinki.

We now describe the population as recipients with a recorded biopsy and one year of follow-up, with outcomes defined as first-year, biopsy-ascertained TCMR and ABMR under Banff-based classification. Biopsy indication and follow-up eligibility may affect which rejection events are observed, and Section 8 notes the resulting limits for recipients with different surveillance, early death or graft loss. The revision does not specify the Banff version or the handling of borderline and mixed rejection. We have also removed the claim that balancing outcome labels addresses demographic bias; Section 8 now states that oversampling provides no evidence of demographic fairness. We agree that the high SHAP rankings for donor and recipient race motivate a race-removal analysis and a subgroup assessment. These analyses are not included in this revision.

Manuscript Sections 3, 6.1 and 8, Appendix D.1 and Ethics statement.

Comments on the paper

1 comment

Response to Reviewers

Dear Area Chair and Reviewers,

Thank you for reviewing our study of first-year T cell-mediated rejection (TCMR) and antibody-mediated rejection (ABMR) after deceased-donor kidney transplantation. The study draws on clinical data from kidney transplant recipients treated at Mayo Clinic, one of the largest organ transplantation centers in the world. To our knowledge, the 319-recipient development cohort, with high-resolution HLA typing of donors and recipients across nine loci, is among the largest kidney transplantation cohorts with high-resolution HLA typing. The separately accrued 82-recipient prospective cohort, also typed at high resolution, allows the models to be assessed in recipients outside the development data. Data of this kind are uncommon in transplantation research, and we believe this resource can directly support clinical researchers studying rejection risk, although, as described below, results in the prospective cohort remain exploratory. We revised the manuscript to identify the 319-recipient development cohort and the 82-recipient prospective validation cohort within the broader Mayo source population, clarify how we derived the E3 molecular mismatch features, and qualify our claims about validation and clinical utility. We also corrected the evaluation. Table 1 now reports a patient-level nested cross-validation benchmark, and the prospective validation cohort is evaluated with models fitted, tuned and thresholded without its outcomes. Table 2 now separates the feature-set analysis (Panel A) from a paired high- versus low-resolution comparison (Panel B). Table 3 reports site-level confidence intervals, and Table 4 reports the corrected high-resolution KLEAR operating-point results in the prospective validation cohort. Several analyses the reviewers recommended, including component ablations, stronger tabular baselines and race-removal and subgroup analyses, are not part of this revision; we acknowledge these limitations in the individual responses.

Sincerely,

The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.