acceptodds
Under review as a conference paper at ICLR 2027

Learning Potentials for Dynamic Matching and Application to Heart Transplantation

Abstract

Each year, thousands of patients in need of heart transplants face life-threatening wait times due to organ scarcity. While allocation policies aim to maximize population-level outcomes, current approaches often fail to account for the dynamic arrival of organs and the composition of waitlisted candidates, thereby hampering efficiency. The United States is transitioning from rigid, rule-based allocation to more flexible data-driven models. In this paper, we propose a novel framework for non-myopic policy optimization in general online matching relying on potentials, a concept originally introduced for kidney exchange. We develop scalable and accurate ways of learning potentials that are higher-dimensional and more expressive than prior approaches. Our approach is a form of self-supervised imitation learning: the potentials are trained to mimic an omniscient algorithm that has perfect foresight. We focus on the application of heart transplant allocation and demonstrate, using real historical data, that our policies significantly outperform prior approaches—including the current US status quo policy and the proposed continuous distribution framework—in optimizing for population-level outcomes. Our analysis and methods come at a pivotal moment in US policy, as the current heart transplant allocation system is under review. We propose a scalable and theoretically grounded path toward more effective organ allocation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Matching, organ allocation, imitation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sogrOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents a learning and optimization problem, designed for decision support in online heart transplant programmes.

The work builds over literature about kidney transplant programs, and in particular on the idea that cascade effect from decisions can be efficiently approximated by adjusting the cost function of the optimization problem solved at each step, rather than by introducing a more expensive look ahead component.

The adjustments take the form or "potential", which are modeled as black-box functions depending on a variety of terms (patient, donor, and queue features) and implemented as ML models (linear, NNs).

The models are trained by imitation learning, trying to replicate the ranking performed by a clairvoyant optimization model, which needs to solve a large-scale assignment problem with perfect information. Data augmentation techniques, based on the idea of subsampling and readjusting the arrival times for real world time series, are used to improve the robustness of the fit.

The results show that the family of approaches introduced in this paper provides very significant improvements w.r.t. both the current organ assignment rule and its proposed replacement; however, some details of the empirical evaluation protocol may have been underestimated in the interpretation of its outcome.

Critical strengths

The proposed approach is both clever and sound: I was already familiar with the related literature in kidney exchanges, and I always felt that the idea of using potential as a cheap replacement for look-ahead steps in online optimization has been underappreciated.

The proposed training technique is indeed very efficient: heart transplantation optimization problems have a much simpler structure compared to kidney transplantation ones, thus making it viable to run a clairvoyant solver even on large scale problems.

Imitation learning is also much more efficient than using black-box optimization over a simulator. In theory, using imitation learning may lead to suboptimal results (due to the possible misalignment between the optimal decisions made under perfect and imperfect information), but the fact that most of the gap w.r.t. the clairvoyant solution is closed by the proposed strategy suggests that the degree of misalignment is limited in this case.

The results are also very good, though with a critical caveat that I'll discuss in the weaknesses section.

Critical weaknesses

While the paper is sound and well presented, I am afraid it suffers from a critical weaknesses that prevents me from recommending acceptance, at least in its current state.

The issue concerns the underlying simulation model and its interaction with the optimization problem formulation. Specifically, the authors are relying on a Cox model for estimating survival (a reasonable choice), but they never seem to account for: 1) epistemic uncertainty on the fitted model itself; and 2) epistemic uncertainty in the estimates used by the optimization problem.

The former point (no epistemic uncertainty on the Cox model) is a minor issue in my opinion: on the one hand, it makes the simulation result unrealistically accurate, which might lead to unrealistically good ML models; on the other hand, however, the available dataset is large enough w.r.t. the size of the model input that I would not expect overfitting to be a significant issues (even without data augmentation).

The latter point (no epistemic uncertainty in the optimizer) is unfortunately critical detail: the current evaluation does not allow one to claim that the main contribution (the learned potential) is driving the gain over the baselines. In fact, the results suggest the opposite: most of the gain over the baselines is already achieved by a myopic approach (using the "perfect" utility functions, but not potential); adding potentials on top of that only marginally improves the results.

In summary, the current evidence does not allow one to infer that the proposed contribution is significant enough.

I think this problem could definitely be fixed, but that would require time. The peculiarities of the ICLR reviewing process (which traditionally allows for submitting revised versions of a paper) provide a chance to address the issue, but that remains to be seen.

My main suggestion to improve the paper is to include a full ablation study. By doing that:

  • Both the effectiveness of data augmentation and the risk of using an overly accurate simulator would be tested, to some degree, by running experiments with no data augmentation.
  • The impact of the potentials could be assessed separately from the utility function, by replacing the utility function with one derived by the CAS score or from the current rule; in fact, it may well be the case that potentials have a more significant impact in such a scenario

Ideally, the authors should run experiments where the reliability of the Cox model and of the utility estimates is reduced (for example via noise injection); however, it is difficult to do that in a way that does not trigger additional concerns (e.g. favorable bias in the injected noise, or unbiased noise). Using the baselines to obtain approximate utilities seems a more realistic approach to achieve a similar goal.

Questions for authors

Have you tried running an ablation experiments to separately assess the impact of the data augmentation procedure?

Have you tried running an ablation experiments to separately assess the impact of the optimization based on the (very accurate) utility function? Alternatively, have you tried to introduce noise in the utility estimates?

I am willing to substantially revise my score based on this two key points.

AFTER THE RESPONSE

The authors have clarified some of my concerns, but the main critical point (impact of the potentials) remains not adequately addressed. Since a considerable part of the contribution depends on that, I am not willing to recommend acceptance, though I won't object if someone provides strong reasons to champion the paper

4 comments
AuthorsOpenReview ↗

Author response (1/2)

We thank Reviewer sogr for their thoughtful review and particularly appreciate their recognition that our approach is clever and sound, computationally efficient, and achieves strong empirical results. We believe that some of the reviewer's central concerns are addressed by ablations that are already in the submitted manuscript and the rest are addressed by additional experiments we conducted in response to these reviews that we received from ICLR. We clarify these results below and encourage the reviewer to reconsider their assessment in light of this evidence. We are also happy to rapidly conduct additional experiments should the reviewer want further evidence.

Have you tried running an ablation experiments to separately assess the impact of the data augmentation procedure?

The requested ablation is included in Appendix K.2, Table A5. We explicitly compare training on the original historical trajectory (January–March 2019) against training on semi-synthetic augmented trajectories, while evaluating both on the same held-out historical event trajectories (April–December 2019). One important clarification is that the data augmentation is used exclusively for training and we always evaluate on the real, unseen trajectory of events (not augmented data).

The results of this ablation show that training with data augmentation improves the learned neural network policies. Table A5 also reports the corresponding results for CAS, linear potentials, and the smaller neural network, including configurations where augmentation does not improve performance.

Ideally, the authors should run experiments where the reliability of the Cox model and of the utility estimates is reduced (for example via noise injection); however, it is difficult to do that in a way that does not trigger additional concerns (e.g. favorable bias in the injected noise, or unbiased noise). Using the baselines to obtain approximate utilities seems a more realistic approach to achieve a similar goal.

Have you tried running an ablation experiments to separately assess the impact of the optimization based on the (very accurate) utility function? Alternatively, have you tried to introduce noise in the utility estimates?

To address the reviewer's concern, we conducted an additional sensitivity analysis introducing increasing levels of noise (ϵ\epsilon) into the predicted match utilities. Given an ϵ\epsilon, we randomly sample a value in [−ϵ,ϵ][-\epsilon, \epsilon] and add it to the utility estimate supplied to the algorithms. The evaluation uses the prediction without noise to determine the PLYG. We provide the average PLYG across 9 simulations per ϵ\epsilon in the table below.

The results show that learned linear potentials can retain their advantage over myopic allocation even with substantial noise in the utility estimates. Our SVM potentials outperforms myopic matching at every tested noise level. The more expressive NN(34) performs best without noise but is indeed more sensitive to larger perturbations. Note that above ϵ=20\epsilon = 20, the utility estimate is practically uniform random since graft survival is typically no longer than 20-30 years.

These results give additional evidence that the benefits of learned potentials are not exclusively dependent on perfectly accurate utility estimates and degrade gracefully as noise is injected.

epsilon052050% drop at 50
Omniscient3444----
NN (34)3261317429842771−15.0%
SVM3252319330472832−12.9%
SMAC3246317830412833−12.7%
Myopic3232316230302827−12.5%
CAS1534----
Status quo1387----

See also the responses to the other reviewers for additional comments and new experiments.

AuthorsOpenReview ↗

Author response (2/2)

The issue concerns the underlying simulation model and its interaction with the optimization problem formulation. Specifically, the authors are relying on a Cox model for estimating survival (a reasonable choice), but they never seem to account for: 1) epistemic uncertainty on the fitted model itself; and 2) epistemic uncertainty in the estimates used by the optimization problem. The former point (no epistemic uncertainty on the Cox model) is a minor issue in my opinion: on the one hand, it makes the simulation result unrealistically accurate, which might lead to unrealistically good ML models; on the other hand, however, the available dataset is large enough w.r.t. the size of the model input that I would not expect overfitting to be a significant issues (even without data augmentation).

The latter point (no epistemic uncertainty in the optimizer) is unfortunately critical detail: the current evaluation does not allow one to claim that the main contribution (the learned potential) is driving the gain over the baselines. In fact, the results suggest the opposite: most of the gain over the baselines is already achieved by a myopic approach (using the "perfect" utility functions, but not potential); adding potentials on top of that only marginally improves the results.

The impact of the potentials could be assessed separately from the utility function, by replacing the utility function with one derived by the CAS score or from the current rule; in fact, it may well be the case that potentials have a more significant impact in such a scenario

We do include several experiments to highlight the effectiveness of the potential learning framework in varying settings. In Appendix K.1 (Table A4), we compare imitation learning against black-box optimization while holding the CAS policy class and feature representation fixed (i.e., learning the weights of CAS using our method and the black-box optimization method). Our imitation learning framework achieves a 39.4% improvement over the black-box optimization when optimizing CAS. This provides evidence that our learning procedure offers substantial advantages independent of the expressiveness of the potential functions and the myopic utility term.

We also already include an experiment where we replace the PLYG utility function with a utility function that is a weighted sum of PLYG and the transplant rate for biologically disadvantaged patients (Section J). We not only show that the potential learning framework is still 96.5% competitive with the hindsight optimal policy, but we also show that it is effective at trading off PLYG to improve the fairness for biologically disadvantaged patients. This shows that the potentials are effective not just for the PLYG utility function, but are adaptable and effective for other utility functions.

The improvement over myopic matching should be interpreted in the context of a strong baseline. Our learned potentials recover a statistically significant fraction of the remaining gap to the optimal solution by accounting for future matching opportunities. Put differently, the learned potentials recover roughly 11% of the remaining gap between myopic matching and the omniscient upper bound. Although the relative percentage appears modest, the absolute impact is very meaningful in this application. The improvement corresponds to approximately 22 additional life-years gained per month, or over 260 additional life-years annually. To put these numbers into context, our policy is extending (on average) the life of each transplanted patient by multiple months. Extending every transplanted patient’s life by several months is invaluable.

To provide more perspective on that, we would like to point out that Eli Lilly developed a cancer drug, ALIMTA, that extends life in certain cancer types by 2-5 months on average, and had costs of hundreds of millions of dollars to develop and significant lethal toxicity risks for the patients. This 2-5-month life extension was considered so significant that they were selected as one of the six finalist teams for the 2005 Edelman Award, an INFORMS award that is the leading award in optimization with measured real-world impact.

Reviewer_sogrOpenReview ↗

Thanks for the clarification, but...

Thanks for the clarification.

I had indeed missed the table from Appendix K.2, which addresses my concern about data augmentation.

I am afraid, however, that the comment about the perfect utility remains not adequately addressed:

  1. Centered (and hence unbiased) noise is exactly one of the examples that I made when pointing out that noise injection is difficult to perform without inducing further concerns. That's the main reason why I was suggesting using the other scores as baselines. That said, this specific issue is not critical
  2. The gap between the myopic and the potential-based approach remains very small, in all noise regimes. Again, this points out that the main factor behind the improved results is the utility estimate, rather than the potentials. In turn, that reduces by a large margin the practical significance of several of the paper contributions.

The experiments from K.1 does not appear to match the one I was suggesting (just using classically trained CAS scores and relying on the potentials to fix the mismatch).

About the Eli Lilly example: one thing is achieving this results in a controlled experiment with real patients, another is one on static historical data. Moving from the latter setting to the former may easily eat on most of the reported statistical margin, especially since unbiased estimates (like those assumed in the noise injection experiment) are very difficult to obtain in practice.

All in all, it seems to me that the main conclusion I can draw from the work is about the effectiveness of the utility estimates, not of the potentials.

I will update the my score, but my opinion remains slightly on the negative side.

We thank the reviewer for their follow-up comment, and for raising their score. We also now understand what the reviewer was looking for to show the effectiveness of the potentials. Here we provide new evidence to highlight this using the reviewer’s suggested experiment. We are grateful to the reviewer for suggesting this experiment as we feel the results strengthen the contribution of our paper.

In this experiment, we replace the estimated PLYG utility with the CAS score itself as the immediate utility function, using uniform weights across all features. We then compare greedy allocation, which selects the patient maximizing CAS, against our potential-based policy, which learns non-myopic adjustments to the same CAS utility through imitation of the hindsight-optimal matching.

We use the same setup as the main experiment, training on January to March 2019 and evaluating on April to December 2019. Our learned potentials improve the total CAS utility by 15.9% over greedy allocation, closing 73.1% of the remaining gap to the hindsight optimum and achieving 95.2% of its utility.

We hope these results illustrate not only the effectiveness, but also the flexibility and computational efficiency of our framework which enabled us to conduct these experiments so quickly. Changing the oracle’s objective requires no modification to the imitation learning methodology. We hope this addresses the reviewer's primary concern and we thank the reviewer again for their engagement in this discussion! We will include these experiments in the paper.

PolicyMean CAS UtilityPercent improvement over greedy
Greedy (argmax CAS)447.9—
Learned potentials519.3+15.9%
Omniscient545.6+21.8%
Reviewer_bm4ZOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper studies online heart transplant allocation as dynamic bipartite matching. It proposes potential-based policies, which match each arriving donor to the patient maximizing estimated utility minus a learned potential. Instead of tuning potentials by black-box optimization, as in prior potentials work, the authors compute hindsight-optimal matchings with an omniscient LP over historical or semi-synthetic trajectories. They then train linear or MLP potentials with learning-to-rank losses so the online policy imitates the oracle’s choices. A bootstrap-style augmentation produces semi-synthetic training trajectories from Q1 2019 UNOS data. On real April–December 2019 trajectories, utility is measured as population life-years gained (PLYG), estimated with Cox models. The learned policies outperform the US status quo, an optimized version of the proposed continuous distribution (CAS) score, and a myopic utility maximizer, and reach about 95% of the omniscient upper bound. A fairness-aware objective is also explored.

Critical strengths
  • Novel training procedure in this application. To my knowledge, learning potentials by imitating a hindsight-optimal matching is new in the organ allocation literature. It replaces sample-inefficient black-box search with a supervised problem and makes higher-dimensional, non-linear potentials trainable.

  • Useful upper bound. The omniscient LP gives a principled reference point for interpreting online policies.

  • Presentation. The paper is clearly written, with reasonably detailed appendices on the oracle, augmentation and baselines.

  • Modularity. The framework is agnostic to the utility function, the outcome model and the policy class. It optimizes a fairness-aware objective and learns CAS weights without methodological changes and different outcome estimators could be plugged in directly.

Critical weaknesses
  • PLYG is causal but is estimated without causal methods. PLYG is an individual treatment effect in the cited works, yet, here it is computed as the difference of two Cox models fitted on observational data. Confounding in who gets transplanted and positivity for unseen pairs are not addressed. Also, the same estimated U produces the oracle labels and the evaluation metric, so policies that optimize U are favoured by construction. Results should be framed relative to the outcome model and, ideally, accompanied by a sensitivity analysis.

  • Gains over myopic are small and their uncertainty is under-characterized. NN(34) improves on myopic by only +0.67% (although I would not expect large gains either), and myopic alone already reaches 94.4% of the oracle, which largely explains the “95%” headline. The Wilcoxon test covers month-to-month variation, not training seeds or outcome-model error. Fixed-horizon oracle labels, together with the arrival-time feature, may also encode end-of-horizon artifacts.

  • ML novelty and contribution are limited or unclear. The non-myopic mechanism itself is not new. Potentials exist for dynamic allocation, and scarcity-aware allocation accounting for future organ arrivals exists too. The learning step is standard learning-to-rank on hindsight-oracle labels. It is unclear to me what the precise ML contribution is.

Questions for authors
  • [Clarification] I don't see a significant ML contribution. Could you state this sharply?

  • [Clarification] Could you state which assumptions are necessary to estimate PLYG as the difference in your Cox models fitted on observational data?

  • [Experiments] Could you run additional experiments to ensure that the observed gains come from the oracle-imitation learning and not from time-horizon or seed-train/run artifacts?

  • [Experiments] Could you add a non-myopic, non-linear baseline policy such as OrganITE or OrganSync (with the same PLYG model(s) used for imitation-learning)?

  • [Experiments] How long does optimizing one trajectory/run take? What is the end-to-end training time (including generating oracle labels)?

4 comments
AuthorsOpenReview ↗

Author response (1/3)

We thank reviewer bm4Z for their feedback. We believe our response below can clarify all the concerns of the reviewer. If the reviewer finds our responses satisfactory, we strongly ask the reviewer to please revise their assessment to reflect this.

I don't see a significant ML contribution. Could you state this sharply?

Our contribution is a general, computationally efficient framework for learning non-myopic policies for online matching through self-supervised imitation of hindsight-optimal allocations. Prior potential-based methods rely on black-box optimization (such as using ParamILS or SMAC) which is computationally expensive and limits the expressiveness of the potential functions that can be learned. We develop a new learning paradigm: solving tractable hindsight-optimal matching problems to generate supervision, then learning an online policy from these labels.

The reviewer states that our method is a “Novel training procedure in this application. To my knowledge, learning potentials by imitating a hindsight-optimal matching is new in the organ allocation literature”. Our methods are also a contribution to data-driven online matching more generally. Our method is agnostic to the objective being optimized. This flexibility is one of the primary methodological advantages of our framework, and we show this through changing the objective function of the omniscient oracle (Section J). We believe this combination of a general methodological formulation, computational scalability, and empirical improvements on a very important application constitute a substantive machine learning contribution.

PLYG is an individual treatment effect in the cited works, yet, here it is computed as the difference of two Cox models fitted on observational data. Confounding in who gets transplanted and positivity for unseen pairs are not addressed. Also, the same estimated U produces the oracle labels and the evaluation metric, so policies that optimize U are favoured by construction. Results should be framed relative to the outcome model and, ideally, accompanied by a sensitivity analysis.

Our choice of survival estimation follows the established practice in organ allocation. US transplant policy evaluation has long relied on statistical survival models, specifically Cox regression, fitted to historical registry data, including models developed by the SRTR for OPTN allocation simulations. This is the standard not just in the US, but also abroad. The UK's National Liver Offering Scheme, implemented in 2018, explicitly computes its Transplant Benefit Score as the difference between predicted post-transplant survival and predicted waitlist survival, each obtained from separately fitted Cox proportional hazards models. We deliberately adopt the exact same predictors for our simulator rather than introduce a new outcome estimator, allowing us to focus on the allocation optimization problem. Furthermore, our methodological contribution does not depend on the specific approach used to estimate PLYG. Our framework operates on a utility function supplied to the matching problem, independently of how that utility was estimated. This separation is intentional: improved survival models, including causal treatment effect estimators, can be incorporated directly without changing our policy optimization approach.

We emphasize again the important distinction between estimating the utility function and optimizing an allocation policy with respect to that function. Our contribution addresses the latter problem. We deliberately use established techniques to construct estimated utilities in accordance with the OPTN, while keeping the policy learning framework entirely agnostic to how those utilities are obtained.

AuthorsOpenReview ↗

Author response (2/3)

To further address the reviewer's concern, we conducted an additional sensitivity analysis introducing increasing levels of noise (ϵ\epsilon) into the predicted match utilities. Given an ϵ\epsilon, we randomly sample a value in [−ϵ,ϵ][-\epsilon, \epsilon] and add it to the utility estimate supplied to the algorithms. The evaluation uses the prediction without noise to determine the PLYG. We provide the average PLYG across 9 simulations per ϵ\epsilon in the table below. The results show that learned linear potentials can retain their advantage over myopic allocation even with substantial noise in the utility estimates. Our SVM potentials outperform myopic matching at every tested noise level. The more expressive NN(34) performs best without noise but is indeed more sensitive to larger perturbations. Note that above ϵ=20\epsilon = 20, the utility estimate is practically uniform random since graft survival is typically no longer than 20-30 years. These results give additional evidence that the benefits of learned potentials are not exclusively dependent on perfectly accurate utility estimates and degrade gracefully as noise is injected.

epsilon052050% drop at 50
Omniscient3444----
NN (34)3261317429842771−15.0%
SVM3252319330472832−12.9%
SMAC3246317830412833−12.7%
Myopic3232316230302827−12.5%
CAS1534----
Status quo1387----

Could you state which assumptions are necessary to estimate PLYG as the difference in your Cox models fitted on observational data?

We do not understand the question exactly. Can the reviewer please elaborate? We are doing that estimation in the paper, as described in the paper. We think that the reviewer is probably asking what assumptions are needed to guarantee some theoretical properties, but we are not sure what property(ies) the reviewer means.

Gains over myopic are small and their uncertainty is under-characterized. NN(34) improves on myopic by only +0.67% (although I would not expect large gains either), and myopic alone already reaches 94.4% of the oracle, which largely explains the “95%” headline.

The improvement should be interpreted in the context of a strong baseline. Our learned potentials recover a statistically significant fraction of the remaining gap to optimal by accounting for future matching opportunities. The learned potentials recover roughly 11% of the remaining gap between myopic matching and the omniscient upper bound. Although the relative percentage appears modest, the absolute impact is very meaningful in this application. The improvement corresponds to approximately 22 additional life-years gained per month, or over 260 additional life-years annually. To put these numbers into context, our policy is extending (on average) the life of each transplanted patient by multiple months. Extending every transplanted patient’s life by several months is invaluable.

To provide more perspective on that, we would like to point out that Eli Lilly developed a cancer drug, ALIMTA, that extends life in certain cancer types by 2-5 months on average, and had costs of hundreds of millions of dollars to develop and significant lethal toxicity risks for the patients. This 2-5-month life extension was considered so significant that they were selected as one of the six finalist teams for the 2005 Edelman Award, an INFORMS award that is the leading award in optimization with measured real-world impact.

Could you run additional experiments to ensure that the observed gains come from the oracle-imitation learning and not from time-horizon or seed-train/run artifacts?

We performed the additional experiment where we evaluated all policies over two week horizons in April-June 2019 despite training them for one month horizons from earlier months. These additional experiments show that the improvements from learned potentials persist when the evaluation horizon is reduced from one month to two weeks and are not an artifact of the time horizon. Our neural potential policy continues to outperform myopic matching, albeit due to the shorter horizon, the improvement is more modest. This is to be expected over a shorter horizon as there is less opportunity to plan into the future by saving high utility candidates.

PolicyMean PLYG
Omniscient1793.0
NN (34)1711.1
SVM1709.1
Myopic1707.3
SMAC1698.5
CAS815.7
Status quo664.0
AuthorsOpenReview ↗

Author response (3/3)

Could you add a non-myopic, non-linear baseline policy such as OrganITE or OrganSync (with the same PLYG model(s) used for imitation-learning)?

To address the reviewer's concern, we implemented an adaptation of OrganITE within our heart transplant allocation simulator. For a controlled comparison, we use the same Cox PLYG estimates, rather than OrganITE's treatment effect estimator. The adapted allocation rule incorporates donor scarcity and the similarity of an arriving organ to each patient's preferred organ, estimated using training data from January-March 2019. To estimate organ scarcity, we cluster the donors from January–March 2019 into 20 representative types using K-means. For each patient, we identify their ideal compatible donor type as the one maximizing PLYG using our existing Cox models. We use the hyperparameters a=b=1 as published in the OrganITE paper. We evaluate OrganITE on the same simulations in April-December 2019 and report the mean PLYG below. Both our learned neural and linear potentials outperform the OrganITE baseline. NN (34) achieves 26.5 additional predicted life-years gained per month compared with OrganITE, while SVM achieves an improvement of 21.8. OrganITE also performs slightly worse than the simple myopic matching.

PolicyMean PLYG
Omniscient3475.2
NN (34)3299.0
SVM3294.3
SMAC3287.8
Myopic3280.2
OrganITE3272.5

How long does optimizing one trajectory/run take? What is the end-to-end training time (including generating oracle labels)?

The efficiency of our approach is one of the key advantages compared to techniques like SMAC. Generating oracle labels takes on average 2.1 seconds per trajectory via Gurobi and the subsequent neural network training using those expert labels takes 3.5 minutes on average, even with dozens of features. In contrast, SMAC takes hours to optimize the weights for just the four-feature potential functions.

With only one day remaining in the discussion period, we wanted to follow up and ask whether the reviewer has any remaining concerns or questions regarding our responses.

We have provided additional experiments and clarifications addressing the points raised in the review, and would welcome the opportunity to discuss anything that remains unresolved before the discussion period closes.

Reviewer_EcPcOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper tackles the problem of optimally allocating donor organs to patients as the donor organs arrive over time. This is a challenging real-world problem and the current policies for transplant organ allocation in the US use very coarse and static criteria to make these allocation decisions. The problem however is far more complex since there is a tradeoff between allocating arriving donor organs to achieve an immediate benefit and preserving matching opportunities for future organs.

The proposed method learns "potential" functions that adjust predicted survival benefits, and trains linear models and neural networks to imitate an optimal hindsight algorithm that knows all future donor arrivals and patient trajectories. To generate the training data, the authors generate synthetic scenarios by resampling historical patients and donors and changing their arrival times. The method is evaluated in simulations using the US transplant registry data, and the authors show that this method substantially outperforms the current implemented policy and other proposed risk scoring baselines, while improving the predicted PLYG over a greedy policy.

Critical strengths

First of all, this paper is excellently written and I really enjoyed reading it. It is so refreshing to read a human-authored paper that is very clear and where the sections follow logically from one idea to the other. Additionally, the authors propose a very simple and elegant solution to the very important problem of organ allocation, which as the authors point out, is currently being approached through various computational methods and policymakers are in the process of adopting some of the solutions proposed by researchers in our community. This problem has been previously approached through the lens of domain adaptation, counterfactual inference, and optimal queuing policies, but here the authors introduce a new conceptual framework by using the idea of potentials to define the utility function and an imitation learning algorithm to copy the decision patterns of an oracle policy. I think this is a technically novel idea that is simple enough and does not involve unreasonable assumption so it could constitute a viable practical approach to organ transplant management.

Critical weaknesses

(1) There seems to be a chicken-and-egg problem with the proposed method and evaluations, because to evaluate a new policy we can only generate the outcomes for counterfactual decisions through synthetic simulations. This problem seems to be possibly invalidating for the proposed method because the authors also need to generate counterfactual scenarios for training the imitation learning policy as described in Section 4. It seems that the synthetic data generator in Section 4.2 only simulates limited perturbations in the factual data (e.g. only changing the relative timing and concurrent availability of patient-donor pairs as described in line 353). Additionally, the paper scores each transplant using the difference between two Cox model predictions for the survival with that organ and survival without transplantation, and it seems that these predictions are used for both the optimization objective and the reported evaluation. I am concerned that the choices of the synthetic data generators may overestimate the performance of the proposed method if the same synthetic models are used for the omniscient algorithm as well as the evaluation. Consequently, I don't think the authors provide a convincing evidence that their method can learn truly counterfactual policies.

(2) Isn't it possible that imitating a clairvoyant's choices still learns the wrong online policy even with unlimited data? There has been some recent interesting work on this issue (e.g. see "Hindsight Learning for MDPs with Exogenous Inputs" in ICML 2023), which explicitly analyzes the failure of cloning hindsight-optimal actions. A discussion of this issue seems absent from the paper's related work or conclusion.

The way this issue could manifest here is that the training losses reward selecting the patient chosen by the hindsight/oracle policy, but the most frequent hindsight optimal action may not maximize the expected utility, i.e. the action a clairvoyant chooses most often can have very poor expected value when you do not know the future. To give one example, assume that the first organ to arrive gives either patient A or B the same utility of value 1. Then the second order can give patient A utility of 1 with probability 0.51, otherwise it can serve patient B with utility 100. Then a clairvoyant policy will allocate the first organ to B in 51% of all simulations and a perfect classifier will choose organ B. However, choosing patient B first will give an expected total utility of 1.51 whereas choosing A will give an expected utility of 50. Thus, even with a perfect supervised learning you can still produce a very poor allocation policy and the immediate utility term will not really prevent this counterexample because both initial utilities are equal.

Questions for authors

Please comment on the two weaknesses I mentioned above.

2 comments
AuthorsOpenReview ↗

Author response (1/2)

We thank reviewer EcPc for their thoughtful assessment and particularly appreciate their recognition of the methodological novelty, practical importance, and presentation quality of our work. We address both concerns below and hope these clarifications reinforce the reviewer's positive assessment.

(1) There seems to be a chicken-and-egg problem with the proposed method and evaluations, because to evaluate a new policy we can only generate the outcomes for counterfactual decisions through synthetic simulations. This problem seems to be possibly invalidating for the proposed method because the authors also need to generate counterfactual scenarios for training the imitation learning policy as described in Section 4. It seems that the synthetic data generator in Section 4.2 only simulates limited perturbations in the factual data (e.g. only changing the relative timing and concurrent availability of patient-donor pairs as described in line 353). Additionally, the paper scores each transplant using the difference between two Cox model predictions for the survival with that organ and survival without transplantation, and it seems that these predictions are used for both the optimization objective and the reported evaluation. I am concerned that the choices of the synthetic data generators may overestimate the performance of the proposed method if the same synthetic models are used for the omniscient algorithm as well as the evaluation. Consequently, I don't think the authors provide a convincing evidence that their method can learn truly counterfactual policies.

We agree that any retrospective evaluation necessarily depends on the utility model used to evaluate policies. This is an inherent limitation shared by essentially all computational work on organ allocation, including evaluations of continuous distribution and other proposed allocation systems. So we would find it unfair if our paper were singled out for rejection on this basis. Our simulator follows the same paradigm by relying on standard survival prediction models to estimate edge utilities. Our simulator is grounded in the historical UNOS registry and uses standard survival models to estimate edge utilities in the same way that UNOS/OPTN/SRTR evaluate policies in simulation. We also want to re-emphasize that we are evaluating on different data than what we use for training.

Importantly, our methodological contribution is orthogonal to the particular utility function. Our framework does not require population life-years gained (PLYG) to be the optimization objective. We adopt this objective since it is the standard optimization objective in nearly all prior work on organ allocation, and the UK deployed a policy that optimizes this metric. If policymakers instead wished to optimize a multi-objective criterion incorporating urgency, fairness, pediatric priority, sensitization, or geographic equity, the omniscient oracle can simply optimize that alternative objective and the resulting potentials would imitate those decisions (which we show in Section J).

Given the reviewer’s comment about semi-synthetic trajectories, we want to re-emphasize that the semi-synthetic trajectories are used solely for training the policies and are not used for evaluation. The empirical results always use the real historical trajectory of events that occurred in 2019, which were not used for training. Appendix K.2 includes the ablation that quantifies the improvement from training with semi-synthetic data.

AuthorsOpenReview ↗

Author response (2/2)

(2) Isn't it possible that imitating a clairvoyant's choices still learns the wrong online policy even with unlimited data? There has been some recent interesting work on this issue (e.g. see "Hindsight Learning for MDPs with Exogenous Inputs" in ICML 2023), which explicitly analyzes the failure of cloning hindsight-optimal actions. A discussion of this issue seems absent from the paper's related work or conclusion. The way this issue could manifest here is that the training losses reward selecting the patient chosen by the hindsight/oracle policy, but the most frequent hindsight optimal action may not maximize the expected utility, i.e. the action a clairvoyant chooses most often can have very poor expected value when you do not know the future. To give one example, assume that the first organ to arrive gives either patient A or B the same utility of value 1. Then the second order can give patient A utility of 1 with probability 0.51, otherwise it can serve patient B with utility 100. Then a clairvoyant policy will allocate the first organ to B in 51% of all simulations and a perfect classifier will choose organ B. However, choosing patient B first will give an expected total utility of 1.51 whereas choosing A will give an expected utility of 50. Thus, even with a perfect supervised learning you can still produce a very poor allocation policy and the immediate utility term will not really prevent this counterexample because both initial utilities are equal.

We thank the reviewer for this insight and the reference to that interesting paper. We agree, and are aware, that imitating hindsight-optimal decisions is not guaranteed to recover the optimal online policy. However, we believe it is important to distinguish such worst-case constructions from the structured matching environments that motivate our approach. At the end of the day, it is an application-specific empirical question as to whether mimicking the hindsight-optimal policy generates strong online policies. Our paper shows that for this very important application it does.

In heart transplantation, blood-type compatibility, geographic disparities in organ supply, and the evolving composition of the waitlist create recurring patterns in the long-term opportunity costs of matching patients. These are precisely the patterns that our potential functions aim to learn from the omniscient oracle. Rather than attempting to reproduce individual clairvoyant decisions perfectly, our framework uses supervision across many trajectories to learn how these structural features should influence online decisions. Our semi-synthetic data augmentation further supports this by varying the timing and availability of patients and donors while preserving their clinical characteristics. While augmentation does not eliminate the theoretical possibility identified by the reviewer, it mitigates reliance on any particular realization of future arrivals.

Our empirical results provide the evidence that hindsight imitation is effective in this setting. This suggests that the structural information learned from hindsight optimal allocations is useful for improving online decisions, despite the absence of a general guarantee. It is also worth emphasizing that myopic matching is itself susceptible to suboptimal early allocations. Our approach is designed to overcome these limitations of greedy matching.

We will add a discussion of this topic and the connection to the provided paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.