- Rating
- 2
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
I found this paper extremely difficult to parse and understand, which may be due to my lack of prior knowledge in this area, or may be due to the writing quality, or both. The following is what I understood the problem setting for this paper to be:
- You have a medical AI assistant that produces some output or decision f(X), in response to some question/prompt/scenario X. It also produces a risk score s(X), which is similar but not indentical to its confidence in its answer. This process may be done in 2 stages, by first having a model or multiple models produce candidate answers f(X)_1, ...., f(X)_n, and then having another model pick one and output a risk score.
- We are told that deploying this assistant requires a threshold t that says "if s(X) > t" then do not release this assistant's response, route it to a human.
- Where to set this threshold depends on the error rate alpha that you are willing to accept. The deployment should contain a mapping that says for a given alpha, here's where you set t.
- This mapping should be certified, which is to say that if you follow its given t for a choice of alpha, with some probability delta the actual error rate should be less than alpha.
- Ultimately, this means there is a policy T(alpha, delta) -> t, which is a mapping from a risk tolerance alpha and certification confidence delta, to a threshold t. This says that if you chose your threshold to be t, then with probability > 1 - delta the error rate of the system will be less than alpha.
- We can rollout policy T on real data and systems, and evaluate the true error rate, and how often it witholds an assistants correct answer and so on.
- This paper is about how to design the policy T and produce statistically sound certificates (proofs) that it has the desired property of "if you chose your threshold to be t, then with probability > 1 - delta the error rate of the system will be less than alpha. "
Critical strengths
- The paper tackles a worthwhile aim, the safe deployment of medical AI
- The paper evaluates over 3 different datasets and multiple cross-validation folds within datasets.
- The authors compare to a few existing methods, but hard to assertain to what extent they are fair comparisons
- For their chosen experimental setup, their method vastly outperforms existing methods
Critical weaknesses
- The paper is extremely hard to follow
- The problem setting is very complicated, and the paper does not give good evidence that it is a real setup (ie laws require medical AI to have a risk score classifier and a certificate etc) or a contrived one
- Further analysis and weaknesses require a greater understanding of the paper and its method than i am able to give
Questions for authors
I believe this paper would benefit massively from well structured and seperated sections that tackle
- problem setup (a model produces answers, a classifier gives risk score, a rule decides when to release etc etc)
- background (what prior work is needed to understand this paper, motivate and validate the problem statement etc)
- related work (existing work, if it fits into your problem setup then should be described and compared in tables, if doesn't fit in problem setup, why not and why that doesn't make your setup wrong)
If the authors are able to restructure and improve the clarity of their writing, then I am happy to review the method etc in more depth and revise my scores.