acceptodds
Under review as a conference paper at ICLR 2027

Risk Controlled Safe Stopping for Sequential Data Acquisition

Abstract

In clinical diagnosis and tomographic imaging, data arrive one measurement at a time and each measurement costs time, money, or patient burden. Deciding *when* enough has been acquired is as consequential as deciding what to acquire, yet existing acquisition policies and heuristic stopping rules offer no guarantee on the risk of the decision made when acquisition halts. We introduce RCS, a post-hoc, distribution-free procedure that selects a stopping rule so that a user-specified risk, such as the false negative or false positive rate, is controlled among the samples on which the system commits to a decision. RCS treats each candidate rule as a trajectory-level selective predictor and certifies it with the Learn-Then-Test framework, giving a finite-sample guarantee on the risk *conditional on stopping*. Importantly, RCS can be applied over any acquisition pipeline without retraining, whether it is fixed, random, or learned. Because the controlled risk is absolute rather than relative to full acquisition, no rule is safe by construction; when none meets the target, RCS certifies that fact and defers to full acquisition instead of returning an unsafe rule. On a synthetic CIFAR-10 benchmark, on *FastMRI* knee k-space acquisition, and on an internal prostate *mri* cohort, RCS holds the target risk at every acquisition rate while stopping a substantial fraction of samples early, and it certifies a learned reinforcement-learning acquisition policy as a post-hoc wrapper.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
AI for health, decision making

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_iBsAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies safe stopping in sequential data acquisition. Measurements arrive progressively, and the system must decide when to stop acquiring data and commit to a prediction. The paper argues that existing acquisition policies and heuristic stopping rules generally optimize average accuracy, cost, or reward, without controlling the risk among the samples on which the system actually stops.

The authors propose Risk-Controlled Stopping (RCS), a post-hoc, distribution-free procedure based on the Learn-Then-Test framework. Each candidate stopping rule is treated as a trajectory-level selective predictor. RCS evaluates the risk only on samples for which the rule stops, applies multiple-testing correction with Holm’s step-down procedure, and selects a certified rule according to a user-specified utility. The method can control overall misclassification risk or class-conditional FNR/FPR, and can be applied to fixed, random, or learned acquisition policies without retraining.

The paper evaluates RCS on sequential CIFAR-10 patch acquisition, FAST MRI knee k-space acquisition, an internal prostate MRI cohort, and a learned JAFA acquisition policy.

Critical strengths
  • The paper identifies an important distinction between controlling marginal risk over the full population and controlling risk among the samples on which a system actually commits to an early decision.

  • RCS is a model-agnostic post-hoc wrapper that can certify fixed or learned acquisition policies without retraining and can control class-conditional FNR/FPR.

Critical weaknesses
  1. Limited positioning and comparison with the closest related work.
    The paper’s main distinction from prior work on risk-controlled early stopping and sequential prediction is not fully established. In particular, Ringel et al. (2024), Gauthier et al. (2025), and Jazbec et al. (2024) are discussed but not systematically compared, and the experiments do not include the most closely related risk-control baselines.

  2. Theoretical scope and practical assumptions are insufficiently clarified.
    The main guarantee is a direct application of the Learn-Then-Test framework rather than a new theoretical result. Its validity depends on the candidate policy family being fixed before calibration and on calibration and test trajectories following the same induced distribution. The relationship between the theorem and the fallback behavior is also unclear: when no policy is certified, full acquisition is used, but full acquisition itself is not shown to satisfy the target conditional risk.

  3. Insufficient empirical validation of the statistical guarantee.
    The experiments show empirical risk curves, but do not directly evaluate the probability of successful certification associated with the 1−δ1-\delta guarantee. The effects of calibration-set size, δ\delta, the number and spacing of candidate policies, and the number of stopped samples are not systematically studied.

  4. Limited analysis of practical behavior and design choices.
    The evaluation covers several datasets and acquisition settings, but the experimental analysis remains largely descriptive. It does not sufficiently examine empty certified sets, low-stopping-rate regimes, distribution shift, adaptive-policy trajectory mismatch, or the contributions of the main statistical design choices such as Holm correction, p-value construction, policy-family parameterization, and fallback strategy.

Questions for authors

Please see Weaknesses.

4 comments

We thank the reviewer for the thoughtful comments on novelty, theoretical scope, and empirical validation.

Positioning relative to prior work. We agree that the finite-sample guarantee follows directly from LTT and do not claim a new concentration inequality. Our contribution is formulating sequential acquisition stopping as a trajectory-level selective prediction problem with absolute ground-truth risk conditional on stopping. Unlike Ringel et al., which control disagreement with the full-data prediction, RCS controls the prediction error itself. Unlike computational early-exit methods such as Jazbec et al., RCS addresses external measurement acquisition. This distinction matters because absolute risk admits no automatically safe full-acquisition configuration and need not be monotone across stopping policies. The paper also situates Gauthier et al. in the broader sequential risk-control literature.

Theoretical scope and fallback. We agree that validity requires the candidate policy family to be fixed independently of risk calibration and calibration/deployment trajectories to follow the same induced distribution. An empty certified set indicates that no candidate could be certified from the available data, not that no safe policy exists. Full acquisition is an operational fallback rather than an automatically certified predictor. We have corrected these distinctions throughout the manuscript.

Calibration sensitivity and statistical power. We added systematic sensitivity experiments across all three datasets (Appendix A.4, Figures 9–11), varying significance level δ\delta, candidate-grid size ∣Λ∣|\Lambda|, and calibration-set size ncaln_{\mathrm{cal}}. The main finding is that certification is primarily limited by effective calibration sample size, particularly for rare positive classes:

  • CIFAR-10: Grids containing 21–501 candidates yield similar stopping behavior, whereas an 11-point grid certifies no policy. Calibration sets of 250–1,000 examples also yield no certificate; certification becomes possible at 2,000 examples.
  • FastMRI Knee: Under the reported slice-level calibration, subsets containing up to 93 ACL-positive slices yield no certificate, while the full set containing 155 positives permits certification. As clarified in our response to R1, within-patient slice dependence limits the formal interpretation of this result.
  • Prostate MRI: Certification fails with 41 or 84 PI-RADS-positive calibration examinations but succeeds from 168 positives onward, with stable selected thresholds.

These results characterize the statistical power required for certification, sensitivity to grid discretization, and behavior when no candidate can be certified. They demonstrate that an empty certified set can result from insufficient calibration evidence rather than the absence of safe policies.

Empirical validation of the guarantee. We agree that the sensitivity experiments do not directly estimate the repeated-calibration probability of incorrectly certifying an unsafe policy. However, the finite-sample guarantee is established analytically through valid policy-level hypothesis tests and Holm's family-wise error correction under the stated independence assumptions. Repeated-calibration trials would provide an additional empirical diagnostic rather than a prerequisite for the theoretical result. We have clarified this distinction and do not claim that the new experiments constitute direct empirical verification of the 1−δ1-\delta probability statement.

Practical design choices. Holm's procedure provides family-wise error control without requiring a monotone ordering of candidate risks, while exact binomial testing is tighter than general bounded-loss Hoeffding bounds for binary FNR/FPR. The new experiments systematically assess sensitivity to significance level, candidate-grid density, and calibration sample size. We acknowledge that these analyses do not independently compare alternative multiple-testing corrections, p-value constructions, or policy-family parameterizations. The latter remain interchangeable components of the RCS framework, with their choice influencing statistical power and achievable stopping efficiency.

Distribution shift and adaptive-policy mismatch. RCS requires calibration trajectories representative of the deployed acquisition process. Arbitrary distribution shift or changes to the adaptive acquisition backbone can invalidate its certificate. We now state these assumptions explicitly and identify representative recalibration or shift-aware procedures as potential approaches for deployment under distribution shift.

The rebuttal clarifies the theoretical scope and fallback behavior, and the reported sensitivity experiments address several concerns about calibration and statistical power. However, direct comparisons with the closest risk-control methods remain missing, and the acknowledged within-patient slice dependence limits the formal certification claims supported by the FastMRI experiments. These responses improve the paper but do not fully resolve the main concerns. I would maintain the rating of 2 (Weak rejection).

  1. As we state in our response to reviewer gWJT, we have added multiple calibration-based stopping baselines for all three datasets.

We added two calibration-based stopping baselines across all three datasets (Appendix A.3, Figures 6–8). At each sampling rate, Operating Point selects a threshold targeting empirical risk over all calibration examples, while Selective Inference targets empirical risk among the stopped samples. Both use the same nominal risk target as RCS, but at a per-sampling rate.

On FastMRI Knee, RCS stops 13% of slices at 5% sampling with zero observed ACL FNR and meniscus FNR at most 0.012. Selective Inference also satisfies the empirical risk target but stops no slices at 5% or 8%, while achieving greater stopping coverage at later stages. On prostate MRI, RCS stops 36–53% of examinations with observed FNR 0.026–0.064, whereas Operating Point (0.14–0.17) and Selective Inference (0.105–0.14) exceed the target of 0.1. These comparisons evaluate the empirical risk–efficiency trade-off under a common nominal calibration target; they do not establish universal efficiency superiority or exact matching of held-out FNR.

  1. Additionally, we have added comparisons to Ringel et al as well, see figures 12, 13, and 14 in the appendix. We observe that RCS controls the risk over the stopped samples while Ringel et al does not. Their method is constructed to control the risk relative to the model with the fully-sampled data as input, which may or may not control the risk at the desired level, as opposed to RCS which controls the risk at a user-specified level not relative to the model being used.

  2. On the CIFAR-10 and Prostate MR experiments, we do have exchangeability as the samples are independent and the results show that RCS controls the risk at a desired risk level, while the baselines do not. Moreover, in the FastMRI knee experiments, despite exchangeability potentially being violated, therefore a more challenging setting, the risk is controlled at the desired level.

Thank you for your valuable feedback, which has greatly improved our paper. As the discussion phase wraps up, we believe we have addressed your final comment. Please let us know if any further clarification is needed.

Reviewer_gWJTOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This work proposes a method called Risk-Controlled Safe Stopping (RCS) to determine when to halt data acquisition in sequential settings to ensure safety. It treats each candidate stopping rule as a trajectory-level selective predictor and calibrates it post-hoc using the Learn-Then-Test (LTT) framework. The method does not require retraining the original acquisition model and provides finite-sample conditional risk statistical guarantees under user-specified absolute risks.

Critical strengths

This method has clear clinical and theoretical value: it proves that merely controlling Marginal Risk is insufficient and successfully achieves conditional risk control on "stopped samples". Moreover, RCS demonstrates extremely high generalizability; as a post-hoc wrapper, it can be directly applied to any existing acquisition pipeline without retraining the model. On clinical MRI datasets, it successfully controlled the False Negative Rate (FNR) below the target level while effectively reducing computational and acquisition costs.

Critical weaknesses

The paper is lacking in baseline comparisons. The experiments mainly compared RCS with static score thresholds or methods based on simple heuristics, but lacked in-depth comparisons with the latest adaptive inference/early stopping methods that possess risk awareness or uncertainty estimation. Secondly, as a statistical method heavily reliant on validation/calibration set data, if a Distribution Shift occurs in the clinical deployment environment, the paper does not discuss whether the statistical guarantees of this condition can still be maintained.

Questions for authors

In addition to the above weaknesses, I also noticed the following issues:

  • Data distribution shifts frequently occur in real-world healthcare scenarios . In such cases, will the theoretical bounds of RCS become invalid? Is dynamic recalibration required for the new distribution?
  • As shown in Figure 5, when RCS is used as a wrapper for the adaptive strategy JAFA , it successfully guarantees safety, but also leads to a decrease in the early stopping rate. Can you explain in detail the specific mechanism behind this conservative behavior? Is it because the Q-value distribution output by JAFA is not smooth enough?
3 comments

We thank the reviewer for recognizing the clinical relevance and model-agnostic applicability of RCS.

Risk-aware baseline comparisons. We added two calibration-based stopping baselines across all three datasets (Appendix A.3, Figures 6–8). Operating Point selects per-checkpoint thresholds targeting empirical risk across all calibration samples, whereas Selective Inference targets empirical risk among selected stopped samples. Both use the same nominal risk level as RCS but do not provide its finite-sample certificate.

On CIFAR-10, Operating Point stops approximately 90% of samples but achieves only 0.68–0.74 stopped-sample accuracy, while Selective Inference ends near 0.84. RCS achieves approximately 0.905 accuracy while stopping 52%. On prostate MRI, both baselines exceed the target FNR of 0.1, whereas RCS maintains empirical FNR below the target at all evaluated checkpoints. On knee MRI, Selective Inference also meets the empirical risk target but does not stop at the earliest acquisition rates, unlike RCS.

We acknowledge that these comparisons do not exhaust the literature on risk-aware early stopping. Importantly, RCS is a post-hoc certification framework rather than a competing acquisition backbone: it can be applied to stopping scores from existing acquisition policies without retraining, as demonstrated with JAFA.

Distribution shift. The finite-sample guarantee requires calibration and deployment trajectories to follow the same induced distribution. Under arbitrary distribution shift, the guarantee need not hold. Recalibration on representative deployment data, or a suitable shift-aware risk-control procedure, would be required to recover statistical validity. We have explicitly stated this limitation in the revised manuscript.

JAFA+RCS conservativeness. JAFA selects actions using a learned Q-function that optimizes expected reward, trading prediction accuracy against acquisition cost without enforcing a hard conditional-risk constraint. RCS constructs a stopping score from the difference between the STOP Q-value and the best acquisition-action Q-value, then calibrates a threshold to certify the desired stopped-sample risk. This can require stronger evidence for stopping than JAFA's original decision rule, causing some early decisions to be deferred. Thus, the efficiency reduction reflects the additional risk constraint and finite-sample certification requirements, rather than necessarily arising from nonsmooth or poorly calibrated Q-values.

Thank you for your response. After considering it, I have decided to maintain my original score.

Please let us know if you have any additional questions or concerns.

Reviewer_HkqDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

This paper proposes Risk-Controlled Stopping (RCS), a method for early stopping in sequential data acquisition while controlling the risk of prediction errors.

Although conventional confidence-based stopping methods and reinforcement learning-based acquisition policies can optimize the trade-off between average prediction accuracy and measurement cost, they do not provide statistical guarantees on the prediction risk associated with early-stopped samples.

The authors formulate the stopping rule as a trajectory-level selective predictor and apply the existing Learn-Then-Test (LTT) framework to obtain the following finite-sample guarantee: PDcal(Rcond(λ^)≤α)≥1−δ.P_{\mathcal D_{\mathrm{cal}}} \left( R_{\mathrm{cond}}(\hat\lambda)\leq\alpha \right) \geq1-\delta.

The proposed method is evaluated experimentally on CIFAR-10, FastMRI Knee, and prostate MRI, as well as in combination with JAFA.

Critical strengths
  • [S1] Addresses a Practically Important Problem

Determining when to stop data acquisition in sequential medical imaging is an important problem from both clinical and economic perspectives. The focus on controlling the actual misdiagnosis rate, rather than relying solely on model confidence, is a valuable aspect of the proposed approach.

  • [S2] Appropriate Formulation Based on Conditional Risk

The paper's discussion of the following relationship is important P(Eτ)=P(Eτ∣S)P(S).P(E_\tau)=P(E_\tau\mid S)P(S). Even if the overall marginal error is controlled, the error rate among samples for which acquisition is stopped early can remain high. The authors clearly identify this issue and appropriately formulate the problem in terms of directly controlling the conditional risk among early-stopped samples.

  • [S3] Compatibility with Existing Acquisition Policies

RCS can be applied to existing prediction models and acquisition policies without requiring retraining.

  • [S4] Evaluation on Medical Imaging Data

Rather than limiting the evaluation to theoretical toy examples, the authors attempt to demonstrate the practical utility of the proposed method in applications where measurement costs are a significant concern, i.e., MRI k-space acquisition and diffusion-weighted imaging (DWI) acquisition processes.

Critical weaknesses
  • [W1] Limited Novelty of the Statistical Guarantee

Corollary 1, which provides the central theoretical guarantee of the paper, follows directly from combining the existing Learn-Then-Test (LTT) framework with the Holm correction. Formulating the stopping rule as a trajectory-level selective predictor is a valuable contribution; however, for a fixed stopping rule, defining the conditional loss over early-stopped samples and applying a bounded-loss concentration inequality together with multiple testing correction is straightforward.

  • [W2] The Term “Certified Infeasibility” Is Inaccurate

The paper claims to “certify infeasibility” when no stopping rule can be certified as safe. However, failure to reject the null hypothesis under the Holm procedure does not imply that a stopping rule is actually unsafe.

The authors should clearly distinguish between the following two statements:

  • No rule can be certified: No stopping rule could be certified as safe using the available calibration data and statistical testing procedure.

  • No safe rule exists: There is genuinely no stopping rule that satisfies the required risk constraint.

The latter claim is not established by the statistical tests presented in this paper.

  • [W3] Insufficient Comparison of the Trade-off Between Risk Control and Acquisition Efficiency

The main baselines are forced prediction, confidence threshold, and entropy threshold; however, most of these methods are not explicitly designed to control the false negative rate (FNR). Therefore, showing that RCS satisfies the risk constraint while uncalibrated heuristics fail to do so is insufficient to establish that RCS is more efficient.

Questions for authors
  • [Q1] Time-specific guarantees: Corollary 1 controls the risk of the final stopping set. Do the authors claim that the guarantee also holds simultaneously for the cumulative stopping sets at each acquisition step shown in Figures 2–4? If so, what testing procedure ensures this guarantee?

  • [Q2] Testing configuration: Please report the value of δ\delta, the size of the candidate grid ∣Λ∣|\Lambda|, and the number of positive stopping cases used for certification of each candidate in the experiments.

  • [Q3] Data independence: Was the calibration--test split of FastMRI Knee performed at the patient, volume, or slice level? How do the authors account for the effect of multiple slices from the same patient on the independence of calibration samples?

  • [Q4] Fallback: If no safe early-stopping candidate is certified, what guarantee is provided for the FNR of predictions after full acquisition? If no such guarantee exists, how should the claim of end-to-end safety be qualified?

  • [Q5] Baselines: Does RCS still achieve improved acquisition efficiency at comparable FNR levels when compared with risk-constrained baselines, such as calibrated confidence thresholds or fixed acquisition rates?

5 comments

We thank the reviewer for recognizing the importance of conditional risk control and its applicability to sequential acquisition.

W1. Limited novelty. We agree that our finite-sample guarantee follows from LTT and do not claim a new concentration inequality. Our contribution is formulating sequential acquisition stopping as trajectory-level selective prediction with absolute ground-truth risk conditional on stopping. Unlike agreement-based approaches, absolute risk admits no automatically safe configuration, and candidate risks need not be monotone in the stopping threshold. Consequently, certification must accommodate potentially disconnected or empty safe sets. RCS instantiates this formulation using policy-level testing with Holm's correction and an explicit fallback when no candidate is certified.

W2. Certified infeasibility. We agree. An empty certified set indicates insufficient statistical evidence to certify any candidate, not proof that no safe rule exists. We have corrected this terminology throughout the manuscript.

W3/Q5. Risk-aware baselines and acquisition efficiency. We added two calibration-based stopping baselines across all three datasets (Appendix A.3, Figures 6–8). At each checkpoint, Operating Point selects a threshold targeting empirical risk over all calibration examples, while Selective Inference targets empirical risk among selected examples. Both use the same nominal risk target as RCS, but neither provides the same finite-sample certificate.

On FastMRI Knee, RCS stops 13% of slices at 5% sampling with zero observed ACL FNR and meniscus FNR at most 0.012. Selective Inference also satisfies the empirical risk target but stops no slices at 5% or 8%, while achieving greater stopping coverage at later stages. On prostate MRI, RCS stops 36–53% of examinations with observed FNR 0.026–0.064, whereas Operating Point (0.14–0.17) and Selective Inference (0.105–0.14) exceed the target of 0.1. These comparisons evaluate the empirical risk–efficiency trade-off under a common nominal calibration target; they do not establish universal efficiency superiority or exact matching of held-out FNR.

Q1. Time-specific guarantees. Corollary 1 controls risk over the complete population of eventually stopped samples, not simultaneously over cumulative stopped subsets at each checkpoint. The intermediate curves are empirical diagnostics. We clarified this distinction throughout the manuscript, including the early CIFAR-10 checkpoints where observed accuracy is below 90%.

Q2. Calibration configuration. Appendix A.4 (Figures 9–11) now reports sensitivity to significance level δ\delta, candidate-grid size ∣Λ∣|\Lambda|, and calibration size ncaln_{\mathrm{cal}}. Default configurations (δ,∣Λ∣,ncal)(\delta,|\Lambda|,n_{\mathrm{cal}}) are (0.05,101,4800)(0.05,101,4800) for CIFAR-10, (0.1,15,6180)(0.1,15,6180) for knee, and (0.1,51,745)(0.1,51,745) for prostate. We also report class-positive counts and empty-certified-set behavior. The effective number of stopped positives depends on the candidate stopping policy and can be substantially smaller than the total number of positive calibration examples.

Q3. Data independence. FastMRI Knee was partitioned at the patient level, ensuring no patient overlap across training, calibration, and testing. Thus, held-out performance measures generalization to unseen patients. We acknowledge that knee prediction and calibration are slice-level and that within-patient slice correlation is not addressed by patient-disjoint splitting alone. Consequently, the formal i.i.d. slice-level certification assumption is not established for this experiment, although the reported held-out FNR and stopping fractions remain valid empirical measurements. The limited number of independent knee volumes, particularly for rare ACL pathology, makes volume-level classification and certification difficult with the available data; we leave this extension to larger cohorts. In contrast, prostate prediction and calibration operate at the patient-disjoint examination/volume level, where sufficient independent examinations are available.

Q4. Fallback. Full acquisition is an operational fallback rather than an automatically risk-certified predictor. When no early-stopping candidate is certified, RCS retains the existing full-acquisition pathway. We have removed claims of automatic end-to-end clinical noninferiority and clarified that the fallback's risk requires separate evaluation or certification.

Thank you very much for clarifying the scope of the guarantee and adding the calibrated baselines. My remaining concerns are the limited methodological novelty and that the i.i.d. guarantee is not established for the slice-level FastMRI Knee evaluation. I maintain my current score for now.

Thank you for response, as we acknowledge in the paper and the rebuttal above, the novelty is in the extension of LTT to the sequential acquisition setting and then demonstrating its use in clinically relevant tasks. To the best of our knowledge, this extension is novel and directly relevant to current deployment settings, particularly the MR experiments.

Besides FastMRI knee, the other two datasets do have i.i.d samples and for all three datasets in our experiments the risk is controlled at the desired level, for various choices of this level. We believe that the lack of exchangeability on the FastMRI knee slices is a limitation of the number of samples that are available for training a volume-level model. For the prostate MR experiment, where we do have a large number of volumetric samples for training and calibration, we show that a volumetric classifier can be trained and RCS can provide significant reductions in sampling time and maintain risk at the user-specified threshold.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revisions to manuscript

We would like to thank all reviewers for engaging with us during the rebuttal process to help us further shape the paper. In this note we wanted to provide a brief summary of the changes to the manuscript to address the concerns raised by the reviewers.

  1. Comparisons to further baselines that control the target risk: We have added two baselines to all our datasets. Briefly, these baselines use the same risk control procedures to control the target risk, however the risk is controlled at each sampling rate, not the sequence level. We observe that none of these baselines control the risk at the desired level.

  2. Comparisons to Ringel et al: Ringel et al control the risk relative to the predictions made by the model with fully-sampled data. Even though the fundamental difference in the target risk being controlled makes a direct comparison unfair, we added experiments to the appendix and demonstrate that it has a higher risk, as the model with fully-sampled data has a high risk.

  3. Robustness to hyperparameter configurations: We have added sensitivity analysis to all hyperparameters to the appendix. We observe that across all settings, RCS either controls the risk at the desired rate or fails to certify a policy. The procedure is robust across various configurations, with highest sensitivity towards the effective calibration set size.

  4. Independence assumption violations on data: On one of the three datasets we consider, FastMRI knee, the independence assumption is violated, rather we have weak exchangeability. Despite this violation, we still observe that the risk is controlled at the desired level. Moreover, on the other two datasets, where independence holds, the risk is also controlled at the desired level. The FastMRI knee dataset does not contain enough data to train and calibrate, however, on the prostate MR dataset, where enough data is available, we are able to train a model and apply the calibration procedure.

To the best of our knowledge, our work is the first to propose sequential acquisition control and demonstrate its applicability to clinical problems. RCS is a distribution-free post-hoc procedure that can control risk for any acqusition policy and classification model, and as our experiments demonstrate, RCS can be applied to practical clinically relevant tasks and obtain significant savings.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.