acceptodds
Under review as a conference paper at ICLR 2027

When Can Expert Corrections Be Trusted? Evidence, Limits, and Risk-Controlled Decisions Across Environments

Abstract

When a strong reference model and an automated expert disagree, the same correction rule can help on one patient's slide and harm on another. We ask when such corrections can be trusted across environments, and propose a unified framework separating the construction of corrective evidence from the decision to act on it. On the evidence side, we introduce CARE-Cell, a cell-classification architecture combining frozen tissue representations, local nuclear features, and target-conditioned context to build both a strong reference predictor and complementary corrective candidates. On the decision side, a risk-controlled layer determines when to revise, retain, or defer. Theory explains why both parts are necessary: we construct observationally indistinguishable environments whose optimal correction decisions are opposite, yielding a worst-case regret lower bound for any router measurable with respect to reference outputs, expert outputs, and confidence. The decision layer accordingly offers two complementary guarantees. With target-environment labels, an independent randomized audit certifies positive aggregate utility on the remaining queries with a finite-sample error guarantee, retaining the reference predictions when certification fails. Without such labels, labeled calibration environments set an accept/defer threshold that, under exchangeability, keeps a new environment's retained-error fraction below a prescribed limit with a marginal probability guarantee. We evaluate on a consecutive dermatopathology cohort collected without case selection based on image appearance or model performance, together with Lizard, PanNuke, Camelyon17-WILDS, and iWildCam-WILDS. CARE-Cell improves inflammation F1 over a UNI2-based cell classifier by 4.59, 8.35, and 1.84 percentage points on the three cell-classification datasets; under identical splits and GT instances, it improves over a CellViT++-based classification baseline by 11.95 points on Lizard and by a comparable margin on the private cohort. Substituting Virchow2 for UNI2 yields comparable gains, with UNI2 strongest overall. Without altering full-cohort predictions, the decision layer retains 92.15% of Lizard and 86.26% of PanNuke samples for automatic prediction, giving inflammation F1 of 0.9714 and 0.8765 on the accepted subsets, while hospital-shift stress tests illustrate failures outside the calibration assumptions. Together these separate three questions often conflated in expert augmentation: whether corrective evidence exists, whether its utility is identifiable, and which deployment decisions the available information can justify.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
distribution shift, prediction revision, identifiability, selective prediction, conformal prediction, risk control, computational pathology

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_jwgCOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper proposes a method for automated revision support framework in histology slides for cell classification with post-hoc conformal prediction. Several baselines and datasets (mostly pathology with WildCam) and show moderate improvement over methods such as CytoFormer.

Critical strengths

The empirical results show some improvement on the task over multiple datasets. Although the experimental setup and evaluation metrics need more exposition instead of notation and large multi panel figures.

Critical weaknesses
  • The paper mixes previous existing methods e.g. conformal prediction with the revision setting and is it unclear where the novel insight or contribution lie. Otherwise the paper is straightforward application of known/existing work.
  • There is no validation of clinical utility. No reader study or reference to how revision helps in practice. It is unclear how such a framework could be integrated into clinical workflow or deployment.
  • In general, much of the main text notation is only used once and can be simplified for readability. There are presentation issues with undefined notation and missing implementation and experimental setup details that make the paper hard to follow.
Questions for authors
  • What general insights beyond this application?
  • What is the scope of the novel contribution? what has been done before? Missing related works. Needs to be more demarcation between what is the contribution versus what is reusing existing or previous results. A explicit contribution list would help.
  • Writing is obtuse and dense and littered with unnecessary notation that makes it difficult to understand the proposed method and implementation details. An algorithm box might help. Or at least add the same notation to Fig 2 i.e. z, W, U, beta. How is a and r formally defined? What is the bank and action space? What does audit mean here? Is b and h the same as B and H? Otherwise h is overloaded for harms and candidate proposal function. Subscript e and i are never defined. Is subscript e it same as superscript e?
  • In Eq. 5, What is a support action? Where does 128 come from?
  • How does the finite-sample assumption affect the audit procedure?
  • What is clinical significance of the revision? is it practically useful? Is there any real user studies showing improvements from this automated/reference setup?
  • It is unclear what the conformalizing procedure is over. Is f_i in Eq. 8 the same as Eq. 1? How is calibration score s defined in your implmentation. The threshold t is not formally defined. n is overloaded. How is alpha chosen in practice?
  • What is the distinction between "fit" and "held"? So for Fig. 3d, how should interpret the AUC plot?
  • Undefined jargon like "Private rich-router" and "Serial information-flow contract". How is information beyond the standard binary predictions analysis?
  • Missing standard deviation and significance testing against baselines like CytoFormer.
  • What is out of distribution (e.g. hospitals, demographics) performance of your method? Can you do cross dataset evaluation such as train on WILDS, test on Lizard?
  • Seems like typo in Table 2 for the PanNuke row?
Reviewer_zpxbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

his paper studies when corrections from expert models can be trusted across different environments. The authors propose CARE-Cell, which combines tissue and nuclear representations with contextual information to construct corrective predictions. They further introduce two decision procedures: an independent audit using target labels to authorize revisions and an environment-level calibration method for selective acceptance. The theoretical analysis addresses the limits of identifying correction utility and provides statistical guarantees under specific assumptions. Experiments are conducted on pathology datasets and two WILDS benchmarks.

Critical strengths

The paper addresses an important problem in expert-assisted prediction. Expert corrections may help in some environments while causing harm in others, and distinguishing corrective evidence from the decision to apply it is a useful perspective.

The paper combines theoretical analysis with experiments across several datasets. The authors also discuss the assumptions of their statistical guarantees and report negative results, which helps clarify the scope of their findings.

Critical weaknesses

The paper presents CARE-Cell, audited revision, and selective acceptance as a unified framework, but evaluates them separately. Without an integrated evaluation, the benefit of combining these components remains unclear.

The theoretical novelty is limited. T1, T2, and T3 largely build on established results, and the experiments do not directly connect the observed correction failures to the theoretical findings.

The theoretical guarantees do not fully match the paper's objectives. The audit guarantees aggregate additive utility rather than F1 improvement, while selective acceptance controls retained errors rather than the error rate among accepted predictions. The required exchangeability assumption also limits applicability under distribution shifts.

The ablation results do not sufficiently justify CARE-Cell's complexity. Simple context aggregation performs similarly to target-conditioned context, and removing geometric information even improves performance on Lizard.

The experimental evidence is insufficient to establish generalizability and practical benefits. The private evaluation uses only six slides, public datasets show minimal gains from audited revision, and comparisons with simpler methods under matched labeling budgets are missing.

Questions for authors

What is the main methodological contribution beyond combining existing approaches to evidence construction, utility auditing, and selective prediction? Could the authors clarify how these components form a unified framework?

How do the theoretical results justify the proposed decisions under distribution shift? In particular, how should the guarantees be interpreted when exchangeability does not hold, and how do they relate to the actual reliability of accepted predictions?

Can the authors provide stronger evidence for the necessity of CARE-Cell's individual components and compare audited revision against simpler decision strategies under the same labeling budget?

Reviewer_7kAjOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors present a corrective, multi-branch/feature model for cell-classification that makes a genuine effort to solve issues around deployment of assistive models for experts. The model uses a risk-controlled layer to assist in whether to revise, retain, or defer decisions in dermatopathology.

Critical strengths
  • The paper is presented with promising results and a sophisticated model for dermatopathology classification that solves an important problem in deploying such models in the real-world.
  • The analysis and modelling are well presented and seem to be clear in their benefit
Critical weaknesses
  • A lack of related works section is concerning as it is difficult to ascertain where previous works have attempted to build similar decision systems and what other works have occurred in dermatopathology. The model itself seems to incorporate multi-modal information in terms of pathological measures and various other inputs, which one would expect to have been done in similar fields.
  • The paper can be difficult to read, see for example second paragraph of introduction. Sentences like “Theory specifies the information each stage needs. T1 constructs observationally indistinguishable environments with opposite optimal corrections.” (pg. 2 ln 82) make it difficult and seem related to AI editing not being cohesively crafted. What theory are we referring to?
  • The use of T1, T2 etc. is undefined (used pg. 2 ln 82, defined much later) and appears to be referring to stages constructed by an AI model during development and have been integrated directly into the manuscript that does not translate well with the flow and understanding of the paper and undermines the quality of the work. There are many short sentences that alter the flow making judging methods and thus novelty difficult. For example, pg. 7, ln 343, this section is full of short sentences that seems to be straight from the output of an AI model developing the methods of the model and reporting outcomes.
  • Although proofs for T1, T2 etc. are provided, these seem plausible but I’m not sufficiently versed in the area to make a concrete assessment of them.
  • There is a lot going on in Figure 2 that is hard to follow. A better more informative caption would go a long way to help the reader. Similar issues occur for other figures that could use better captions. For example, the second paragraph of the introduction would make a good caption for figure 1.
  • Its good to see the comparisons in Table 1. Would be nice to see the citations with each of the methods presented in each row.
  • There is no limitations and future work to speak of, which is important to understand where the work is placed in the context of the literature and what remains to be done.
Questions for authors
  • There is reference to “Theory explains why useful evidence alone does not authorize action” (abstract) and “Theory specifies the information each stage needs.” In Introduction. What theory is being referred to in each? Feels if though some information is missing. Apologies if I missed something.
  • Would a theory like Dempster–Shafer Theory (DST) be useful here in grounding the proofs on existing theories to make things simpler for future work? Again, apologies if I misunderstood.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.