acceptodds
Under review as a conference paper at ICLR 2027

Evidence Contracts for Domain-Mapped Prediction

Abstract

A predictive feature needs not be admissible evidence for the interpretation attached to an output. We formalize source-qualified evidence contracts that constrain a semantic model while preserving an unrestricted predictor. Positive, negative, unknown and conflicting observations compile to permitted concept completions. We distinguish expected-band invariance from probability certification and prove a conditional error bound under concept-dependent reporting shift, with a counterexample showing why ordinary calibration is insufficient. In a five-seed replication, probability certificates achieve maximum mean selective error of 1.66% across seven correct-contract regimes, compared with 4.19% for a same-contract MLP and 3.08% for a typed MLP at matched coverage; source-like error is slightly worse. Exhaustive checks show that fitted heads do not satisfy the theorem's pointwise assumption, so the empirical result is not promoted to a nominal guarantee. A text-to-contract diagnostic exposes extraction errors, and a semi-synthetic study on 5,070 real questionnaire records establishes an exact arithmetic baseline. Together, these results separate admissible evidence, assumptions for reporting robustness and measured performance.

Primary area:
interpretability and explainable AI
Keywords:
Evidence contracts, concept bottleneck models, reporting shift, selective classification, interpretability, ordinal classification, certifiable robustness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_tJTuOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes evidence contracts to distinguish information used for prediction from evidence that can support a semantic claim. It uses four-state observations to construct admissible concept bounds and provides conditional guarantees under reporting shift. The experiments include synthetic reporting-shift settings, text diagnostics, and real questionnaire data.

Critical strengths

I think the distinction between predictive information and admissible evidence is worth studying. The paper also provides theoretical analysis and several experimental comparisons. I appreciate that the authors explicitly check their assumptions and acknowledge where the guarantees do not hold for the learned models.

Critical weaknesses
  1. My main concern is the gap between the theoretical guarantee and the actual experiments. Proposition 4 requires a pointwise condition, but Section 5.2 shows that all five Round-3 learned heads violate it on 103–141 out of 4,096 concept configurations at α = 0.1. Eq. (7) is a cross-entropy training objective and does not enforce this condition. I understand that the authors acknowledge this limitation, but I still don't see how the nominal probability guarantee can be applied to the learned models. The reported selective errors are empirical results, and the slack-based bound is considerably weaker.

  2. I am also not convinced by the novelty. The Introduction contains no citations, and although Section 2 discusses related work, I still find it difficult to identify the main technical advance over existing concept-based models, monotone prediction, and selective classification. The paper also builds on the original dual-map system, but I don't think the difference is explained clearly enough.

  3. The presentation could be clearer. Figure 1 does not explicitly show the semantic head or the certification step. It also makes the envelope look like an output of completion, whereas Eq. (5) computes its endpoints directly using the semantic head. I found this confusing when trying to understand how the different parts of the method fit together.

Questions for authors
  1. How can the pointwise assumption in Proposition 4 be enforced or verified for learned heads? Without this, what guarantee can actually be provided in practice?

  2. What is the main technical contribution beyond existing methods and the original dual-map system? I would appreciate a clearer explanation of this distinction.

  3. Could the authors clarify the relationship between completion, the semantic head, and the output envelope, particularly in Figure 1?

Reviewer_5934OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes evidence contracts that separate admissible evidence for semantic predictions from unrestricted predictive features. It combines four-state evidence representations, bounded concept completion, a monotone semantic head, and selective prediction under reporting shift. Evaluation includes synthetic experiments, small text diagnostics, and a semi-synthetic NHANES study.

Critical strengths

The distinction between predictive features and admissible evidence is useful and clearly motivated. I also appreciate that the authors report negative results and explicitly state when their assumptions do not hold.

Critical weaknesses

My main concern is limited novelty. Much of the method combines existing ideas such as four-valued representations, monotonic models, interval reasoning, and selective prediction.

More importantly, the learned models violate the pointwise assumption required for the main probability guarantee. Therefore, the strongest theoretical claim does not actually apply to the fitted models. The empirical evidence is also mostly synthetic, while the NHANES experiment is essentially an interval-arithmetic sanity check rather than a realistic learned prediction task.

Questions for authors
  • Can the authors demonstrate a learned model that actually satisfies the pointwise assumption required by Proposition 4? Otherwise, why should the empirical predictions be described as “certificates”?

  • Can the authors compare against a flexible monotone model using exactly the same contract inputs, to separate the contribution of evidence contracts from ordinary monotonicity?

Reviewer_c6vFOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses an important gap between predictive accuracy and whether the evidence used to support a prediction is actually appropriate. This is relevant and useful in high-stakes applications such as mental health. I would classify this work as interpretable machine learning with evidence constraints, at the intersection of XAI, trustworthy AI, and selective prediction. They evaluate their framework mainly on synthetic data, comparing it with MLP baselines and use real-world depression questionnaire data to assess how their approach handles missing observations.

Critical strengths
  • An interesting and important problem by distinguishing between features that help a model make accurate predictions and evidence that can actually justify those predictions.
  • mathematical properties concerning prediction bounds, noninterference, and robustness under reporting shifts
  • Promising results with MLP and under distribution shift and considering missing observations
Critical weaknesses
  • Although the idea is interesting, it is not entirely clear what the proposed framework adds beyond existing concept bottleneck and concept-residual models that have already been explored.
  • Most of the experiments are on synthetic data
  • The experiments mainly use relatively simple MLP architectures. This limits the evidence for its broader applicability.
  • The writing could be a bit smoother and easier to follow overall. A few sections feel a little disconnected, without smooth transitions between ideas.
Questions for authors

Can you demonstrate the method with an independently pre-trained black-box model? Can you run a few extra experiments on more realistic datasets to see how it compares with related work?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.