acceptodds
Withdrawn from ICLR 2027

Evidence Contracts: Verifiable Control of Evidence Reliance in Multimodal Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multimodal models can produce correct predictions while relying on evidence they are meant to ignore, or failing to rely on evidence they are asked to use. Existing evidence-control methods can alter routing patterns or reliance scores, but these changes do not establish the stronger property of behavioral compliance: whether a requested evidence-use relationship holds under admissible intervention mechanisms excluded from adaptation and model selection. We introduce Evidence Contracts, a specify–enforce–verify framework that compiles natural-language evidence-use instructions into explicit, behaviorally testable requirements. Evidence Contracts support compositional IGNORE, REQUIRE, PREFER, and PRESERVE clauses, respectively specifying invariance to designated evidence, task-consistent dependence on required evidence, relative dependence between evidence sources, and preservation of task performance. We ground each clause in multimodal inputs, enforce the resulting contract through contract-constrained adaptation, and verify compliance using validated intervention families held out from adaptation, model selection, and calibration. Across seven multimodal benchmarks, Evidence Contracts achieve 68.4% held-out contract satisfaction rate (CSR) on the common applicable benchmark–backbone settings, compared with 36.5% for GUIDE, while retaining 98.6% of original task utility. Compliance also transfers to unseen contract targets, operator–target pairings, and clause compositions, reaching 63.9% average CSR. These results show that evidence use can be specified, enforced, and independently verified as a behavioral commitment within declared, validated intervention distributions, rather than treated only as a steering preference.

Primary area:
interpretability and explainable AI
Keywords:
Multimodal Models, Evidence Reliance, Controllable Multimodal Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fEdbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces a problem of behavioral compliance and propose a framework by designing four types of behavioral constraints to optimize violations. They perform extensive experiments to show the effectiveness of their method.

Critical strengths
  1. They clearly emphasize the difference between the predition correctness and behavioral compliance in the main text, which provides a clear setting.

  2. Their designed conception or the four operators are intuitive and help us understand the intended evidence-use requirements. The experiments are extensive.

Critical weaknesses
  1. The presentation is very difficult to understand due to AI-generated contents and incomplete definition in the main text. Based on my perspective, the problem they study is not common/important from the related work and it's hard to find the applications in reality. From the methods, they only introduce or fomulate some conditions/constraints to optimize their violations. The contribution/novelty is not enough. And many operations in the method part are based on the existing work.

  2. The empirical details and evidence are lacked to support their claim in the main text. The contract compiler, grounding models, and intervention generators are not clear. For example, the CSR results are based on Evidence Contracts and intervention protocols built by the authors themselves, which can not support their claims just based on the current manuscript.

Questions for authors

NA

Reviewer_o9K2OpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper addresses the problem of controlling and verifying which evidence multimodal models rely on when making predictions. The central motivation is that a model may produce the correct answer while relying on inappropriate evidence, and conventional task accuracy or internal reliance measures cannot establish whether the model actually follows a specified evidence-use instruction. The authors propose Evidence Contracts, a specify–enforce–verify framework that translates natural-language instructions into four behavioral requirements: IGNORE, REQUIRE, PREFER, and PRESERVE. These requirements are enforced through contract-constrained adaptation and independently evaluated using held-out intervention mechanisms. Experiments across seven multimodal benchmarks report a held-out Contract Satisfaction Rate (CSR) of 68.4%, compared with 36.5% for GUIDE, while retaining 98.6% of the original task utility.

Critical strengths
  1. A meaningful and important research problem.
  2. A well-motivated framework with encouraging experimental results.
Critical weaknesses
  1. Insufficient clarity regarding training data construction and the actual capability of instruction-conditioned evidence control
  2. Limited clarity regarding the interpretation and scope of experimental validation
Questions for authors
  1. Could the authors provide concrete training and testing examples for the unseen-contract experiments?
  2. Can a single adapted model follow different evidence-use instructions without further parameter updates?
  3. Could the authors provide case-level comparisons with the original backbone and baseline methods?
Reviewer_KbGrOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Evidence Contracts, a specify–enforce–verify framework that compiles natural-language evidence-use instructions into explicit, behaviorally testable requirements. Evidence Contracts support compositional IGNORE, REQUIRE, PREFER, and PRESERVE clauses, respectively specifying invariance to designated evidence, task-consistent dependence on required evidence, relative dependence between evidence sources, and preservation of task performance. Across seven multimodal benchmarks, Evidence Contracts achieve 68.4% held-out contract satisfaction rate (CSR) on the common applicable benchmark–backbone settings.

Critical strengths
  1. The problem definition is clear: to identify that model predictions being correct do not necessarily mean that the specified evidence was actually used, and to make the compliance of evidence usage behavior a separate research goal.
  2. The method design is systematic: it transforms the requirements for evidence use into four types of combinable behavioral constraints, covering ignoring specified evidence, depending on necessary evidence, comparing evidence priority, and maintaining task performance.
Critical weaknesses

Dependence on evidence for location and intervention quality: The method requires external tools to locate evidence and verify whether the intervention is effective; inaccurate location or intervention may affect the execution of constraints and the evaluation results.

Another one is that, the number of references in this paper is relatively scarce. As I am not the expert for this domain, I can not judge that if the paper omits some important relevant references. And the authors should also explain this.

Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.