acceptodds
Under review as a conference paper at ICLR 2027

Reading Is Not Verifying: Auditing Relation-Conditioned Evaluators for Selection and Guidance

Abstract

Evaluation scores for relation-conditioned generation may respond to a declared relation yet prefer outputs that violate it. We separate these as relation sensitivity and relation completeness and audit each: one intervention changes the declared relation at fixed output, the other breaks the relation at fixed declaration. In covalent protein–ligand structure prediction, a transferred contact score treats the required bond as a clash and accepts only 1.3% of 4,587 deposited complexes. A public pipeline already ships an exclusion for it; a CASP16 entry demotes predictions failing such checks. Excluding graph-local bonded neighbours raises acceptance to 95.8% while keeping sensitivity to distal clashes, yet withholding the false penalty adds no test that the bond exists: for this capped contact-score family, we prove mask-only exclusion relation-incomplete under large ligand translations. The repaired score prefers a broken bond under displacement on 65 of 272 frozen predictions. Bounded rigid-body search finds bond-violating solutions on 93.0% of tasks under the original score and 30.5% under the repair. Despite that gap, the repair improves controlled guidance, preserving 197 of 200 bonds against 0 of 200 unrepaired. Selection depends on the pool: correct-bond selection rises from 36.5% to 73.9% in mixed six-candidate pools, but a preregistered within-generator test does not establish an effect. Five further panels probe the audit's limits. On HumanEval+ a text-similarity score selects a program with explicit failing-test labels on 36.1% of eligible tasks, and an English–Chinese replication gains 18.5 percentage points in terminology compliance under checker-gated selection. Repairing a mismatch can therefore improve decisions without ensuring the requirement is met, so sensitivity and verification need separate tests before a score drives selection or guidance.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.