acceptodds
Under review as a conference paper at ICLR 2027

Mechanism Evidence Contracts Reverse Biomedical Reranker Selection

Abstract

Molecular biologists made 235 of 288 correct adjudications with evidence packets that jointly changed passage order, type labels, and calibrated confidence, versus 203 of 288 with relevance-ranked packets; contradiction misses fell from 28/104 to 16/104. MEC-Audit traces this gap upstream by making the evaluator an experimental variable. Eight systems rank common top-100 PubMed Central pools, and their frozen outputs are mapped from one shared adjudication record to entity-level relevance or mechanism support requiring the correct operator, direction, and context. On 918 time-held-out claims and 8,426 judgments, generic judged-set Precision@5 selects BioLinkBERT (0.772), whereas mechanism judged-set Precision@5 selects MEC-Rank (0.742), 0.061 above REACH+BioLinkBERT. Maximizing sets change for 371/918 (40.4%) claims, 26.8% of 25,704 claim–system-pair preferences reverse, and mean rank agreement is 0.48. MEC-Rank realizes the contract with slot interactions, mechanism-preserving hard negatives, separate support and contradiction orders, and calibrated four-way types; contradiction recall is 0.621 versus 0.549 over 596 contradiction-bearing claims. Complete top-100 assessment of 240 claims preserves its leads at 0.711 versus 0.653 Precision@5 and 0.594 versus 0.522 contradiction recall. Across the 4,218-claim, 38,712-pair benchmark, independent labels, transfer, ablations, and the packet study show that evidence semantics can determine both the selected reranker and the scientific decisions it supports.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.