acceptodds
Under review as a conference paper at ICLR 2027

Verdicts Without Evidence: Directional Critique Rewrites Abstention

Abstract

In fact verification and multi-agent review, a language model increasingly acts as the final adjudicator. It reads a claim, the retrieved evidence, and a critique written by another party, then returns SUPPORTS, REFUTES, or NOINFO. The evidence is meant to supply the grounds for a verdict; the critique is meant only to flag what the evidence may have missed. Nothing in the task makes that division binding. To see what a critique does when the evidence cannot settle the claim, we replace the evidence with registry metadata and keep NOINFO available. Across 200 claims, three commercial models, and five conditions of 2,400 runs each, we vary only a single critique sentence that recommends a direction. That sentence alone raises unwarranted verdicts from 6/2400 to 380/2400, a rise that holds on each of the three models. The new verdicts do not simply follow the recommendation, since 90 of the 380 contradict it. When we stratify the claims by material, every verdict on synthetic claims is REFUTES whatever direction the critique asks for, while on external claims every verdict follows the recommendation. Because frequency and direction move apart, we report two quantities: the commitment rate, how often the model leaves abstention, and the direction adoption, how often a verdict follows the recommendation. Under that decomposition, a careful-reading prompt reduces the frequency of commitment without changing the direction of the remaining verdicts. A rule stating that critique text is not evidence lowers commitment to 3.04% and reverses the composition of the verdicts that remain. Neither that rule nor a stronger evidence-only convention returns the model to its no-critique baseline. We propose Closed-Evidence Abstention Invariance as the target for abstention-enabled review: a critique mechanism has to be judged on commitment, direction adoption, and accuracy under evidence at once.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.