acceptodds
Under review as a conference paper at ICLR 2027

MedGauge: Evidence-Grounded Commit Protocols for Reliable Tool-Using LLM Agents in Medical Measurement

Abstract

Language-model agents increasingly operate the pipelines that turn medical images into reported measurements such as ventricular volume and cardiothoracic ratio. When such a pipeline errs, its output stays well formed and physiologically plausible, and an agent given quality signals still explains them away in favour of the value it holds. Runtime enforcement moves the decision to a program that blocks a violating action or substitutes a predefined one, but neither response states what evidence would make a blocked measurement reportable. We present MedGauge, a small commit layer that decides which values may be signed as trusted and which action would make a rejected value acceptable. Its observation channels compute evidence without executing the stage they audit, and sources that share an origin count as one witness. Each alarm names the call that clears it, which turns a blocked report into a repair the agent can execute, and a commit protocol accepts a report only when preconditions checked on the tool-call record hold. On a benchmark of 1,068 cases spanning cardiac MRI, abdominal CT and chest radiography, MedGauge signs no wrong value across the 812 covered fault-and-reasoner combinations of its two primary reasoners, and across four reasoners it releases none of 416 conflicts that the available evidence cannot resolve. It also achieves the highest usable output of any compared agent on every task (56% to 66%), because declared discharge conditions let the agent clear false alarms and repair faulty cases. On abdominal CT, repairs rise from 3 cases to 62. With channels, alarms and discharge conditions fixed, removing only the commit check produces 30 silent commits and 107 released conflicts. One adapter per task extends the layer from volumes to a two-dimensional ratio, and we release the code and the benchmark construction.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.