Consistency Loss Is Secretly Picking a Side: Risk-Directed Consensus for High-Stakes Decisions
Abstract
Consistency regularization makes predictions agree across semantically equivalent inputs. Under asymmetric decision risk, however, agreement is under-specified: it contracts within-orbit disagreement without determining where the shared prediction settles relative to a risk-critical boundary. In emergency triage, stronger symmetric consistency cut permutation-induced prediction flips by more than two-thirds but also increased under-triage. We formalize this missing degree of freedom as shared-solution selection, distinct from agreement contraction, and introduce disagreement-conditioned risk-directed consensus (DGN), which biases the direction in which inconsistent views approach consensus while letting that directional force vanish as agreement forms. On three triage corpora, DGN lowers under-triage relative to matched symmetric consistency at less than one point of over-triage per point removed. Across four additional 7B–14B LLM backbones, matched CE to BWOT to DGN interventions reproduce the same two-stage contraction–selection geometry, and DGN lowers both under-triage and flips relative to CE without a corresponding loss of ordinal predictive quality. A frozen-readout intervention roughly halves MIMIC under-triage without relearning the representation, reaching operating points that no train-selectable global threshold attains. A CIFAR-10-LT study with ResNet-32 and unordered labels reproduces both the contraction–selection separation and directional control outside clinical language, LLMs, and ordinal prediction. These results identify consensus location as a distinct and controllable design variable for consistency learning under asymmetric risk.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.