acceptodds
Under review as a conference paper at ICLR 2027

CREDO: Probe Signal Quality Is the Binding Constraint on Certified VLM-Pool Routing

Abstract

A per-tier false-accept bound does not bound the error of the traffic it admits; that error is set by the base rate and by how well the probe separates right answers from wrong ones. We show this on CREDO, a training-free router over a pool of vision-language models (VLMs) and, to our knowledge, the first to certify a user-set answer-quality budget at every operating point of such a pool: a finite-sample, distribution-free, per-tier bound at the user's budget α on the rate at which a tier's wrong answers are accepted, built from a rule-based probe, split-conformal thresholds and a cheapest-certified-first cascade. It covers the queries clearing those thresholds; under the default strongest-tier fallback the rest ship uncertified. On a frozen four-tier pool over six benchmarks and 9 calibration splits, the bound holds on average for the quantity it bounds and fails for the quantity a deployer reads: the conditional error of certified traffic exceeds α at 23 of 30 operating points, stable across splits. The calibration fold predicts the gap before it is measured, a label audit leaves it essentially unchanged, no weak-signal channel raises pooled discrimination appreciably on any split under the shipped pooling, and a learned fusion of the same inputs and a hidden-state probe above the registered bar still leave most operating points over budget. Routing adds no accuracy over the best fixed tier and, once the probe is charged, no cost saving over the strongest tier. The certificate's reach is real and its reach grows with the risk level, but at every level the error a deployer reads is set by the base rate and the probe rather than by the bound.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.