Auditing Optimization and Probabilistic Reliability in Frozen Vision-Language Representation Intervention
Abstract
We study whether optimization fit, probabilistic prediction quality, and discrete behavior move together under scheduled representation intervention in a frozen vision-language model. PAAF organizes the analysis into a deterministic Formation Schedule, normalized token-wise Allocation, and the Action that applies the resulting vector field. We compare the held-out training objective with Accuracy, ECE, Brier score, and NLL. In paired-coordinate KnG evaluations ( design), gain scheduling lowers the held-out training objective by while increasing NLL by and Brier score by . A controlled Allocation perturbation changes internal readouts but changes only 3 of 500 decisions. These results show that optimization fit, probabilistic prediction quality, and decision sensitivity are distinct properties under the evaluated intervention configuration. The practical implication is that representation-intervention schedules should be selected using held-out proper scores and decision-level checks, not the training objective alone.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.