acceptodds
Under review as a conference paper at ICLR 2027

Measuring One-Step Visual Acquisition: Probe Shifts and Outcome-Supervised Value

Abstract

Visual-acquisition scores depend on how model responses are elicited and how task outcomes are scored. We examine these measurement dependencies using same-state, one-step crop interventions. Empirical Visual Intervention Effect (VIE) estimates an outcome-supervised continuation-reward contrast and serves as a costly offline reference. The comparator, termed realized information gain (RIG), computes KL divergence between four-choice readout distributions before and after acquisition under a fixed probe interface. On 120 constructed questions and 360 saved Qwen3-VL-32B-Instruct states, frozen empirical-VIE and RIG selections achieve 96.25% and 89.31% fresh stored-key agreement, respectively: a +6.94 percentage-point difference (95% question-cluster CI [4.10, 10.28]), conditional on realized discovery selections. A replication with Mistral-Small-3.2-24B-Instruct-2506, using its own states and separately sampled continuations on the same questions and stored keys, yields +10.56 points [6.39, 14.86]. Task construction limits these comparisons: 13 of 30 Geometry questions have no exact correct option, and source-conditioned option rules recover all 120 stored keys without images, exposing a shortcut without establishing model use of it. Exact-value rescoring with the original inputs and selections fixed yields a Qwen difference of +4.65 points [2.43, 7.22]; this is not evaluation on repaired tasks. A pre-specified 12-question development study also finds selection sensitivity to the tested prompt and readout changes, but fresh outcomes for variant-selected actions remain unmeasured. Learned-policy comparisons do not establish VIE-based superiority. The contribution is a controlled measurement protocol and a diagnosis of reward, readout and selection dependencies, rather than a deployable acquisition policy or a general ranking of acquisition principles.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.