Read the Trajectory, Not Just the Endpoint: Calibrated Utility Readout for Transfer Attacks

Iterative transfer attacks are widely used to evaluate adversarial vulnerability on unseen models. Although their optimization trajectories contain multiple legal adversarial actions, existing methods almost universally deploy the final iterate. However, source optimization and cross-model transferability need not peak at the same iteration, and the preferred action can vary across images. Replacing the endpoint with one global checkpoint is therefore insufficient, while target-guided action selection would violate the zero-query transfer setting. To address these problems, we propose Calibrated Pointwise Relative Utility (CPRU), a target-output-free trajectory readout that estimates the signed utility of each saved action relative to the conventional final iterate. We show that the target-clean mean of this utility is exactly equal to the paired attack success rate (ASR) gain. CPRU uses four target-disjoint proxy models and an attack-specific utility regressor to score all non-final actions, independently returning the highest predicted positive-utility action while retaining final as an exact zero-utility fallback. Extensive experiments across different iterative attacks, data sets, source models, and target architectures demonstrate the effectiveness of image-specific trajectory readout.