Continuous Search Allocation: Jointly Shaping Probes and Updates for Query-Efficient Black-Box Video Attacks

Zeroth-order video attacks reduce query cost by structuring where the search is performed: they restrict the search space, select key frames or regions, or reuse a transformed perturbation across frames. We study another degree of freedom: how strongly each coordinate is explored. Continuous search allocation (CSA) derives a graded coordinate-wise allocation from the input before any victim query and applies it to both the zeroth-order probes and the update. A controlled factorial shows that probe weighting alone increases query cost, whereas joint probe–update weighting reduces it, with interactions of $-950$ and $-806$ queries on two victims. We instantiate CSA with codec motion vectors obtained without victim queries or additional model inference; the advantage persists when the motion-warped base probe is replaced by i.i.d. noise. Across five victims and two zeroth-order optimizers, CSA lowers CANQ, which charges failures the full query budget, in all ten comparisons against a control matched on probe amplitude and per-iteration update $L_1$, by $24.8\%$ on Kinetics-400 and $9.9\%$ on UCF101. At the default budget, larger savings are associated with more concentrated allocations. Under a tight $L_{\infty }$ bound, controlled sweeps reverse this ordering as high-weight coordinates increasingly saturate under projection.