When Does Local Refinement Help? Diagnosing Routing Targets for Sparse Video Frame Interpolation

Sparse video frame interpolation allocates a local Expert to a small subset of image regions. Effective allocation requires distinguishing prediction difficulty from the improvement that the available Expert can deliver. We freeze a full-frame predictor and a local residual Expert, then measure Actual Expert Gain: the signed reduction in reconstruction error from each tile-level intervention. An additive formulation separates the target-aware oracle, the feature-conditioned optimum, and the learned policy. On 500 Vimeo90K development examples, 37.59% of 56,000 interventions increase error. At six selected tiles per image, the gain oracle improves PSNR by 0.243 dB, compared with 0.132 dB for endpoint-disagreement routing. On a pre-frozen locked evaluation of 3,281 valid examples, three Gain-Gate seeds show no statistically resolved improvement over Uncertainty at six or twelve selected tiles (paired image-bootstrap 95% confidence intervals include zero). Separately, vectorizing identical halo crops yields a 1.62× local-stage median speedup, while full sparse and dense systems have near-matched CPU latency. These results distinguish realized intervention value, its predictability from deployable features, and its execution cost, which must be evaluated separately when assessing sparse refinement.