- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Sparse VFI: a lightweight "Light Path" predicts a coarse frame; a small "local Expert" refines only k tiles chosen by a Gate. The paper's central move is Actual Expert Gain, G_t = MSE(coarse_t, gt_t) − MSE(expert_t, gt_t) — the signed reduction from actually executing the frozen Expert. Tile interiors are disjoint and the Expert processes crops independently, so frame MSE is additive in G_t and top-k over signed gains is an exact oracle. This separates V_or (full-information value), V_Z (best from input features), and V_s (what a policy realizes).
Critical strengths
- The paper identifies a clear and practically meaningful problem in sparse refinement.
- The paper is clearly written and appropriately cautious about its conclusions.
Critical weaknesses
- The conceptual novelty appears somewhat limited. The central principle—routing according to the realized benefit of a downstream expert rather than an input-side difficulty/confidence proxy—is closely related to learning-to-defer and outcome-aware routing. Under the deliberately disjoint tile setting, Proposition 1 then follows rather directly from the additivity of MSE.
- The paper introduces a useful decomposition between full-information oracle value V_{or}, the feature-conditioned optimum V_Z, and the learned policy V_s, but empirically leaves the most important quantity V_Z essentially unresolved. Consequently, the large oracle-to-Gate gap cannot be attributed to intrinsic unpredictability from deployable inputs versus limitations of the chosen features, the very small Gate, or the surrogate objective.
- The empirical conclusions are demonstrated on a rather small and highly specific VFI system.
Questions for authors
- Jaccard overlap of tiles selected by Uncertainty vs each Gain-Gate seed (dev and locked)?
- Held-out Spearman between Gate score and realized G_t?
- Can you train an over-capacity predictor on the same Z and report PSNR at k=6/k=12? (Highest-value addition.)
- How often does the coarse value lie between endpoints, and what is ρ(m, u)?
- Predicted-gain > 0 threshold policy: PSNR, harmful-call rate, average coverage?
- Do the latency conclusions hold on GPU and at 1080p?
Response to Q1–Q6 — Selection diagnostics, capacity, thresholding, and runtime
Thank you for the detailed experimental suggestions. We have carried out the additional selection, capacity, and thresholding analyses. Below, we report these results and clarify the scope of the existing runtime measurements.
Selection diagnostics. Jaccard measures selected-set overlap with Uncertainty and is averaged per image. Spearman correlates router scores with realized gains across all 112 tiles within each image, using average ranks for ties, and is then averaged across images.
Uncertainty's evaluation score–gain Spearman is 0.43740. The original Gates select substantially overlapping tiles but have lower full-ranking correlation with realized gain.
Capacity control. We used the same 18-channel features, signed-gain labels, training cache, and development-based checkpoint selection. The table reports mean per-image evaluation PSNR gains (dB) at k=6/12, with one common initial seed per configuration.
For the largest Gate, three-seed mean ± sample SD is 0.14785 ± 0.00032 / 0.22681 ± 0.00026 dB (Smooth L1) and 0.14774 ± 0.00019 / 0.22637 ± 0.00013 dB (MSE). Uncertainty gives 0.14849/0.22697 dB; the gain oracle gives 0.2586/0.3562 dB. Increasing tested pointwise Gate capacity about 120-fold does not recover the oracle margin under this protocol. These controls use regression alone; the submitted Gate also used ranking. The unrestricted feature-conditioned optimum VZ is not estimated.
The coarse RGB value lies within the closed endpoint interval at 69.18%/68.08% of scalar channel positions on development/evaluation. Mean within-image tile-level Spearman between motion m and disagreement u is 0.9912/0.9908. The cues therefore induce very similar tile rankings here.
Using the width-64 MSE Gate, we select predicted-positive tiles up to cap k. Uncertainty is matched to the actual number of selected calls within each image.
Gains are relative to the common coarse output. Harmful-call rate is the fraction of all selected calls with Gt<0. Evaluation coverage is 5.36%/10.70% of 112 tiles, with no zero-call images. Paired differences from matched Uncertainty are +0.00026/-0.00033 dB, with 95% image-bootstrap CIs [-0.00146, 0.00191]/[-0.00189, 0.00121]. The zero threshold almost fills both caps while about 19–20% of calls remain harmful, so this zero-threshold policy provides little useful abstention.
The measured scope is eight-thread CPU at 256×448: output-matched vectorization reduces local-stage median time from 44.31 to 27.37 ms (1.62-fold). The submitted full-system sparse/dense medians are 241.00/229.70 ms. GPU and 1080p timings have not been measured.
Evaluation protocol. These follow-up analyses are exploratory and reuse the originally held-out split after repeated inspection. The paired image-bootstrap intervals quantify image-sampling variation conditional on the evaluated models; seed variability is reported separately.
Thank you for helping us examine the routing results in greater detail. The larger-Gate experiments directly test the capacity question, and the selection and thresholding diagnostics clarify how the fitted policies allocate and stop refinement calls. We will include these results and their experimental settings in the revision, and make the scope of the CPU latency measurements explicit.