- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper extends test-time visual prompt tuning for zero-shot depth completion (TestPromptDC; Jeong et al., 2025) from single frames to video. In TestPromptDC, an additive input-space prompt is optimized per frame so that a frozen UniDepthV2 matches sparse depth measurements. KFTP instead treats the prompt as a time-evolving state with an element-wise uncertainty map, updated by a Kalman-style predict–update recursion:
Prediction step: the previous prompt and its uncertainty are warped to the current frame with optical flow (MemFlow). A hand-designed process-noise term is added, combining a constant floor, inverse feature variance, forward–backward flow inconsistency and flow magnitude. Observation: the current-frame observation z_t is produced by 50 steps of test-time optimization initialized from the warped prompt. The loss combines sparse L1+L2, a second-order smoothness/"Poisson" term, and a KL temporal-consistency term on soft depth-bin histograms. Update step: the warped prior and z_t are fused with an element-wise Kalman gain, using an observation-noise map built from UniDepthV2's predictive uncertainty and the flow inconsistency. Experiments on KITTI, DDAD, vKITTI2, DyDToF, Sintel and NYUv2 report lower RMSE, MAE and a flow-based temporal error (TEPE) than frame-wise test-time adaptation baselines, with very large improvements on DDAD, DyDToF and Sintel. The appendix adds an iPhone ARKit evaluation, a sparsity study, cost analysis and comparisons with CAPA and supervised video depth completion.
Critical strengths
The idea is simple and sensible. Carrying the adaptation state (the prompt) forward in time, instead of re-solving from scratch each frame, is a natural way to add temporal consistency to test-time adaptation without retraining. The approach is a plug-in on top of TestPromptDC with almost no runtime overhead relative to it (Table 7: +1% time, +4% memory). The authors are honest about the Kalman framing. They do not overclaim: Sec. 4.4 and App. H state clearly that the update is the exact minimizer of a per-element weighted least-squares objective, that the uncertainty maps are designed rather than calibrated, and that no optimality of a probabilistic filter is claimed. App. K acknowledges prior Kalman-based depth fusion (Matthies et al., 1989). I appreciate this candour. The evaluation is broad. Six benchmarks spanning indoor/outdoor and real/synthetic data, plus real iPhone captures, a sparsity stress test (App. E), a cost comparison (App. F) and a comparison with the concurrent CAPA. The dataset-wise discussion around Table 3, which explains that gains are large where the frame-wise baseline is unstable, is a useful sanity check. The fusion-rule ablation is informative. Table 4 shows that the adaptive gain beats every fixed EMA weight on all three datasets, while fixed weights hurt on KITTI and NYUv2. The exploratory analysis supports the premise. App. J shows that frame-wise prompts become more correlated after flow alignment in the low- and mid-frequency bands, which motivates propagating the prompt.
Critical weaknesses
The only possible weakness I can think of is that the source of the large accuracy gains is not clearly identified: The headline improvements are very large: DDAD RMSE 5.65 → 2.98, DyDToF 2.31 → 0.94, Sintel 5.21 → 1.58. Plain temporal smoothing usually improves consistency, not per-frame accuracy, by factors of 2–3. Several mechanisms other than Kalman fusion could explain the gains such as: (a) Warm-starting. z_t is initialized from the warped previous prompt, so after t frames the prompt has effectively received about 50t optimization steps instead of 50. If frame-wise TestPromptDC with 50 steps is far from convergence, which its poor DDAD and Sintel numbers suggest (worse than raw UniDepthV2 on Sintel: 5.21 vs. 4.26), warm-starting alone could account for most of the gain. (b) Accumulating sparse measurements over time. If sparse points are re-sampled independently at each frame (e.g., 500 random points from dense GT on DyDToF, NYUv2 and Sintel), temporal propagation effectively aggregates many more depth measurements than a single-frame method sees. That would be an unfair advantage over frame-wise baselines, and it would also explain why gains are largest on datasets with synthetic sparse sampling and smallest on KITTI, whose LiDAR pattern is fixed. (c) The extra loss terms. L_poisson and L_temporal may contribute on their own.
The paper is missing an ablation of :
(i) TestPromptDC warm-started from the warped previous prompt, with no Kalman fusion and no extra losses; (ii) TestPromptDC with a matched total number of optimization steps; (iii) KFTP without L_temporal and without L_poisson; (iv) KFTP without each Q term and without R_Φ; (v) an explicit statement of whether sparse samples are fixed or re-sampled across frames, with results under a fixed pattern. with that ablation in place, coming back with clear results, I would consider a 4.
Minor problems:
In Table 2, DepthCrafter's Sintel row (RMSE 14.277, MAE 12.191, TEPE 1.112) is identical to its DDAD row in Table 1, which strongly suggests a copy-paste error. The Fig. 4 caption mentions only NYUv2 and KITTI, but the figure also shows DyDToF. Sec. 4.3 is titled "Prediction Step" but describes the observation/update step. There is a stray "d" at the end of the first sentence of Sec. 4.4.
Questions for authors
- Are the sparse depth samples fixed or re-sampled at each frame, for each dataset? If re-sampled, please report results with a fixed sampling pattern.
- What is the performance of TestPromptDC warm-started from the warped previous prompt, without Kalman fusion or extra losses? And with an equal total step budget?
- Please provide component ablations for L_temporal, L_poisson, each term of Q_t, and R_Φ.
- In Table 4, do the "None" and "EMA" rows use warm-start initialization and the same loss as KFTP?
- What are the values of the proportionality constants, Q_base, β, Σ_0 and K/σ for the soft binning, and how sensitive are results to them?
- How are UniDepthV2's depth-space uncertainties mapped into a prompt-space variance?