- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes StreamTTO, an online test-time optimization framework for video depth completion. A frozen feature extractor with causal temporal attention computes features once per frame and caches them, and only the shared depth decoder is updated using sparse observations. CSWO performs R = 2 window passes over a W = 3 causal window per incoming frame, and decoder parameters and optimizer state are carried forward. The paper also introduces MS-Depth, approximately 40 hours of synchronized RGB–LiDAR recordings with continuous transitions among indoor, underground, and outdoor environments. On KITTI, Bonn, NYUv2 Raw, TUM RGB-D, DDAD, and MS-Depth, StreamTTO achieves the lowest RMSE on most benchmarks, at approximately 15 FPS (VGGT) and 25 FPS (MoGe-2) on MS-Depth.
Critical strengths
On MS-Depth, StreamTTO achieves 15.73 (VGGT) and 25.90 (MoGe-2) FPS, compared with 0.426–1.652 FPS for Marigold-DC, TestPromptDC, and CAPA-LoRA. In all four conditions, the MoGe-2 and VGGT variants achieve the lowest and second-lowest RMSE, respectively. The compute-matched transition experiment (Table 4) and warm-start applied to other TTO methods (Table 13) support the design.
Critical weaknesses
-
Retaining model parameters across frames, updating on current and recent frames in a sliding window, and performing few updates per frame were already proposed in Wang et al. (2025b), which also reports that one update per frame is sufficient and faster than reset with 20 updates per frame. The paper cites this work only in Appendix E, as adapting encoders through reconstruction objectives, without discussing these points. The main differences are the adaptation location (the depth decoder on a frozen feature extractor) and supervision from sparse observations, which enable reuse of cached features. This relation should be stated in the Related Work.
-
The motivation stresses variations in measurement density, spatial distribution, and noise. Because decoder parameters and optimizer state are carried forward, errors from incorrect sparse observations may persist across frames. However, no experiment adds noise to the sparse observations or changes their spatial distribution on the same dataset; the only controlled robustness experiment reduces observation density on NYUv2 Raw (Appendix A.8). Robustness to sensor variation, a main motivation of the paper, therefore remains unverified.
Questions for authors
-
How does StreamTTO perform when noise is added to the sparse observations, or when their spatial distribution changes within a sequence (e.g., from LiDAR rings to uniform sampling)? Since decoder parameters and optimizer state are carried forward, does accuracy recover after such frames?
-
How were λ = 0.1, the clipping threshold of 1.0, and weight 2 for the current frame selected? In Table 7, equal weights at W = 3 and R = 2 give an unobserved RMSE of 0.8696 m, nearly the same as the default 0.8694 m.
-
Appendix E states that some frames have fewer than 50 LiDAR points. The scale and shift in Eq. 3 are used both in the adaptation objective and in the final prediction (Eq. 2), and the updated decoder is carried forward. How does StreamTTO perform on these frames compared with the depth completion baselines, and is the alignment or the update handled differently for such frames?
Dear Reviewers,
Thank you for taking the time to review our paper so thoughtfully. We know this is a busy time for everyone. If you have even ten minutes to spare, we’d really appreciate it if you could take a look at our rebuttal.
We wanted to leave a quick follow-up while the discussion is still open. If your schedule is tight, we’d be grateful for any feedback you’re able to share toward the end of the discussion period.