- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates how to accelerate feed-forward dynamic scene reconstruction from a fixed, synchronized, and calibrated multi-camera system. The authors propose T-LRM, a training-free framework that first classifies image patches as static or dynamic based on their appearance variation relative to the temporal mean. Before global interaction, static tokens at corresponding locations are averaged across timestamps and processed jointly, while dynamic tokens are retained separately for each timestamp. The shared static group and the timestamp-specific dynamic groups independently undergo global attention or TTT fast-weight interaction, after which replication and reordering restore the original token layout. The method is evaluated on N3DV, MeetRoom, ZJU-SelfCap, and ENeRF, and is applied to TTT-LRM and WorldMirror. The reported results show approximately twofold inference acceleration with a small average loss in reconstruction quality and a reduction in temporal optical-flow error.
Critical strengths
-
The paper addresses a practical problem with a simple, training-free method. Repeatedly processing static backgrounds independently at every timestamp introduces substantial redundancy in videos captured by fixed cameras. Exploiting this structure to reduce global interaction computation is sensible. The method requires neither additional training data nor a dedicated temporal modeling network, making it relatively straightforward to adopt.
-
The experiments include a useful control for ordinary frame parallelism. Table 2 shows that ordinary frame parallelism provides only approximately 1.11-1.12x overall acceleration, whereas the proposed method further reduces the runtime of Global Token Interaction substantially. This is more informative than simply comparing frame-wise and chunk-wise inference, and supports the conclusion that the gains do not arise entirely from batching.
-
Results across two global interaction mechanisms provide initial evidence of general applicability. Table 3 demonstrates acceleration on N3DV for both WorldMirror and TTT-LRM, supporting applicability to attention-based and TTT-based backbones. The appendix also clarifies that the main experiments use a shared classification threshold across scenes, which strengthens the experimental setup.
Critical weaknesses
-
The methodological novelty appears limited and primarily concerns incremental changes to the inference computation. In my view, the efficiency gains mainly arise from two design choices: organizing multiple timestamps into temporal chunks and reusing computation for static tokens across time, and processing static and dynamic tokens in separate global interaction groups to reduce computation. These choices have practical value, but the overall contribution appears closer to an optimization of the computation performed by existing 3D reconstruction backbones. Table 2 already establishes that the gains are not solely due to ordinary frame parallelism. The authors should therefore more clearly establish what static temporal reuse and grouped interaction contribute beyond existing token merging and computation reuse strategies.
-
The positioning of the method and the motivation for its temporal design need further clarification. The method accelerates reconstruction at individual timestamps by sharing static tokens across time, ultimately producing a corresponding set of 3D Gaussians for each timestamp. It does not construct a unified, continuous-time 4D Gaussian representation or explicitly model Gaussian trajectories or deformations across frames. This is a reasonable design for accelerating feed-forward reconstruction, but the rationale for this particular temporal processing strategy remains insufficiently explained.
-
The ablations do not isolate the effects of temporal reuse and the removal of static-dynamic interaction. According to Section 3.3 and Appendix B, the method simultaneously introduces two changes: sharing static tokens across timestamps and removing Global Token Interaction between the static and dynamic groups. For attention-based backbones, the interaction cost changes from T(S+D)^2 to S^2+TD^2, so savings arise from both static computation reuse and the removal of cross-group interaction terms. Table 2 rules out an explanation based solely on frame parallelism, but does not further disentangle these two changes. The experiments therefore support the effectiveness of the complete computation strategy, while leaving the incremental benefit of temporal-aware merging over simple grouped interaction unclear.
-
Full-image average metrics do not sufficiently establish preservation of dynamic details and temporal consistency across chunk boundaries. The method is motivated by the high proportion of static background in the input videos. However, this also means that full-image PSNR, SSIM, and LPIPS can be dominated by background regions. Noticeable degradation on moving subjects, occlusion boundaries, or fine structures may have only a limited effect on these aggregate metrics.
-
The timing scope is unclear, and throughput, output latency, and memory costs should be reported together. The Overall runtime in Table 2 exactly equals the sum of Frame-Attention and Global Token Interaction; for example, the 20-frame configuration reports 309+197=506 ms/frame. The manuscript does not clarify whether this includes patch classification, token merging and restoration, other network modules, Gaussian decoding, and rendering. It is therefore difficult to determine whether these numbers represent the runtime of the core modules or the complete reconstruction pipeline.
Questions for authors
-
Methodological novelty: What are the key contributions of static token reuse within temporal chunks and separate static-dynamic global interaction beyond existing token merging and computation reuse strategies? Could representative, applicable alternatives be compared using the same backbone, inputs, and evaluation protocol, reporting reconstruction quality, runtime, and memory consumption to establish the incremental value of the proposed method?
-
Method positioning and design motivation: The method shares static features across time while reconstructing 3D Gaussians separately for each timestamp. Could the authors further explain the rationale for this temporal processing strategy and how it balances motion fidelity and temporal consistency without explicitly modeling Gaussian trajectories or deformations across frames? Please clarify its advantages and limitations in computational efficiency, dynamic reconstruction quality, and applicable scenarios.
-
Attribution of the design contributions: Could controlled experiments separately quantify the effects of temporal static-token reuse and the removal of static-dynamic global interaction on speed and quality? For example, how does the method perform with the same static-dynamic partition but without temporal reuse? How does a sharing scheme that retains information exchange between static and dynamic tokens perform? These comparisons would help establish the individual contributions of the two choices and the need to combine them.
-
Dynamic regions and temporal consistency: Beyond full-image averages, could the authors report reconstruction quality on moving foregrounds, motion boundaries, and occlusion regions, together with separate temporal errors within chunks and across chunk boundaries? Can these results establish whether improvements in static backgrounds mask degradation on moving subjects? Please also specify the optical-flow estimator, error units, and frame-pair selection protocol used for tOF.
-
Complete efficiency assessment: Does the Overall runtime in Table 2 include patch classification, token merging and restoration, other network modules, Gaussian decoding, and rendering? Please define the timing boundaries and report complete pipeline runtime, processing time per temporal chunk, and latency to the first output. Memory consumption and gains over ordinary frame parallelism under a fixed memory budget would also help characterize the efficiency-resource trade-off of chunk-wise inference.
Dear Reviewers,
The authors have submitted their rebuttal to address your concerns. As the discussion deadline approaches, please review their response and share any remaining concerns or questions soon so the authors have enough time to respond.
Thank you, -AC