acceptodds
Under review as a conference paper at ICLR 2027

DDD3R: Directional Decomposition and Dampening for Recurrent 3D Reconstruction

Abstract

Recurrent 3D reconstruction models process video streams by iteratively updating a latent state, yet we find that this update process suffers from three compounding failures. Update magnitudes are systematically too large, causing progressive degradation over long sequences. All existing adaptive gating mechanisms collapse to near-constant values, providing no meaningful temporal adaptivity. And update vectors contain substantial directional redundancy, repeating historical drift rather than encoding new geometric information. These failures share one cause. Any scalar gate modulates the update uniformly across all directions in feature space, so it cannot distinguish redundant drift from novel content and reduces to blunt magnitude control. Motivated by this diagnosis, we propose DDD3R (Directional Decomposition and Dampening for Recurrent 3D Reconstruction), a training-free module that replaces scalar gating with directional decomposition. DDD3R projects each per-token state update onto a tracked drift direction and applies differentiated suppression, strongly attenuating drift while preserving orthogonal, novel information. This formulation strictly generalizes prior approaches. Constant dampening, temporal braking, and full directional decomposition all emerge as special cases of a single update rule parameterized by how much to differentiate drift from novelty. A drift-energy-adaptive extension parameterizes a continuous spectrum from full directional decomposition toward isotropic dampening that can be tuned per scene. Applied as a plug-in to existing models without retraining, a single fixed configuration (DDD3Rbrake, the scalar relaxation of the full rule) reduces camera pose error by 62-68% on long indoor sequences (TUM, ScanNet at 1000 frames), and the directional-decomposition variant (DDD3Rortho) yields a further 13% relative reduction over DDD3Rbrake on TUM where drift carries mostly harmful repetition. DDD3R also improves video depth estimation by 13-20% across three benchmarks (KITTI, Bonn, Sintel), while adding no learnable parameters and negligible computational overhead.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_MDAWOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the state-update dynamics of recurrent 3D reconstruction models, focusing on long-sequence performance degradation. The authors identify three potential issues: excessive update magnitudes, limited temporal variation in existing adaptive gates, and directional redundancy among consecutive state updates.

To address these issues, the paper proposes DDD3R, a training-free framework that decomposes per-token state updates into components parallel and orthogonal to an exponentially averaged update direction. The two components are then reweighted separately to suppress redundant updates while preserving potentially novel information.

The authors investigate several configurations, including constant dampening, temporal braking, and explicit directional decomposition. Experiments on TUM, ScanNet, KITTI, Bonn, Sintel, and 7Scenes demonstrate improvements in camera pose estimation, video depth prediction, and 3D reconstruction. Additional analyses examine the relationship between drift energy and the effectiveness of different update strategies, including evaluations on ARKitScenes and Point3R.

The central contribution is an empirical analysis of recurrent state-update dynamics, together with a lightweight inference-time regulation strategy intended to improve long-sequence reconstruction without retraining.

Critical strengths

S1. A practically valuable diagnosis of recurrent state-update dynamics.

The paper provides useful empirical evidence that excessive state updates contribute substantially to long-sequence reconstruction degradation. In particular, the observation that simple constant dampening already produces substantial improvements is informative and highlights an important issue in recurrent reconstruction architectures. The analysis of update magnitudes, gate statistics, and directional similarity provides a useful perspective for understanding long-horizon state accumulation.

S2. Strong practical improvements without retraining.

The proposed regulation strategies are lightweight and require no additional learnable parameters. DDD3Rbrake reduces ATE from 0.103 to 0.063 on TUM and from 0.406 to 0.261 on ScanNet relative to TTT3R at 1000 frames. The improvements across multiple reconstruction-related tasks suggest practical value. The comprehensive evaluation across multiple datasets and tasks provides valuable insights into the effectiveness and limitations of different update regulation strategies.

Critical weaknesses

W1. The empirical results do not sufficiently establish the independent contribution of directional decomposition.

The central methodological contribution is the separation of state updates into drift-aligned and orthogonal components. However, the strongest and most consistent results are often obtained by DDD3Rbrake, which applies scalar temporal modulation without explicit directional decomposition.

The full directional variant improves TUM 1000-frame ATE from 0.063 to 0.055 relative to DDD3Rbrake, but performs substantially worse on ScanNet (0.488 versus 0.261) and also degrades performance on ARKitScenes relative to simpler alternatives.

Although the authors acknowledge these dataset-dependent effects, the evidence does not yet establish when directional decomposition provides a reliable advantage over simpler scalar regulation. This distinction is particularly important because the default directional hyperparameters were selected using TUM, which contains only eight evaluation sequences.

W2. Several mathematical inconsistencies weaken the claimed unified formulation.

Equation (12) defines the adaptive parallel coefficient using etγe_t^\gamma. For 0<et<10 < e_t < 1, increasing γ\gamma causes this term to approach zero, so the resulting coefficient approaches α∥\alpha_\parallel, corresponding to stronger directional differentiation rather than isotropic dampening. This contradicts the interpretation of the parameter sweep in Appendix A.4. Moreover, the special treatment of γ=0\gamma = 0 in Algorithm 1 introduces a discontinuity in the claimed continuous spectrum.

Equation (7) computes the EMA drift direction using the current update before performing the projection. Consequently, setting βema=0\beta_{\mathrm{ema}} = 0 makes the reference direction equal to the current normalized delta, rather than the previous delta as assumed in Appendix A.8.

Questions for authors

Q1. What is the actual implementation of the drift-energy-adaptive interpolation?

Equation (12) implies that increasing γ\gamma moves the method toward the directional configuration rather than isotropic dampening, contrary to the discussion in Appendix A.4. Could the authors clarify whether the experiments use the stated equation and explain the discontinuity between γ=0\gamma = 0 and γ→0+\gamma \rightarrow 0^+?

Q2. Which EMA direction is used for projection, and how is temporal braking related to the unified formulation?

Algorithm 1 appears to construct dtd_t using the current delta before projection, whereas Appendix A.8 assumes that setting βema=0\beta_{\mathrm{ema}} = 0 recovers the previous update direction. Could the authors clarify the implementation and provide a comparison between projecting onto dt−1d_{t-1} and the updated dtd_t?

Additionally, is DDD3Rbrake intended to be an exact special case of Equation (6), or only a qualitatively related scalar baseline?

Q3. Can the authors provide stronger evidence for the independent benefit of directional decomposition?

Since DDD3Rbrake outperforms DDD3Rortho on several benchmarks, the distinction between scalar regulation and directional decomposition is central to the paper's contribution.

Could the authors report per-sequence paired comparisons, confidence intervals, and statistical significance for DDD3Rortho versus DDD3Rbrake on TUM? Given that the default directional parameters were tuned on TUM, additional evidence from independently selected hyperparameters or held-out sequences would also help establish whether the reported improvement generalizes.

Reviewer_PSN7OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes that the degradation of 3D estimation on long sequences is due to three factors: (1) latent state is overly updated, (2) the gate change along the time is small, (3) update direction is repeated. Then it proposes to decompose the update of each token into the parallel and orthogonal directions w.r.t the history direction.

Critical strengths
  1. The paper provides valuable insights and visions on how the updates of latent states affect the streaming 3D estimators. It found that reducing the update brings improvement and further studied the cause behind. A carefully designed direction decomposition is proposed to improve the state update.

  2. The method is training-free and plug-and-play on related streaming base models, which is practical and useful.

  3. The degradation on ScanNet, Sintel, and ARKitScenes are also reported, demonstrating the limitation and point out the future improvement direction.

Critical weaknesses

The improvement from direction decomposition is not cleanly isoluated or ablated. The brake baseline is quite good without direction decomposition. Hence we couldn't clearly tell the improvement is from less pace or corrected direction.

Questions for authors

If the per-token update follows the same norm as ortho without direction decomposition, how is the performance on TUM and ScanNet? Would that change the conclusion?

Reviewer_qpVAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes DDD3R, a training-free state-update regulator for recurrent 3D reconstruction. It decomposes each token’s update into components parallel and orthogonal to an EMA-tracked update direction, and applies stronger attenuation to the parallel component. The paper also studies constant dampening and a scalar temporal-brake variant. Experiments cover camera pose estimation, video depth prediction, and 3D reconstruction, with additional evaluations on ARKitScenes and Point3R.

Critical strengths
  1. The results are promising, and the analysis is thorough. The work aligns well with the scope of the conference.
  2. Constant dampening provides an informative baseline. The sequence-length experiments show how reconstruction errors accumulate over time.
  3. Experiments cover pose estimation, depth prediction, and 3D reconstruction. ARKitScenes further tests the proposed diagnosis on unseen data.
Critical weaknesses
  1. The framework largely shifts the problem to hyperparameter search. The preferred configuration changes across datasets, yet no reliable rule is provided for selecting one on unseen scenes.
  2. The directional variant does not consistently outperform existing methods. On ScanNet-1000, its ATE is 0.488, worse than both TTT3R (0.406) and TTSA3R (0.378), and it also underperforms TTT3R on ARKitScenes.
  3. The added value of directional decomposition remains unclear. Constant dampening already achieves substantial gains, while the most robust variant still uses scalar gating, leaving the need for directional control insufficiently demonstrated.
  4. The gate-collapse diagnosis overstates the limitations of prior methods. Stable frame-averaged gates do not rule out useful token-level temporal adaptation in TTT3R or TTSA3R.
  5. The mathematical description is internally inconsistent. The EMA timing differs between the equations and Figure 2, while the claimed γ interpolation direction contradicts Equation (12).
Questions for authors

see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.