acceptodds
Under review as a conference paper at ICLR 2027

MeMix: Selective State Preservation for Streaming 3D Reconstruction

Abstract

Streaming 3D reconstruction must absorb new observations while retaining historical context in bounded memory. This requires deciding not only how strongly to update a recurrent state, but which locations should temporarily avoid a write and when to reselect them. We present MeMix, a training-free module that allocates a fixed preservation budget using current-frame compatibility. At each frame, it scores candidate state locations against the observation, preserves the previous values at the highest-scoring locations, and leaves the remaining locations on the original update path. MeMix adds no learnable parameters and keeps recurrent memory constant in stream length. Matched interventions show that compatibility-guided selection and current-frame reselection improve long-horizon performance beyond gate binarization or a fixed protected subset. We evaluate MeMix across update rules sharing a pretrained CUT3R checkpoint. On 16 1,000-view 7-Scenes streams, MeMix reduces TTSA3R's accuracy and completeness errors by 29.9% and 30.7%. With the FILT3R Kalman update, it lowers 1,000-view ScanNet pose ATE by 17.8% across 65 scenes.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
streaming 3D reconstruction, recurrent state, selective state preservation, long-horizon reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3zbyOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MeMix proposes a training-free state preservation mechanism for recurrent streaming 3D reconstruction models. After computing the candidate state and current-frame readout, the method measures compatibility between candidate state tokens and the mean projected image feature, freezing a small fraction of old state entries while allowing the rest to follow the native update rule. The authors apply this masking strategy across direct, soft-gated, and covariance-aware updates (built around the CUT3R state family) and evaluate it on long-horizon 3D reconstruction and pose estimation benchmarks with minimal runtime overhead.

Critical strengths
  1. The proposed intervention is plug-and-play—requiring no retraining or extra parameters—and the before/after comparisons are cleanly set up using identical checkpoints, inputs, and evaluators. It adds almost no computational overhead while delivering clear error reductions on long sequences (such as the 1,000-view TTSA3R benchmark) and generalizing to covariance-aware updates like FILT3R.

  2. I really appreciate the thorough sanity checks and honest reporting of limitations. The authors rule out trivial explanations by comparing against random masks, periodic updates, reverse allocation, stale masks, and continuous write-strength sweeps, while using scene-level bootstrap resampling rather than treating correlated frames as independent. The paper is also upfront about short-horizon trade-offs and very long-horizon failure cases instead of cherry-picking only positive results.

Critical weaknesses
  1. Because all tested update rules (CUT3R, TTT3R, TTSA3R, FILT3R) rely on the shared CUT3R pretrained backbone and state representation, it remains unclear whether this compatibility heuristic generalizes to other recurrent 3D architectures. Moreover, the feature-compatibility score is purely empirical, and the theoretical analysis—while transparent about its assumptions—does not guarantee downstream reconstruction gains. In practice, hard-freezing old state entries can become a liability if the initial state was poorly estimated or the scene is dynamic, which likely contributes to the regressions observed on very long trajectories. In addition, the write-strength sweep suggests that strong update damping works nearly as well, meaning exact zero-write preservation is not strictly required.

  2. A few experimental inconsistencies and missing scripts need to be cleared up. First, the default TUM numbers (even for the unmodified baseline) do not match between Table 3 and the write-strength ablation, which needs an explanation of whether different sequence subsets or settings were used. It is also unclear whether the default preservation ratio was tuned on a held-out split or directly on the test benchmarks. Second, while the provided code covers the base CUT3R/TTT3R/TTSA3R evaluations, the scripts and commands needed to reproduce the FILT3R experiments, the 5-scene 3,000-view stress test, and the paired bootstrap intervals are missing. Finally, because ATE is computed after prefix-specific Sim(3) alignment, it is fair across methods but should not be over-interpreted as maintaining a globally consistent metric scale over time.

Questions for authors
  1. On which specific sequences and frame horizons do the longer-run regressions occur, and do they stem from locking in noisy early states or actual scene changes?

  2. How was the default preservation ratio chosen—was it tuned on a separate validation split?

  3. Why do the baseline and default MeMix numbers on TUM differ between Table 3 and the write-strength sweep?

  4. Could you provide the exact code scripts and commands for the FILT3R integration, the 3,000-view experiment, and the paired bootstrap resampling, as well as clarify which sequences were used to compute the 79.3% retained-alignment statistic?

4 comments

W1 shared backbone · Q1 / W3 hard-freezing and long-run regressions · W4 exactness

Thank you for the careful check and the supportive assessment. In short: applied unchanged to Mem3R, a separately trained model with its own learned gate, MeMix lowers ScanNet ATE by 10.5% on 65 scenes (interval excluding zero). The Table 5 TUM numbers, the choice of p, the scripts, Sim(3), and the empirical score are in our next comment.

Beyond the shared CUT3R checkpoint (W1). The four update rules we test are structurally different (direct replacement, two soft gates, and a Kalman gain with covariance), and on the same 65 ScanNet scenes MeMix lowers long-horizon ATE under all four with every paired interval excluding zero (CUT3R/TTT3R/TTSA3R 50/65, 54/65, 58/65 scenes for the 400–1,000-view mean; FILT3R 54/65 at 1,000 views; Appendix A9). To go beyond the shared checkpoint, we tested Mem3R (code released after our submission). It writes its 768-token state through a learned channel-wise gate and adds a fast-weight memory for camera tracking; it is initialized from CUT3R's weights and trained further with the encoder frozen. MeMix sets that gate to zero at the 60 protected tokens. We used the authors' public code and 64-view checkpoint. As a check of this setup, our runs on the eight TUM streams against the values reported by the Mem3R authors (ATE, m):

OursReported
CUT3R / TTT3R / TTSA3R, 1,000 views0.1680 / 0.1079 / 0.09510.17 / 0.11 / 0.091
Mem3R, 400 / 1,000 views0.0781 / 0.13040.078 / 0.15

Applied unchanged, MeMix lowers Mem3R's mean ATE on both benchmarks (Δ = Mem3R − Mem3R+MeMix):

Mem3R+MeMixΔ [95% CI]MeMix better
ScanNet, 65 scenes, 1,000 views0.68480.6128 (−10.5%)0.0720 [0.0405, 0.1075]49/65
ScanNet, mean over 400–1,000 views0.50830.4315 (−15.1%)0.0768 [0.0498, 0.1052]55/65
TUM, 8 streams, up to 1,000 views0.13040.1208 (−7.4%)0.0096 [−0.0048, 0.0227]5/8

On ScanNet the interval excludes zero at every prefix from 300 views on; on TUM's eight streams the interval includes zero. MeMix thus transfers on ScanNet to a separately trained model with its own learned gate (Section 4.2; Appendix A11); recurrent models not derived from CUT3R remain untested.

Hard-freezing and long-run regressions (Q1 / W3). Where and when. Through 3,000 views MeMix lowers ATE on all five long ScanNet scenes at each evaluated horizon (15/15 scene–horizon cells; scene 0747 is tied at 3,000 views). The regressions come later, on the three scenes that extend to 4,000 views: scene 0747 is worse from 3,100 views on (by up to 44.4%, at 3,700 views) and turns the three-scene mean into a loss after 3,400 views (mean ATE at most +9.4%, at 3,700 views; +4.8% at 4,000), while the other two scenes keep lower ATE through 3,900 views. The per-scene table is in our second reply to H7Yz and in Appendix A5 (Table 7).

Cause. Not scene change: the regression occurs on static ScanNet scenes, while on TUM's dynamic sitting/walking streams MeMix has lower ATE than TTSA3R on all eight at every horizon from 800 to 1,000 views. Against a locked-in noisy early state the evidence is indirect: scene 0747 is 37.8% better with MeMix at 1,000 views and 24.5% better at 2,000 before it turns after 3,000, and the protected set is reselected every frame (a fixed initial mask is worse, ATE 0.0979 vs 0.0767). Releasing the protection after 3,000 writes does not recover the ATE benefit (three-scene mean 0.0714 m higher at 4,000 views than with continued protection; higher on 2 of 3 scenes), so protection late in the stream does not explain the ATE reversal, although the release does remove most of a late rise in rotational RPE (Appendix A5). Weaker suppression from the start (λ = 0.1) keeps the three-scene mean below TTSA3R at every evaluated horizon (−4.7% at 4,000 views; −6.6% vs exact zero's −10.3% at 3,000). With three scenes, the cause is not identified.

Exactness (W4). What MeMix adds is the choice of locations: at the same budget, protecting the 60 locations the update rule itself would write most strongly is worse than MeMix in TUM ATE, both 7-Scenes errors, and 65-scene ScanNet ATE (0.332 vs 0.300 m), all intervals excluding zero. What the selected locations need is strong suppression; exact zero is its parameter-free default, not a unique optimum, as Section 4.4 and the Table 5 caption now state: on 7-Scenes, the λ = 0.1 interval against exact zero includes zero (accuracy Δ −0.0025 [−0.0070, 0.0017]) while the full native write is worse (+0.0267 [0.0154, 0.0392]); on TUM, the intervals for λ = 0.1 and 0.25 include zero and λ = 0.5 and 1 are worse (next reply; Appendix A9).

Q3 TUM numbers · Q2 choice of p = 60 · Q4 scripts and the 79.3% statistic · Sim(3) · W2 empirical score

TUM numbers in Table 3 vs Table 5 (Q3). No sequence subset differed: both tables use the same eight streams, 7,023 views, and evaluator. Table 5's TUM column came from a separate, earlier run. Its λ = 0 and λ = 1 arms, mathematically identical to MeMix and TTSA3R, gave 0.0793 and 0.0945 instead of 0.0767 (Table 3) and 0.0951 (Appendix A4). We reran the TUM sweep on the released code path: its λ = 0 and λ = 1 trajectories are byte-identical to the MeMix and TTSA3R runs on all eight streams, and Table 5's TUM column is replaced:

λ00.10.250.51
TUM ATE (new column)0.07670.08510.09190.08760.0951
Δ vs λ = 0, 95% CI—[−0.0009, 0.0210][−0.0000, 0.0355][0.0042, 0.0197][0.0065, 0.0319]

The conclusions are unchanged: the new column orders the five λ values exactly as the old one did, no mean moves by more than 0.0026 m (a seventh of the TTSA3R − MeMix difference, 0.0184 m), and the per-stream slope of ATE on λ is positive on all eight streams (Appendix A9). On the released code path, the TUM MeMix and TTSA3R trajectories have since been reproduced byte-for-byte in three separate batches, and Table 5's 7-Scenes column matches Table 3 and Section 4.1 exactly. Only that one run deviated: its trajectories differ from the released-path runs by about 10⁻⁶ m at the first frame, and the difference accumulates along each stream; we could not trace its origin.

The default p = 60 (Q2). No separate validation split was used: p = 60 was fixed during development, alongside the short-sequence budget sweep of Appendix A3, and kept unchanged across benchmarks, update rules, and horizons, so the experiments added afterwards (1,000-view 7-Scenes, ScanNet through 4,000 views, FILT3R, Mem3R, and all controls) test a pre-fixed value. Post hoc sensitivity (* = paired interval against p = 60 excludes zero):

Budget pTUM ATE7-Scenes Acc / CompFILT3R ScanNet ATE
none (TTSA3R; FILT3R)0.0951*0.0892* / 0.0292*0.2886*
300.0845*0.0725* / 0.02210.2529*
60 (default)0.07670.0625 / 0.02020.2372
1200.0830*0.0613 / 0.02720.2261

Every budget has lower mean error than the unmodified rule; p = 30 is worse than p = 60 on all three benchmarks (on 7-Scenes, in accuracy), and p = 120 is worse on TUM, with intervals against p = 60 that include zero on the other two. Section 4 states the provenance; Appendices A4 and A9 give the contrasts.

Scripts (Q4). The updated supplement contains the covariance-consistent FILT3R integration, the ScanNet 3,000-view runners and analyzers, the paired stream bootstrap (100,000 resamples, fixed seed), the Mem3R integration with its TUM and ScanNet runners, and the runners, protocols, and analyzers behind every new control and rerun (gate-ranked protection, the gate-overlap trace, uniform damping alone and with MeMix, the full Figure 3 rerun, and the Table 2 rerun on 7-Scenes at 500 views), with commands in the README; the 4,000-view scripts are included.

The 79.3% statistic (Q4). All eight TUM-dynamics streams (fr3 sitting/walking × halfsphere/rpy/static/xyz), first min(1000, available) frames, 7,023 frames in total. On 79.3% of these frames, the mean old-state compatibility of the 60 protected locations exceeds that of the 708 updated locations, with a positive mean difference in all eight streams. Appendix A4 now states the comparison and this cohort.

Per-prefix Sim(3). Section 4 now states this explicitly: per-prefix Sim(3) ATE measures trajectory-shape error at each horizon, not the consistency of one global metric scale over time.

Empirical score (W2). The claims rest on measurement and intervention, not on the analysis: the revision measures what the score selects and tests each element of the rule. The score is the agreement of a location's proposed new content with the current frame, and it splits exactly (Eq. 10) into the old content's agreement plus that of the proposed change. On 7,023 TUM frames, the 64 top-ranked locations exceed the 64 bottom-ranked by 2.12 within-frame SDs of old-state alignment (8/8 streams; 95% CI [2.01, 2.22]; Appendix A4), and a passive trace shows that the update rule would rewrite the 60 protected locations strongly (87th/85th native-gate percentile on TUM/ScanNet; Appendix A10). Each of the following matched interventions is worse than MeMix in TUM ATE with an interval excluding zero: reverse, random, periodic, old-state scoring, stale masks, and now the model's own gate ranking (the last also in 7-Scenes accuracy and 65-scene ScanNet ATE).

Thanks for the detailed response, I have no more questions. And I will preserve my positive rating.

Thank you for reading our response and for your support.

Reviewer_H7YzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses long-term forgetting and geometric drift in streaming 3D reconstruction models with fixed-capacity recurrent states. It proposes MeMix, a training-free state-update module based on the idea that not every state location should be updated at every frame. MeMix scores the compatibility between candidate state tokens and current image features, preserves the previous values at the 60 highest-scoring locations, and leaves the remaining 708 locations on the base model's native update path. The protected subset is reselected using the current observation, and preservation takes effect after the current-frame prediction has been read out. MeMix can be combined with the update rules of CUT3R, TTT3R, TTSA3R, and FILT3R without adding learnable parameters or retraining.

Critical strengths
  1. A simple method with practical value. MeMix operates directly on the state-update process of existing models, requires neither retraining nor additional learnable parameters, and maintains recurrent storage that is constant in sequence length. Table 6 reports unchanged peak GPU memory and small changes in inference throughput. This makes the method an attractive, low-cost improvement for existing streaming reconstruction systems.

  2. The core design choices are supported by substantial controlled experiments. Beyond comparing models with and without MeMix, the authors separately investigate protected-location selection, score source, mask reselection timing, and write strength at selected locations. Random, periodic, and reverse selection controls retain the same preservation budget and native update rule at the remaining locations, providing useful evidence for compatibility-guided selection. Comparisons against old-state scoring and update-displacement scoring further strengthen the interpretation of the results.

  3. Improvements extend across multiple state-update rules. The benefits are observed with direct CUT3R writes, soft-gated updates, and FILT3R's covariance-aware updates. The FILT3R experiment on 65 ScanNet scenes is particularly useful because it shows additional gains over an update mechanism that already provides substantial historical retention. The corresponding treatment of both state and covariance makes this integration reasonably complete.

Critical weaknesses
  1. The methodological novelty is primarily an incremental extension of existing state-update mechanisms. TTT3R already uses state-observation alignment confidence to regulate per-token update strength without training, while TTSA3R further combines temporal state changes and spatial observation information to control historical retention and the integration of new observations. MeMix follows this direction. Its main addition is to allocate a fixed number of exactly preserved locations at each frame by ranking compatibility between candidate states and current image features, on top of the native update rule. This design has practical value, but it retains the same pretrained representation, recurrent state, and basic update formulation. The methodological advance therefore lies primarily in write allocation rather than a new memory representation or recurrent architecture.

  2. The independent advantage of exact preservation over strong soft suppression is not yet established, and the novelty should be positioned more precisely. Table 5 shows that allowing 10% of the native write at selected locations produces slightly lower mean reconstruction errors on 7-Scenes than exact preservation, with the authors noting that the difference remains statistically unresolved. These results support greatly reducing updates at selected locations, but do not show that stopping those updates completely is necessary.

  3. Longer-run regressions are insufficiently quantified, limiting the assessment of long-term stability. In Figure 4, the relative ATE improvement on the same five ScanNet scenes decreases from 31.6% at 1,000 frames to 8.33% at 3,000 frames. Although improvements remain at all three evaluated horizons, these results do not directly establish benefits over still longer runs.

  4. The statistical reporting is incomplete. The paper describes a paired bootstrap procedure, but several comparisons are reported only through statements that the analysis supports an improvement or that a difference remains unresolved, without providing the corresponding numerical confidence intervals.

  5. Table 2 could be presented more concisely. All entries for VGGT and StreamVGGT in Table 2 are marked as out of memory (OOM). These repeated entries occupy substantial space without providing quantitative comparisons. The authors could remove these rows and simply state in the text.

Questions for authors
  1. What are MeMix's main methodological innovations relative to the existing training-free adaptive state updates in TTT3R and TTSA3R? Please clarify the new designs or insights contributed by candidate-state scoring, a fixed preservation budget, and per-frame reselection, and explain their relationship to existing gated updates.

  2. In Table 5, allowing 10% of the native write yields slightly lower mean errors on 7-Scenes than exact preservation. What are the paired differences and their 95% confidence intervals?

  3. Figure 4 shows a narrowing relative ATE improvement as sequence length increases. On which datasets, sequences, and horizons do the longer-run regressions mentioned in the paper occur? Please report the number of scenes, per-scene results, regression magnitudes, and frequency of regressions, and use these results to clarify the method's effective operating range and possible failure modes.

  4. Could the authors provide 95% confidence intervals for paired differences, per-sequence results, and the proportion of sequences that improve for the main comparisons and key ablations? These would make the stated levels of statistical support easier to assess. For the 3,000-frame experiment involving only five scenes, could the authors also report a leave-one-scene-out sensitivity analysis to determine whether the average gains are driven primarily by individual scenes?

7 comments

Q1 / W1 — What is new relative to TTT3R/TTSA3R, and the relationship to gated updates

Thank you for a detailed and constructive review. In short: MeMix adds a decision that TTT3R and TTSA3R do not make (which few locations receive no write at the current frame), and compatibility makes this choice better than TTSA3R's own gate ranking at the same budget in TUM ATE, 7-Scenes accuracy, and ScanNet ATE (all intervals excluding zero). Here: positioning and tested design choices (W1/Q1). Next: write-strength CIs (W2/Q2), every long-stream scene with frequencies (W3/Q3), and leave-one-scene-out (Q4). Last: the statistics appendix and Table 2 (W4/Q4, W5).

What is new. TTT3R and TTSA3R decide, for every location, how strongly to write (from state–observation alignment, or temporal and spatial cues); FILT3R sets a Kalman gain. MeMix adds a decision none of them makes: under a fixed budget, which few locations receive no write at the current frame, chosen by the compatibility of the post-observation candidate with the current view and renewed every frame. MeMix overrides the gate rather than reinforcing it: on TTSA3R, the locations it keeps sit at the 87th/85th percentile of the native gate (TUM/ScanNet), with about twice the mean gate of the updated locations, so MeMix withholds the write where the gate would write strongly; the stored content of the kept locations also agrees with the current view more than that of the updated locations (on 79.3% of TUM frames; measurements in our second reply to Rhmj). New controls show that this decision itself helps and that compatibility chooses best: protecting the 60 largest TTSA3R gates improves on TTSA3R on TUM, 7-Scenes, and ScanNet, protecting the 60 smallest does not, and compatibility selection beats both in ATE/accuracy on all three datasets (every interval excluding zero). The revised Related Work positions MeMix as this separate decision on top of existing update rules.

The three designs you ask about each contribute one thing. Candidate scoring: protection must anticipate the write, so the score includes the proposed change; old-state alignment alone or the size of the change alone is worse (TUM ATE 0.1119 / 0.0956 vs 0.0767). Fixed budget: a small fixed budget keeps 708/768 locations adaptive at every frame, whereas thresholding the gate at 0.5 freezes 750.9/768. Per-frame reselection: the useful set moves with the view (adjacent Jaccard 0.737; across the eight streams, 754/768 locations protected at some point), and fixed or stale masks are worse. Protocol: eight TUM streams at up to 1,000 views, ATE in m (MeMix 0.0767); Δ = alternative − MeMix, paired 95% bootstrap CI over streams (100,000 resamples); wins = MeMix better.

Design choiceAlternativeATE ↓Δ 95% CIWins
Candidate (post-observation) scoreOld-state score0.1119[0.0062, 0.0715]7/8
Protect the most compatibleProtect the least compatible0.1643[0.0291, 0.1493]8/8
Fixed small budget on native gatesThreshold gates at 0.5 (preserves 750.9/768; full writes at the rest)0.1086[0.0078, 0.0600]6/8
Reselect every frameFixed initial / one-step-stale mask0.0979 / 0.1025[0.0072, 0.0385] / [0.0026, 0.0639]7/8, 7/8
Compatibility allocationRandom / Periodic 600.0909 / 0.0868[0.0051, 0.0253] / [0.0015, 0.0223]8/8, 7/8
Compatibility, not gate rankingGate-low / Gate-high 60 (new)0.0955 / 0.0831[0.0048, 0.0350] / [0.0018, 0.0114]5/8, 7/8

Every interval excludes zero. The gate-ranked controls (60 smallest / largest final gates protected, native writes elsewhere) are also worse than MeMix on 7-Scenes and on 65 ScanNet scenes at 1,000 views: accuracy 0.1417 / 0.0717 vs 0.0625 m (16/16, 14/16) and ATE 0.508 / 0.332 vs 0.300 m (61/65, 47/65), all intervals excluding zero.

Relationship to existing gated updates. MeMix composes with these rules rather than replacing them. On top of TTSA3R it lowers 1,000-view 7-Scenes accuracy/completeness errors by 29.9%/30.7% (15/16, 14/16 streams); on top of FILT3R's Kalman gain, which already retains history well as you note, it lowers 1,000-view ScanNet ATE by 17.8% (54/65 scenes); and it lowers long-horizon ATE under CUT3R and TTT3R as well (Appendix A9). It also composes with uniformly damped gates (G = c·β), which make the base rule itself more conservative: adding MeMix lowers ScanNet ATE by 20–24% at each of six damping levels (c = 0.922 to 0.5). With Mem3R's learned channel-wise gate, it lowers 1,000-view ScanNet ATE by 10.5% (49/65 scenes, interval excluding zero; Section 4.2). Appendix A10 gives the damping tables; our first reply to Rhmj discusses them.

Q2 / W2 — Write-strength contrasts with CIs · Q3 / W3 + Q4 (LOSO) — Longer-run results

Write-strength contrasts (Q2). The contrasts support strong suppression at the selected locations, not exactness itself: the λ = 0.1 intervals against exact zero include zero on both datasets, while the full native write is worse on both. λ scales the native write at the 60 selected locations (0 = exact preservation, 1 = native write). Δ = λ-arm − (λ = 0), so negative favors the λ-arm; counts are streams where λ = 0 is better.

7-Scenes, 16 streams, 1,000 viewsAcc Δ [95% CI]Comp Δ [95% CI]
λ = 0.1 − λ = 0−0.0025 [−0.0070, 0.0017], 9/16−0.0001 [−0.0026, 0.0027], 6/16
λ = 1 − λ = 0+0.0267 [0.0154, 0.0392], 15/16+0.0090 [0.0038, 0.0164], 14/16
TUM, 8 streams, up to 1,000 viewsATE Δ [95% CI]
λ = 0.1 − λ = 0+0.0084 [−0.0009, 0.0210], 7/8
λ = 0.25 − λ = 0+0.0152 [−0.0000, 0.0355], 7/8
λ = 0.5 − λ = 0+0.0109 [0.0042, 0.0197], 8/8
λ = 1 − λ = 0+0.0184 [0.0065, 0.0319], 8/8

On TUM, every λ > 0 has higher mean ATE, and the per-stream slope of ATE on λ is positive on all eight streams. Exact zero is therefore the parameter-free default, not a unique optimum, and, as you suggest, the revision positions the contribution precisely as the choice of locations (Section 4.4; Table 5 caption). The TUM values are the rerun Table 5 column (second reply to 3zby).

Longer runs (Q3). Through 3,000 views MeMix lowers ATE on all five long ScanNet scenes at each evaluated horizon (15/15 cells). The regression in our Limitations lies beyond that, on the three scenes that extend to 4,000 views: scene 0747 is worse from 3,100 views on and turns the three-scene mean into a loss after 3,400 views. Protocol. TTSA3R vs TTSA3R+MeMix on consecutive ScanNet frames; per-prefix Sim(3) ATE; equal scene weights; paired 95% bootstrap over scenes. The five scenes are all ScanNet evaluation scenes with ≥3,000 consecutive frames and finite ground truth over the whole sequence; the three with ≥4,000 such frames form a nested subset, not an outcome-based selection.

ATE change of MeMix vs TTSA3R (%; negative favors MeMix; mean rows: change of the mean ATE; new Table 7):

Scene1,0002,0003,0003,4003,5003,7004,000
0739−8.1−24.8−27.4−13.7−11.0−7.6+0.1
0747−37.8−24.5−0.1+17.8+30.1+44.4+29.3
0757−25.6−12.4−16.1−21.5−21.9−20.7−18.2
0766−31.7−2.7−3.5————
0785−39.6−15.4−12.0————
Mean ATE, five scenes−31.6−13.5−8.3————
Mean ATE, four scenes (any one left out)−29.3 to −34.0−9.8 to −18.9−6.6 to −10.9————
Mean ATE, three scenes−28.4−19.9−10.3−2.7+3.0+9.4+4.8

Beyond 3,000 views (100-view grid), the mean loss peaks at 3,700 views, and MeMix has lower ATE on two of three scenes through 3,900 views. Through 3,000 views the relative gain narrows faster than the absolute one (five-scene mean reduction 0.118, 0.101, and 0.073 m) because TTSA3R's own mean ATE grows from 0.374 to 0.873 m.

Leave-one-scene-out (Q4). No single scene drives the gain through 3,000 views: every four-scene mean remains a reduction (smallest 6.6%; row above; Appendix A5), and the five-scene intervals exclude zero at all three horizons (table in our next comment).

Frequency.

Setting (TTSA3R unless noted)MeMix better / total
TUM, up to 1,000 views8/8 streams
7-Scenes, 1,000 views15/16 (accuracy), 14/16 (completeness)
ScanNet, 65 scenes, 1,000 views54/65; also 54/65 with FILT3R
ScanNet, five scenes, through 3,000 views15/15 cells, one by 0.1%
ScanNet, three scenes, 3,100–4,000 views19/30 cells; the other 11 are scene 0747 (10) and 0739 at 4,000 views (+0.1%)

Operating range and failure modes. Supported range: 150–1,000 views for all three update rules on TUM and ScanNet, where MeMix has the lower mean ATE (at 50–100 views it is up to 1.8% higher; Figure 3 rerun), and through 3,000 views with TTSA3R on all five long scenes. Beyond that, on three scenes, two failures are observed: scene 0747's ATE loss, and a late rise in rotational RPE that releasing the protection after 3,000 writes largely removes, while the ATE loss remains (third reply to Rhmj). Weaker suppression from the start (λ = 0.1) keeps the three-scene mean ATE below TTSA3R at every evaluated horizon (−4.7% at 4,000; −6.6% vs exact zero's −10.3% at 3,000). With three scenes, the cause is not identified (Appendix A5).

Q4 / W4 — Paired statistics and the Figure 3 / Table 2 reruns; W5 — Table 2 layout

Statistics. In new Appendix A9, the comparisons of MeMix with unmodified TTSA3R (TUM, 7-Scenes, five ScanNet scenes through 3,000 views) and with unmodified FILT3R on 65 ScanNet scenes all have paired 95% intervals excluding zero (Tables 10–11), as do all six Figure 3 rerun comparisons (Table 13). For these and for the controls in Section 4.4 and the budget controls in Appendix A4, Tables 10–11 list the reference and MeMix means, the paired mean difference Δ = reference − MeMix (positive favors MeMix), the 95% percentile-bootstrap CI over streams or scenes (100,000 resamples), and wins/n where recorded; Section 4 now gives paired 95% CIs inline for the 7-Scenes, FILT3R, and Mem3R comparisons and for λ = 0.1. The main comparisons and three of the controls:

ComparisonΔ [95% CI]Wins
TUM, TTSA3R vs +MeMix, ATE (m)0.0184 [0.0065, 0.0319]8/8
7-Scenes 1k, TTSA3R vs +MeMix, Acc0.0267 [0.0154, 0.0392]15/16
7-Scenes 1k, TTSA3R vs +MeMix, Comp0.0090 [0.0038, 0.0164]14/16
ScanNet (5 scenes), ATE at 1k / 2k / 3k0.1180 [0.0604, 0.1653] / 0.1014 [0.0542, 0.1717] / 0.0727 [0.0301, 0.1072]5/5 each
FILT3R ScanNet 1k (65 scenes), ATE0.0514 [0.0268, 0.0818]54/65
7-Scenes 1k, Random 60, Acc / Comp0.0175 [0.0078, 0.0284] / 0.0055 [0.0009, 0.0114]12/16, 9/16
TUM, old-state score, ATE0.0352 [0.0062, 0.0715]7/8
TUM, fixed initial mask, ATE0.0212 [0.0072, 0.0385]7/8

Intervals that include zero are reported as such, e.g., 7-Scenes against the literal 0.5 threshold, p = 120, and λ = 0.1, and TUM against the rank window 473–532.

Per-sequence results: Appendix A5 (Table 7) lists every long ScanNet scene; the eight TUM streams at their final cap are below (ATE, m; TTSA3R → MeMix); and the updated supplement adds per-stream and per-scene values behind the paired-statistics rows of Appendices A9 and A10 as CSV files (coverage is listed in its README).

TUM streamhalfsphererpystaticxyz
sitting0.1586 → 0.11630.0579 → 0.05550.0260 → 0.02070.0590 → 0.0448
walking0.1427 → 0.11710.1506 → 0.14620.0195 → 0.01890.1464 → 0.0942

Figure 3 and Table 2 reruns. Per-stream records of the submitted Figure 3 and Table 2 runs were not kept, so to report paired per-stream statistics we reran Figure 3 in full and one Table 2 setting with the released code and checkpoint, recording every stream (Appendix A9, Tables 12–13; code in the updated supplement). The submitted values are kept and the reruns are reported beside them; the Figure 3 rerun's average reductions are larger on both datasets.

Figure 3 rerun. All six intervals exclude zero, as do the six at 1,000 views (Table 13). Δ = base rule − base rule with MeMix in mean ATE over 400–1,000 views (m), with 95% CI and wins:

CUT3RTTT3RTTSA3RAverage reduction
TUM, 8 streams0.0335 [0.0050, 0.0663], 6/80.0126 [0.0024, 0.0294], 7/80.0141 [0.0041, 0.0261], 7/818.7% (13.2% in the submitted figure)
ScanNet, 65 scenes0.0484 [0.0314, 0.0659], 50/650.0484 [0.0295, 0.0693], 54/650.0586 [0.0439, 0.0754], 58/6515.5% (10.0% in the submitted figure)

The rerun reproduces exactly the TUM TTSA3R and MeMix values used elsewhere in the paper (0.0951 and 0.0767 m at 1,000 views); it does not reproduce the submitted Figure 3 curves, and the ScanNet scene set of the submitted runs was not recorded.

Table 2 rerun (7-Scenes-D, 500 views, 18 streams; TTSA3R and TTT3R; Δ = base rule − base rule with MeMix):

Base ruleAcc Δ [95% CI], winsComp Δ [95% CI], wins
TTSA3R0.0107 [0.0053, 0.0173], 17/180.0018 [0.0007, 0.0030], 13/18
TTT3R0.0046 [−0.0009, 0.0113], 10/18−0.0003 [−0.0036, 0.0036], 9/18

Every rerun mean and median of these four variants is within 0.0034 m of Table 2 (0.0038 in normal consistency). TTSA3R's gains reproduce (0.0467/0.0258 → 0.0360/0.0240 m), with both intervals excluding zero. With TTT3R, the accuracy gain is smaller than in Table 2, and both intervals include zero at this setting.

Table 2. We removed the six VGGT/StreamVGGT rows; the caption and text state that both run out of memory at all three lengths under this protocol.

Reviewer_RhmjOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes MeMix, a training-free method to improve recurrent state updates for streaming 3D reconstruction. At each frame, it scores candidate state tokens against a pooled image feature, preserves the previous values at the highest-scoring locations, and applies the original update rule to the remaining locations. The method requires no training and can be added to several existing state-update rules. MeMix is evaluated with CUT3R, TTT3R, TTSA3R, and FILT3R, and shows improvements over them in long-sequence reconstruction and pose estimation. The paper also studies the effects of selection criteria, update strength, and reselection frequency.

Critical strengths
  1. The method is simple. MeMix doesn't require retraining nor additional learned parameters. Its composition with both soft-gated and covariance-aware updates makes it a practical addition to existing systems.
  2. MeMix achieves meaningful improvements over baselines.
  3. The paper provides extensive controlled analysis. The comparisons against random, periodic, reverse-ranked, fixed, and stale protection provide useful evidence that both allocation and current-frame reselection matter.
Critical weaknesses
  1. My main concern is the limited conceptual novelty relative to existing adaptive state-update methods. MeMix adds a hard preservation mask on top of those updates, but the explanation for its particular selection rule remains largely empirical. In particular, it is not obvious why a candidate token that aligns strongly with the current observation indicates that its previous value should be preserved. The analysis describes the consequences of freezing a token, but does not fully explain this choice.
  2. I am also less convinced that exact preservation is the essential ingredient. The write-strength ablation suggests that allowing a small update can work similarly well. This does not undermine the method’s usefulness, but it makes the distinction from existing soft-gating approaches less clear.
  3. One useful comparison is missing: selecting tokens to preserve using the existing model’s gate values while leaving the other updates unchanged. The current hard-gate baseline also changes the update strength elsewhere, so it does not isolate the benefit of the proposed scoring rule.
  4. Finally, the longer-sequence results deserve more discussion. The benefit becomes smaller as sequences grow, and the paper mentions regressions in exploratory longer runs without examining them in detail. Understanding these failures would help establish when preservation becomes harmful, especially when old state values become stale.
Questions for authors
  1. What does MeMix add beyond making existing gates more conservative? The hard-gate baseline also changes the updates to unprotected tokens, while the write-strength ablation suggests that exact preservation is not essential. Could the authors provide a comparison that isolates the benefit of their selection rule from simply reducing update strength?

  2. How substantial are the longer-stream failures acknowledged in the limitations? Since long-horizon reconstruction is the central motivation, please report these results and explain the conditions under which MeMix becomes worse than the baseline. Do they indicate that preservation delays drift but eventually compromises adaptation?

5 comments

Q1 / W3 — What MeMix adds beyond a more conservative gate

Thank you for a precise review. MeMix adds a decision that a more conservative gate does not make: which locations keep their old value. In the control you asked for (same 60-location budget, native writes elsewhere), ranking by the model's own gate is worse than MeMix in ATE/accuracy on TUM, 7-Scenes, and 65 ScanNet scenes, even for the 60 largest gates; and MeMix lowers ScanNet ATE on top of each of six uniformly damped gates (all intervals excluding zero). Three comments: Q1/W3 here, then W1/W2, then Q2/W4.

Protocol. Shared checkpoint, TTSA3R gate, inputs, and evaluators; TUM: 8 streams, up to 1,000 views; 7-Scenes: 16 streams and ScanNet: 65 scenes, 1,000 views; errors in m; paired 95% bootstrap over streams/scenes.

(i) Same budget, native writes elsewhere: only the choice of the 60 protected locations differs. Gate-low protects the 60 smallest final gates (Rank-hard's set); Gate-high protects the 60 largest, the set that withholds the most gate mass.

60 protected locationsTUM ATE7-Scenes AccScanNet ATE
None (TTSA3R)0.09510.08920.386
Least compatible0.16430.1824—
Smallest gates (Gate-low, new)0.0955 (+0.0188 [0.0048, 0.0350])0.1417 (+0.0791 [0.0450, 0.1212])0.508 (+0.208 [0.163, 0.256])
Random0.09090.0800—
Periodic0.08680.0759—
Largest gates (Gate-high, new)0.0831 (+0.0064 [0.0018, 0.0114])0.0717 (+0.0092 [0.0034, 0.0176])0.332 (+0.032 [0.016, 0.049])
Most compatible (MeMix)0.07670.06250.300

Every entry differs from MeMix with an interval excluding zero; the new rows also give control − MeMix [95% interval].

  • With native writes elsewhere, as you proposed, Rank-hard's set (Gate-low) gives TUM ATE 0.0955 instead of Rank-hard's 0.1641: level with TTSA3R on TUM and above it in 7-Scenes accuracy error and ScanNet ATE.
  • Protecting the gate's largest values improves on TTSA3R on all three datasets (intervals excluding zero); compatibility selection lowers ATE/accuracy error a further 8%, 13%, and 10% (TUM, 7-Scenes, ScanNet; intervals excluding zero). The locations worth keeping thus lie among those the gate writes hardest, and candidate compatibility picks them better than the gate does.

(ii) Uniformly more conservative gates (Appendix A10). Every gate is scaled, G = c·β (c = 1 is TTSA3R); with MeMix added, its 60 locations stay exact and the other 708 write c·β.

cScanNet ATE: damped → +MeMix7-Scenes AccTUM ATE
10.386 → 0.300* (54/65)0.0892 → 0.0625* (15/16)0.0951 → 0.0767* (8/8)
0.9220.364 → 0.280* (59/65)—0.0870 → 0.0792* (7/8)
0.8610.348 → 0.265* (60/65)0.0746 → —0.0830 → 0.0770 (7/8)
0.750.311 → 0.250* (56/65)—0.0809 → 0.0755* (7/8)
0.70.302 → 0.230* (56/65)—0.0821 → 0.0732* (7/8)
0.60.269 → 0.217* (51/65)0.0447 → 0.0331* (16/16)0.0739 → 0.0727 (5/8)
0.50.234 → 0.187* (60/65)0.0334 → 0.0282* (15/16)0.0695 → 0.0692 (3/8)

* interval excludes zero; in parentheses, streams or scenes on which adding MeMix is better; — not run.

  • At the same withheld gate mass, selection gives about twice the gain of uniform damping (ScanNet ATE, 7-Scenes accuracy). c = 0.861 removes as much gate mass as MeMix withholds (measured on TUM); damping alone is then worse than MeMix alone: ScanNet ATE 0.348 vs 0.300, 7-Scenes accuracy 0.0746 vs 0.0625 (intervals excluding zero); on TUM the interval includes zero (0.0830 vs 0.0767).
  • MeMix adds on top of the damped rules. It lowers the damped rule's ScanNet ATE by 20–24% at all six levels and 7-Scenes accuracy error by 26% and 16% at the two levels where it was added (intervals excluding zero), including c ≤ 0.6, where damping alone already has lower error than MeMix alone on ScanNet and 7-Scenes (on TUM the intervals include zero). Over this range a stronger base rule and MeMix are complements, not alternatives, just as TTT3R and TTSA3R alone have lower ATE than CUT3R with MeMix. On TUM, the ATE gain from adding MeMix has an interval excluding zero at three of the six levels.
  • Damping has a cost on dynamic streams. On TUM, damping alone has higher translational and rotational RPE than MeMix alone at five of the six levels, and at c = 0.5 its translational RPE is 10% worse than unmodified TTSA3R's (intervals excluding zero); adding MeMix to the damped rule does not remove this cost (Table 16).

Takeaway. How strongly to write and which few locations keep their old value are separate decisions that compose. Table 3 and Section 4.4 include the gate-ranked controls; Appendix A10 reports every metric.

W1 + W2 — Why keep the most compatible locations, and is exactness essential?

In short: MeMix keeps locations whose stored content agrees with the current view more than the rest, yet which the native rule is about to rewrite strongly, and every alternative in the first table below is worse in TUM ATE (intervals excluding zero). This selection, not exactness, is what separates MeMix from soft gating.

W1. Why high candidate compatibility should trigger preservation. The score decomposes exactly (Eq. 10) into two terms: the old state's alignment with the current-frame feature, and the alignment of the proposed change (candidate − old state) with that feature. Measured on MeMix's own runs, the 60 top-scoring locations have two properties:

  1. Their stored content agrees with the current view more than the rest. On 79.3% of the 7,023 TUM frames, their mean old-state alignment exceeds that of the 708 updated locations (positive mean in 8/8 streams); over the full ranking, the top 64 ranks exceed the bottom 64 by 2.12 within-frame SDs (8/8; CI [2.01, 2.22]; new in Appendix A4).
  2. The native rule is about to rewrite them hard. They sit at the 87th/85th percentile of TTSA3R's own gate on TUM/ScanNet (new in Appendix A10).

Neither property alone picks this set: its Jaccard overlap is 0.125 with the 60 locations of highest old-state alignment and 0.29 with the 60 largest gates (random 0.04), and protecting either of those sets instead is worse (TUM ATE 0.1119 and 0.0831 vs 0.0767; intervals excluding zero); the first even has a higher mean ATE than TTSA3R without protection (0.0951). Our interpretation: where both hold, a strong write trades content accumulated over many frames for one frame's proposal, frame after frame, which can feed long-horizon drift. MeMix carries those locations forward unchanged and leaves the other 708 to absorb what is new. Each element of the rule is tested by an intervention (eight TUM streams, up to 1,000 views; ATE in m, MeMix 0.0767; Δ = alternative − MeMix; paired 95% bootstrap over streams, 100,000 resamples):

What is testedAlternativeTUM ATEΔ 95% CIMeMix better
DirectionProtect the 60 least compatible0.1643[0.0291, 0.1493]8/8
Post-observation candidateRank by old-state alignment0.1119[0.0062, 0.0715]7/8
Rank by size of the proposed change0.0956[0.0070, 0.0325]7/8
Set follows the current frameFixed initial mask0.0979[0.0072, 0.0385]7/8
One-step-stale mask0.1025[0.0026, 0.0639]7/8
Reselect every 16 / 64 writes0.1097 / 0.1444[0.0083, 0.0707] / [0.0150, 0.1307]8/8, 7/8

Every interval excludes zero. The least-compatible set is also 2.9×/3.2× worse in 7-Scenes accuracy/completeness; old-state ranking is also worse under direct CUT3R writes (0.1815 vs 0.1363, 8/8, CI [0.0137, 0.0807]); and within the gate's own ranking the direction is the same: protecting its largest values helps and protecting its smallest does not (first reply).

W2. Exactness. Selection, not exactness, is what separates MeMix from soft gating: a soft gate scales each location by its own signal, and ranking by that signal picks a different, worse set (Gate-low and Gate-high in our first reply). What the selected locations need is strong suppression, and exact zero is its parameter-free default (an exact identity carry), not a unique optimum, as Section 4.4 and the Table 5 caption now state. With the selection rule fixed, λ scales the native write at the selected locations (7-Scenes: 16 streams, 1,000 views; Δ = λ-arm − exact zero; positive favors exact zero):

λ7-Scenes Acc Δ [95% CI]TUM ATE Δ [95% CI]
0.1−0.0025 [−0.0070, 0.0017]+0.0084 [−0.0009, 0.0210]
0.25—+0.0152 [−0.0000, 0.0355]
0.5—+0.0109 [0.0042, 0.0197]
1 (native write)+0.0267 [0.0154, 0.0392]+0.0184 [0.0065, 0.0319]

— not run. Restoring the full native write is clearly worse on both datasets, while no tested λ > 0 beats exact zero with an interval excluding zero.

Q2 / W4 — Longer streams: how far the gain holds, and where it reverses

Through 3,000 views, three times our 1,000-view benchmark horizon, MeMix lowers ATE on all five ScanNet scenes of that length whose ground truth is finite over the whole sequence, at every evaluated horizon, with paired intervals excluding zero, and no single scene carries the gain. Beyond 3,400 views, on the three scenes that extend to 4,000 views, the mean benefit reverses, mainly because of one scene. Appendix A5 (new Table 7) reports every scene. Protocol. TTSA3R vs TTSA3R+MeMix on natural consecutive ScanNet frames; ATE in m after per-prefix Sim(3); equal scene weights; paired 95% bootstrap over scenes (100,000 resamples).

Through 3,000 views (all five ScanNet evaluation scenes with ≥3,000 consecutive frames and finite ground truth over the whole sequence). MeMix lowers ATE in all 15 scene–horizon cells:

ViewsMean ATE reductionTTSA3R − MeMix, 95% CI (m)Leaving out any one scene
1,00031.6%[0.0604, 0.1653]29.3–34.0%
2,00013.5%[0.0542, 0.1717]9.8–18.9%
3,0008.33%[0.0301, 0.1072]6.6–10.9%

The relative reduction narrows faster than the absolute one: the mean ATE reduction is 0.118, 0.101, and 0.073 m, while TTSA3R's own mean ATE grows from 0.374 to 0.873 m.

Beyond 3,000 views (the three of these scenes with ≥4,000 frames: a nested subset, not selected by outcome; evaluated every 100 views). Scenes 0739 and 0757 keep a lower ATE with MeMix through 3,900 views. Scene 0747, tied at 3,000 views (−0.1%), is worse from 3,100 views on (by up to 44.4%) and turns the three-scene mean into a loss after 3,400 views: the change of the mean ATE peaks at +9.4% at 3,700 views and is +4.8% at 4,000 views. Per-scene values are in our second reply to H7Yz and in Appendix A5 (Table 7).

Conditions. Among the evaluated horizons (1,000 views on), MeMix has higher ATE than TTSA3R on these streams only beyond 3,000 views; all of them are static scenes. Dynamic content has not produced a late loss within the horizons tested: on TUM's dynamic sitting/walking streams (up to 1,000 views) MeMix has lower ATE than TTSA3R on all eight at 800, 900, and 1,000 views. The protected set is not fixed either: it is reselected at every frame while 708 of 768 locations take the native write (across the eight TUM streams, 754/768 locations are protected at some point; a fixed mask is worse than reselection, ATE 0.0979 vs 0.0767).

Does preservation delay drift but eventually compromise adaptation? In ATE, one of the three scenes shows that pattern (scene 0747, above); scene 0757 does not (18–22% lower with MeMix at every horizon beyond 3,000 views), and scene 0739's gain narrows from 27.4% at 3,000 views to a tie at 4,000. Two follow-up runs on the same scenes probe whether compromised adaptation is the reason:

  • Restoring native writes late does not recover the ATE benefit. Releasing the protection after 3,000 writes (native writes at the selected locations from then on) leaves the three-scene mean ATE at 4,000 views 0.0714 m higher than with continued protection (higher on 2 of 3 scenes). The same release removes most of a late rise in rotational RPE (three-scene mean at 4,000 views: 1.99° with continued protection, 0.78° after the release, 0.65° for TTSA3R).
  • Weaker suppression from the start avoids the reversal of the mean. With λ = 0.1 at the selected locations, the three-scene mean ATE stays below TTSA3R at every evaluated horizon (−4.7% at 4,000 views), with a smaller gain at 3,000 views (−6.6% vs exact zero's −10.3%).

Releasing late protection thus removes most of the rotational-RPE rise but not the ATE reversal, while weaker suppression over the whole stream keeps the mean ATE below TTSA3R; with three scenes, the cause is not identified (Appendix A5). The revised Limitations state this boundary.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of new evidence and revisions

We thank all reviewers for careful, concrete reviews. Three questions recur across them; each is answered with new matched evidence and a targeted revision (changes are marked in blue in the revised PDF).

1. Is MeMix more than a more conservative gate? (Rhmj, H7Yz) Yes. Training-free update rules (TTT3R, TTSA3R, FILT3R) set how strongly each state location is written. MeMix decides which few locations receive no write at the current frame, and can be added to any of them. Two new experiments on TUM, 7-Scenes, and 65 ScanNet scenes show that this second decision matters:

  • The control Rhmj requested: the same 60-location budget, ranked by the model's own gate, with native writes elsewhere. MeMix is better than both gate rankings in ATE/accuracy on all three datasets, every interval excluding zero (ScanNet ATE: TTSA3R 0.386, largest gates 0.332, MeMix 0.300). Protecting the 60 largest gates improves on TTSA3R and protecting the 60 smallest does not: the locations worth keeping lie among those the gate writes hardest, and candidate compatibility finds them better than the gate's own ranking.
  • Uniformly more conservative gates (every gate scaled by c ≤ 1). MeMix adds on top of them: it lowers ScanNet ATE by 20–24% at each of six damping levels (c = 0.922 to 0.5) and 7-Scenes accuracy error by 26% and 16% at the two levels where it was added. Our first reply to Rhmj and Appendix A10 give the tables, including damping alone versus MeMix alone.

2. Does the gain come from selection or from exactness? (all) From selection. With exact zeros at a fixed budget, the choice of locations decides the outcome (point 1), whereas at the selected locations the paired intervals of a 10% write against exact zero include zero. What those locations need is strong suppression (restoring the native write there is clearly worse), and exact zero is its parameter-free default. Section 4.4 and the Table 5 caption now state the claim this way, and every write-strength contrast has a paired 95% CI. Table 5's TUM column is replaced by a rerun on the released code path, which matches Table 3 and resolves the mismatch 3zby noted.

3. What happens on longer streams? (all) Through 3,000 views, three times the 1,000-view benchmark horizon, MeMix lowers ATE on all five ScanNet scenes of that length whose ground truth is finite over the whole sequence (15/15 scene–horizon cells; the five-scene interval excludes zero at each horizon), and no single scene carries the gain. Beyond 3,400 views, on the three scenes that extend to 4,000 views, the mean ATE is higher with MeMix (by 9.4% at the 3,700-view peak and 4.8% at 4,000 views), mainly because of one scene; the other two keep a lower ATE through 3,900 views. Appendix A5 (Table 7) reports every scene.

Beyond the shared checkpoint (3zby). We applied MeMix unchanged to Mem3R, which is initialized from CUT3R but separately trained with a learned channel-wise state gate and a fast-weight pose memory (code released after our submission). On 65 ScanNet scenes at 1,000 views, MeMix lowers Mem3R's ATE from 0.685 to 0.613 m (−10.5%; better on 49/65 scenes; paired 95% CI [0.041, 0.107] m, excluding zero; Section 4.2). On the eight TUM streams it lowers ATE from 0.1304 to 0.1208 m (−7.4%), where the interval includes zero (Appendix A11).

Other revisions.

  • Statistics (Appendix A9): paired differences, 95% CIs, and win counts for the main long-horizon comparisons and ablations; a complete Figure 3 rerun (every interval excludes zero); a rerun of the 7-Scenes 500-view Table 2 setting (TTT3R and TTSA3R).
  • Paper: Table 3 and Section 4.4 add the gate-ranked controls; Table 2 drops the out-of-memory rows; Section 4 states the provenance of p = 60 (no validation split; fixed during development and never changed) and that per-prefix Sim(3) ATE measures trajectory shape, not global scale consistency; Appendix A4 defines the 79.3% statistic.
  • Supplement: runners, protocols, and analyzers for the FILT3R integration, the long ScanNet runs, the paired bootstrap, and every new control and rerun.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.