acceptodds
Under review as a conference paper at ICLR 2027

Validation over Contexts: Robust Long-Context Inference for 3D Reconstruction Models

Abstract

Feed-forward 3D reconstruction recovers cameras and scene geometry from un-ordered image collections in a single pass but its reliability deteriorates once the input is no longer curated. Real captures, internet albums and retrieval results routinely contain distractor views from other scenes, near-duplicates and visually similar but geometrically incompatible images, which silently corrupt cross-view geometry reasoning. Cleaning such inputs is non-trivial at scale. Jointly examining all images exceeds practical memory budgets while judging each image from a single bounded window is brittle. A valid view can appear unsupported simply because its overlapping neighbors are not in that window. An effective input-side filter must therefore be memory-bounded, robust to distractors and careful to retain the clean views that downstream reconstruction depends on. We resolve this trade-off by repeatedly validating each image against a small set of scene-consistent anchors within bounded, memory-feasible subsets of the collection. We retain an image if any anchor in any subset provides convincing support. We realize this as Multi-Anchor Context Validation (MACV), a training-free front-end that requires no retraining of the reconstruction backbone. Across multiple datasets and backbones, MACV consistently improves filtering quality and downstream camera pose, depth and point-cloud reconstruction, and remains feasible on long noisy collections where prior single-context robust filtering is not. Codes will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D reconstruction, neural geometry, view pruning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_bee5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the sensitivity of feed-forward 3D reconstruction models to uncurated image collections containing irrelevant distractors. Existing test-time filtering methods such as Robust-VGGT operate within a single bounded context, running out of memory on long collections or over-pruning valid views when restricted to a local window. To overcome this, the authors propose Multi-Anchor Context Validation (MACV), a training-free input-side filtering front-end.

The framework of MACV is built on following three steps:

  • Global anchor selection: Identifies and verifies seed-consistent reference anchors across the scene using DINOv2 feature retrieval and a seeded farthest-point procedure.
  • Bounded context partitioning: Covers the collection with overlapping context windows and constructs anchor-local validation pools bounded by a strict capacity limit (Pmax⁡=64P_{\max} = 64).
  • Best-support validation: Evaluates candidate views using internal backbone features and anchor-centric attention, following Robust-VGGT, and aggregates support across evaluated contexts using a max-score decision rule.

The retained subset is forwarded to an e2e 3D reconstruction backbone (e.g., VGGT, π3\pi^3, MapAnything). Experiments across six benchmarks show that MACV preserves more valid views than single-context filtering baselines and scales to collections of up to 1,200 images.

Critical strengths

I found two main strengths in this work:

First, the conceptual motivation to frame input filtering as a "preservation-aware" problem is valuable and well-argued. In 3D reconstruction, over-pruning valid views destroys scene coverage and can be just as catastrophic as admitting distractors. Addressing the memory-robustness-coverage trade-off through repeated bounded validations is a sensible objective.

Second, the paper provides extensive empirical characterization across three modern e2e 3D reconstruction backbones (VGGT, π3\pi^3, MapAnything) and six datasets. The ablation and diagnostic analyses in the appendix (e.g., Tables 12–16) are informative as well.

Critical weaknesses

My main concerns are the practical trade-offs and whether the comparisons sufficiently isolate the source of the reported gains:

First, the practical trade-off against no filtering needs clearer discussion. MACV improves over Robust-X in the reported averages, but this does not always translate into a benefit over using the unfiltered backbone. For example, Table 5 reports VGGT AUC@20 of 84.1 and 81.6 on the clean and small-noise inputs, compared with MACV’s preservation-aware scores of 53.0 and 76.9. Table 1 also shows that MACV removes approximately 22–28% of valid views on clean inputs, averaged across datasets. These are coverage-adjusted performance losses, not necessarily worse estimates on the retained views. The gains in noisier and longer collections remain valuable, but the paper should more clearly discuss when filtering is beneficial and when its coverage cost outweighs its benefit.

Second, the thousand-image comparison does not clearly separate target-reference availability from multi-anchor validation. Appendix E attributes the poor performance of Chunked Robust-VGGT partly to the absence of a verified target-scene reference in its chunks. Since the task already provides a target seed, a natural control is to use the same seed as the reference in every chunk. Table 3 includes a single-anchor ablation on 7-Scenes, but a corresponding seed-controlled comparison on ArtSci would make the large-scale result more convincing. The pool-size limit should be kept comparable, and the number of validation passes should be reported.

Third, the operating-point dependence of best-support aggregation needs further characterization. A max rule gives a candidate more opportunities to exceed the fixed threshold as additional anchor–context evaluations are added. This is a potential false-acceptance risk, rather than evidence that MACV necessarily admits more distractors. In fact, Table 3 reports the same DRR for Max, Mean, and Majority Vote, with higher CKR for Max at the tested setting. (Tables 16–17 also provide useful threshold analyses.) What remains unclear is whether Max retains its advantage over the other aggregation rules at matched CKR, particularly as the number of evaluations increases or the distractors become more visually ambiguous.

Questions for authors

Could the authors address the following points during the rebuttal?

  1. Characterization of Operational Regimes vs. No-Filtering:

    Could you characterize the tested regimes in which MACV improves over no filtering, separated by dataset, noise level, and collection length? Please distinguish cases where direct inference remains feasible from cases where filtering is needed to meet the memory budget.

  2. Controlled Chunked Baseline with Global Seed:

    In Table 11, does Chunked Robust-VGGT use the global target seed as its reference in every chunk? If not, could you evaluate this variant under a comparable pool-size limit, counting the seed within that limit, and report the validation-pass count and runtime?

  3. Trade-off Analysis of Best-Support (Max) Aggregation:

    Could you provide precision-recall curves comparing Max Support against Mean Support and Majority Voting (Table 3), or report DRR at matched CKR? How does the false-positive behavior of the max rule hold up in scenes with visually ambiguous distractors (e.g., the repetitive rooms in Figure 15)?

I am open to revising my rating if the authors address the concerns raised above.

2 comments
AuthorsOpenReview ↗

Response to Reviewer bee5 — Part 1

We thank the reviewer for valuing the preservation-aware framing and the evaluation across three backbones and six datasets. We reply to each point below and are open to further discussion.

Q1. Could you characterize the tested regimes in which MACV improves over no filtering, separated by dataset, noise level, and collection length? Please distinguish cases where direct inference remains feasible from cases where filtering is needed to meet the memory budget.

Based on the preservation-aware scores, the noise level (distractor ratio) matters more than the dataset or the collection length. On clean inputs, no filtering is better. MACV removes 22% to 28% of the clean views (Tab. 1), and the lost coverage outweighs the accuracy gained on the retained views. At Small noise (25% distractors), no filtering is as good or slightly better on average. At Medium and Large noise (50% and 62.5% distractors), MACV is better on all four datasets, with VGGT and with π³. On 7-Scenes (50% distractors), it is also better at every tested length from 98 to 398 images.

These conclusions rest on a comparison that uses the same backbone for filtering and reconstruction. The filtered rows of Tab. 5 reconstruct the retained views with π³, so its comparison of 84.1 / 81.6 with 53.0 / 76.9 also changes the backbone. The table below gives the preservation-aware pose AUC@20° as unfiltered → MACV, with bold marking the cells where MACV is better.

DatasetBackboneCleanSmallMediumLarge
ETH3DVGGT72.7 → 40.168.2 → 64.061.4 → 71.156.6 → 71.4
On-the-GoVGGT92.1 → 65.691.3 → 91.588.7 → 92.186.0 → 92.1
PhototourismVGGT85.6 → 65.983.0 → 85.580.6 → 85.579.7 → 84.6
ScanNet++VGGT85.9 → 51.183.8 → 81.779.5 → 83.275.6 → 82.5
AverageVGGT84.1 → 55.781.6 → 80.777.6 → 83.074.5 → 82.7
Averageπ³80.9 → 49.878.0 → 70.166.4 → 75.055.3 → 72.4

With π³, MACV is also better on each dataset at Medium and Large noise. On the views it retains, MACV-VGGT is more accurate than unfiltered VGGT at every level (87.5 vs. 84.1 on clean inputs), so the clean-input deficit is a loss of coverage. For depth, distractors have little effect and filtering does not pay off.

For the collection length, we compare unfiltered π³ → MACV-π³ on 7-Scenes sequences, with both reconstructed by π³.

N (images)CD ↓F ↑ATE ↓RTE ↓
50 (98)0.164 → 0.0520.563 → 0.6520.047 → 0.0430.032 → 0.026
100 (196)0.206 → 0.0750.567 → 0.6490.050 → 0.0380.023 → 0.017
150 (300)0.191 → 0.0730.571 → 0.6830.049 → 0.0370.018 → 0.013
200 (398)0.157 → 0.0850.592 → 0.6830.049 → 0.0380.015 → 0.011

MACV-π³ is better on every metric at every length. CD and F-score are computed on the whole output set, namely all retained views including retained distractors, against a ground truth built from all clean frames. They therefore penalize both removed clean views and kept distractors in a more systematic way. Unlike the pose errors, which are computed on the retained clean frames (82% to 100% of them), they need no preservation-aware correction. This geometry evaluation therefore supports the conclusions above without any correction.

Direct inference remains feasible in every setting above, since unfiltered inference fits in 64 GB. MACV's gains over no filtering are therefore accuracy gains, not a way around a memory limit. We also ran unfiltered VGGT and π³ on 698 images without running out of memory. The memory limit applies to full-context robust filtering instead. The released Robust-VGGT runs out of memory beyond about 170 images. MACV peaks at 15 to 27 GiB up to 400 images (Tab. 12) and at 35.0 GiB on the 1200-image ArtSci collection. ArtSci has no ground-truth geometry and we did not run unfiltered inference on it, so we make no claim against no filtering there.

We acknowledge that, like Robust-VGGT, MACV cannot fully avoid the coverage loss inherent to view pruning, as the clean-input results above show. Judging more accurately when view filtering should be applied in feed-forward 3D reconstruction remains future work.

AuthorsOpenReview ↗

Response to Reviewer bee5 — Part 2

Q2. In Table 11, does Chunked Robust-VGGT use the global target seed as its reference in every chunk? If not, could you evaluate this variant under a comparable pool-size limit, counting the seed within that limit, and report the validation-pass count and runtime?

In Tab. 11, each disjoint 168-image chunk is scored against its own first view, so the target seed is the reference of the first chunk only. The Chunked Robust-VGGT variant with the same evaluation setting of MACV evaluates the ArtSci collection used in Tab. 11 and produces the following numbers. It scores each disjoint window once together with the seed:

MethodMCCF1CKRDRRValidation passesTime (s)Peak (GiB)
Seed in every window, 63 views0.0960.6440.5910.5102048019.5
Seed in every window, 168 views0.4150.7690.7110.725862143.7
MACV0.9760.9920.9880.993380190335.0

With 63-view windows, each pool holds at most 64 images including the seed, which equals P_max. MACV's own pools range from 2 to 130 images on this collection, and 11 of its 380 pools exceed 64. Its coverage step inserts each uncovered view into the pool of its nearest anchor and can grow a pool past P_max. We therefore also report MACV's window size of 168 views.

With 63-view windows, the variant needs 19 times fewer passes and runs 4 times faster than MACV, but its MCC is 0.096 against 0.976. With MACV's window size, its MCC rises only to 0.415, and its peak of 43.7 GiB exceeds MACV's. With the seed as the only reference, neither window size recovers the numbers. Validating views against multiple verified anchors in different contexts is what brings MCC to 0.976.

Q3. Could you provide precision-recall curves comparing Max Support against Mean Support and Majority Voting (Table 3), or report DRR at matched CKR?

We swept the threshold for the three rules (Max / Mean / Majority) and all rules use identical forwards. In the sweep, anchors are always kept and are excluded from the counts. At the CKR that Max reaches at τ = 0.4, the DRR of Max / Mean / Majority is 0.903 / 0.888 / 0.885 in Table 3's setting. Correspondingly, reaching the same DRR (0.903) at τ = 0.4, their CKR is 0.910 / 0.846 / 0.843.

We ran the same sweep on the UrbanScene3D (ArtSci) collection of Tab. 11 and Fig. 6. This collection represents large-scale scenes with visually ambiguous distractors. Of its 1200 aerial images, 400 are distractors showing another campus area (PolyTech) from the same dataset. At the CKR that Max reaches at τ = 0.4, the DRR of Max / Mean / Majority is 0.993 / 0.900 / 0.762. At the DRR that Max reaches at τ = 0.4, their CKR is 0.987 / 0.894 / 0.888. On this collection, Max rejects the most distractors at matched CKR and keeps the most clean views at matched DRR.

Reviewer_C4eHOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

This paper proposes MACV, a training-free method for outlier removal from an unordered collection of images containing both views of a target scene and unrelated outliers. A seed image identifying the target scene is provided. MACV aims to improve upon Robust-VGGT (CVPR 2026) through a multi-stage approach. First, anchor images are selected. Then, overlapping local pools are constructed around these anchors. Finally, each pool is processed by a feed-forward 3D reconstruction backbone, and images are scored using a strategy inspired by Robust-VGGT, combined with a best-support rule.

Experiments show that MACV often, though not consistently, outperforms Robust-VGGT in terms of outlier removal performance, while achieving substantially lower peak memory usage.

Critical strengths
  1. Practical relevance and scalability -- MACV could be of significant practical interest, as it can scale to larger image collections than Robust-VGGT, particularly thanks to its reduced memory requirements.

  2. Extensive experimental evaluation -- The paper provides extensive experiments across diverse benchmarks and reconstruction backbones, offering a broad evaluation of the proposed method.

Critical weaknesses
  1. Heuristic nature of the method. The proposed method relies heavily on heuristics, with little theoretical justification for its design choices. In particular, the paper does not provide formal results or analyses that would explain why the different components should work well together or under which conditions they are expected to succeed. While the practical benefits of MACV are clear, I find its methodological contributions somewhat limited, and it is unclear to me whether they are sufficiently substantial to be of broad interest to the ICLR community. To me this is the central weakness of this paper.

  2. Lack of clarity in the method presentation. I found the presentation unnecessarily difficult to follow, especially given the relative simplicity of the proposed method. MACV is essentially an algorithm composed of three stages, yet it is not presented in a clear, algorithmic form. Instead, each stage is described primarily in prose, which is sometimes difficult to interpret, while the equations do not always help clarify the procedure (see my questions below). I believe that a more structured presentation, ideally including explicit pseudocode, would substantially improve the readability and accessibility of the paper.

  3. Insufficient discussion of limitations. The limitations of the proposed method are not discussed in the main paper. While Appendix G is dedicated to this topic, it consists of only a few lines of discussion and a single failure case. Given the heuristic nature of MACV, a more thorough analysis of its limitations and potential failure modes would be particularly valuable.

Questions for authors
  1. Simplification of Equation (4) -- Unless I am missing something, Equation (4) should simplify to Kcand=2KeffK_{\mathrm{cand}} = 2K_{\mathrm{eff}}, since KeffK_{\mathrm{eff}} is an integer by Equation (3) and N≫1N \gg 1 according to Equation (1). Could the authors clarify whether there is a reason for keeping the current expression?

  2. Potential outliers among anchors -- During anchor selection, it seems possible for outlier images to be selected as anchors. Is this indeed the case? If so, how does MACV handle such situations? In particular, can an outlier still be rejected if it has been selected as an anchor? This is unclear to me because line 283 states that the seed image and verified anchors are kept for the final reconstruction, but the notion of a "verified anchor" is not clearly defined. I would appreciate clarification on this point.

  3. Suggestion for improving the presentation -- I strongly encourage the authors to include explicit pseudocode describing the complete MACV pipeline. Since the method is fundamentally a sequence of algorithmic steps, presenting it as an algorithm would, in my opinion, make the approach substantially easier to understand and reproduce.

2 comments
AuthorsOpenReview ↗

Response to Reviewer C4eH — Part 1

We thank the reviewer for finding the method practically relevant, noting that it scales with a lower peak memory and recognizing the extensive evaluation. We reply to each point below and are open to further discussion.

W1. Heuristic nature of the method: the paper gives no formal results or analyses that explain why the components work together or under which conditions they succeed, and its methodological contribution is limited.

MACV builds on existing components, namely the Robust-VGGT score and DINOv2 retrieval. Its technical contribution lies in the problem it analyzes, the way it combines these components and the evaluation that tests the combination. To our best knowledge, MACV is the first work to analyze two limitations of robust feed-forward 3D reconstruction together, namely the over-pruning of clean views and the modeling of large-scale scenes. Both follow from how Robust-VGGT judges each view, namely once, from one reference in one context. On a clean collection, it removes 42% of the clean views (Tab. 1), a cost that retained-view metrics hide. Its single context must also hold the whole collection, so the released implementation runs out of memory near 170 images on a 64 GB GPU.

To overcome both limitations, MACV validates each view over multiple local contexts rather than over the whole input at once. Each context is a bounded pool around an anchor verified against the seed, so peak memory is set by the pool size rather than the collection size. A view is kept when at least one context supports it, so a valid view is not penalized for contexts it does not overlap.

A short analysis explains why these components work together and when they succeed. Let E(i)E(i) be the anchor–pool evaluations of view ii, with support scores se(i)s_e(i). MACV keeps ii if se(i)≥τs_e(i) \ge \tau for at least one e∈E(i)e \in E(i). Two bounds follow without any independence assumption. For a clean view, Pr⁡[i rejected]≤min⁡e∈E(i)Pr⁡[se(i)<τ]\Pr[i \text{ rejected}] \le \min_{e \in E(i)} \Pr[s_e(i) < \tau]. A clean view is thus lost no more often than in its best context, and for fixed scores an added evaluation can only help it. For a distractor, Pr⁡[i kept]≤∑e∈E(i)Pr⁡[se(i)≥τ]\Pr[i \text{ kept}] \le \sum_{e \in E(i)} \Pr[s_e(i) \ge \tau]. Best support therefore pays with false acceptance, which stays small when every reference shows the target scene but grows with the number of evaluations.

The bounds give each component its role. Anchor verification keeps each term of the sum small, the cap on the anchor count keeps the sum short, and bounded pools keep each evaluation within memory. MACV is therefore expected to succeed when (C1) every anchor shows the target scene and (C2) every clean view shares a pool with an anchor it overlaps. Anchor verification serves C1, and anchor-local pools with the coverage step serve C2.

The ablation in Tab. 3 tests each component against the condition it serves, and the measurements follow these predictions. Without anchor verification, C1 fails and DRR falls from 0.903 to 0.183 (Tab. 3). Outlier anchors and a seed from another scene, two of the limitations in our reply to W3, are the cases where C1 fails.

Beyond these tests, we evaluate on six datasets, with three backbones (VGGT, π³ and MapAnything), at four noise levels and on collections of up to 1200 images. Preservation-aware metrics count the clean views a filter removes as well as the distractors it keeps. Under these metrics, MACV keeps more clean views and reaches a higher F1 than Robust-VGGT and its π³ and MapAnything counterparts, at every noise level (Tab. 1). On clean inputs, CKR rises from 0.580 to 0.782 with VGGT. On the 1200-image ArtSci collection, MACV reaches MCC 0.976 at a 35.0 GiB peak, against 0.415 at 43.7 GiB for chunked Robust-VGGT with the seed in every 168-view window. It needs no training and runs unchanged on all three backbones.

Together, these three aspects clearly distinguish our paper from the existing literature. Robust-VGGT judges each view once, within a single context that must hold the whole collection. MACV identifies the two costs of this design, namely pruned clean views and peak memory that grows with the collection. It addresses both with best support over multiple verified local contexts, and its evaluation counts the clean views removed as well as the distractors kept.

Finally, the paper is submitted to ICLR's primary area on applications to computer vision, and its question is broader than 3D. A pretrained model must judge inputs that do not fit in one context, under a fixed memory budget and without retraining. MACV answers this question for feed-forward 3D reconstruction with a training-free decision rule and the two bounds above make its trade-off explicit.

AuthorsOpenReview ↗

Response to Reviewer C4eH — Part 2

W2 and Q3. Clarity of the method presentation; explicit pseudocode.

In Sec. 3, we are adding a new single algorithm box (Algorithm 1) that lists the complete pipeline step by step, including anchor selection and verification, window and pool construction, per-pool scoring and the final decision. The equations are being aligned with the algorithm.

W3. Limitations are discussed only briefly, in Appendix G.

MACV has the following limitations. First, MACV removes 22% to 28% of the clean views of a clean collection (Tab. 1). With no distractors or only a few, direct inference is therefore as good or better. Second, anchor verification is an appearance test, so a distractor that resembles the seed can become an anchor. Such an anchor is kept without being scored and can admit the distractors of its own pool that are compatible with it. Third, distractors that look like the target, such as the repetitive rooms of Fig. 15, can score high in one pool, and best support then keeps them. Finally, the seed defines the target, so MACV keeps the seed's scene whatever it is. On 7-Scenes at N = 100 with a distractor as the seed, it keeps every view of the seed's scene and only 7% to 20% of the intended target views.

Q1. Eq. 4 should simplify to K_cand = 2K_eff.

The reviewer is right. As written, Eq. 4 reduces to Kcand=2KeffK_{\mathrm{cand}} = 2K_{\mathrm{eff}}. We will simplify the wording in the revision.

Q2. Can outlier images be selected as anchors? If so, how does MACV handle this, and can an outlier still be rejected once it is an anchor? The notion of a "verified anchor" is not clearly defined.

Yes. A distractor that resembles the seed can become an anchor but this is rare. Over the 144 noisy sequences, 11 of the 640 non-seed anchors are distractors (Tab. 14). Anchors are kept without being scored (line 283), so such an outlier is not rejected and can also admit the compatible distractors of its own pool. If anchors were scored by the pools of the other anchors like any other view, 4 of these 11 outliers would be rejected.

A verified anchor is a view selected in two steps (Sec. 3.2). First, its DINOv2 cosine similarity to the seed must be at least τ_ver = 0.65 (Eq. 5). Among the views that pass, we pick K_cand = 2K_eff candidates and then the K_eff most similar to the seed become anchors. Verification is thus an appearance test and involves no geometric check.

Reviewer_MjZiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies distractor filtering for feed-forward 3D reconstruction from long image collections. Existing single-context methods such as Robust-VGGT can be memory-intensive and may discard many valid views. MACV is a training-free framework that selects multiple seed-consistent anchors, uses overlapping windows to form anchor-local pools, applies Robust-VGGT-style consistency scoring within each pool, and retains an image if any pool provides sufficient support. Experiments across four short-sequence benchmarks, 7-Scenes sequences of up to 400 images, and a 1,200-image UrbanScene3D collection show that MACV generally removes distractors while preserving more clean views. It also remains practical at scales where the released single-context baseline runs out of memory and improves downstream pose and depth estimation in several noisy and long-sequence settings.

Critical strengths

The paper convincingly identifies and quantifies over-pruning. On clean inputs, MACV retains more valid views than Robust-VGGT. This clean-view analysis is absent from the Robust-VGGT and is a useful contribution in its own right. Even a more permissive threshold does not close the clean-retention gap and still performs worse than MACV under heavy corruption (Tab. 16).

Critical weaknesses

The relationship to Robust-VGGT is not characterized precisely enough. Sec. 3.4 credits Robust-VGGT for the scoring cues, but the cue weighting, acceptance threshold, feature layer, and preprocessing also appear to be inherited from its combined variant without explicit attribution. Moreover, subject to the normalization used, thresholding the best support across anchor-centered pools can be interpreted as taking the union of the thresholded decisions produced within those pools, apart from the force-kept seed. This places MACV's main novelty in anchor selection, pool construction, and multi-context aggregation rather than in the validator itself. Robust-VGGT also identified hierarchical or cluster-wise filtering as a possible scaling strategy in its limitations section, so MACV should be positioned as one realization of this broader direction.

Questions for authors
  1. Is Eq. 9 normalized independently within each pool using min–max normalization, as in the Robust-VGGT code? Does this include both the anchor and the seed? How much clean-view loss remains without per-pool rescaling, using a single threshold *
  2. How is an unordered image collection sequentialized for the sliding-window procedure, and where are the distractors inserted in 7-Scenes?
  3. Are the pose errors in Tabs. 4 and 8 computed only on retained clean frames? What clean-image retention does MACV achieve at N=150 and N=200?
  4. On clean ETH3D, why does unfiltered MapAnything have much worse AbsRel than the filtered variants, while the corresponding VGGT and π3 gaps are small (Tab. 6)? Is the unfiltered MapAnything pipeline otherwise identical?
  5. How do the unfiltered backbones process 400-image sequences on a 64 GB MI210 in Tabs. 7–8, and could full-context Robust-X use the same memory settings?

Minor: Define the "DINO retrieval threshold" reported in Tab. 17. The "best trade-off" claim for the default configuration is not fully supported: cost is reported only for Full Context, while the [12,64] configuration in Tab. 3 matches or exceeds the default on all four reported metrics.

3 comments
AuthorsOpenReview ↗

Response to Reviewer MjZi — Part 1

We thank the reviewer for finding that over-pruning is convincingly identified and quantified, and that the clean-view analysis is a contribution in its own right. We reply to each point below and are open to further discussion.

Weakness. The relationship to Robust-VGGT is not characterized precisely enough.

Robust-VGGT addresses view pruning holistically, judging every view once against one reference within a single context that holds the whole collection. The key contribution of MACV is to localize this problem into bounded local contexts, each built around an anchor verified against the seed, and then to aggregate the local decisions into one decision per view. Within each context, the per-view score is that of Robust-VGGT. The two cues (late-layer feature cosine and reference attention), their equal weighting, the min–max normalization within a context, the threshold of 0.4, the scoring layer and the preprocessing follow its combined variant. The only change is that MACV excludes the pool's anchor from the normalization range. Apart from the seed and the anchors, which are always kept, thresholding the best support equals the union of the per-pool decisions, as the reviewer observes. This union is how MACV aggregates the local results. We deliberately share the rest of the implementation with Robust-VGGT, so that the gains over Robust-VGGT can be more clearly attributed to the localization and aggregation rather than to a different score, threshold or preprocessing.

The limitation that this union addresses becomes visible once filtering is evaluated on both sides. Our preservation-aware metrics count removed clean views, and they show that single-context filtering removes 42% of the clean views of a clean collection (Tab. 1). A lower threshold does not repair this without admitting distractors (Tab. 16). MACV starts from the observation that support is measured in a context, whereas validity is a property of the scene. A valid view can look unsupported against one reference in one context, so MACV keeps a view when at least one verified target-scene context supports it. This rule is sound when the references show the target scene and every valid view shares a context with a reference it overlaps. Anchor verification, anchor-local pools and coverage are the components that meet these two conditions under a memory bound. A controlled comparison shows that the gain comes from the verified anchors rather than from evaluating each view several times, as shown in Tab. 3.

We agree that MACV belongs to the scaling direction that Robust-VGGT outlines in its limitations. It is, however, more than a direct instance of cluster-wise filtering. A cluster-wise filter splits the collection and judges each view once, against the reference of its own cluster. MACV instead scores a view in every pool that contains it, each pool against its own seed-verified anchor, and keeps the view if any pool supports it. The principle behind this design, validation over contexts, is not specific to view pruning. A feed-forward reconstruction model forms each per-view prediction within the context it is given, so the prediction can change with the context. For collections that exceed the memory budget, other per-view decisions could likewise be made over several bounded contexts with verified references and then aggregated. Examples are deciding which views to trust when chunked reconstructions of a long sequence are aligned, and choosing reference views under a memory budget. More broadly, the idea of MACV could be a starting point for ensemble-based 3D reconstruction models, which treat several bounded contexts with verified references as ensemble members and aggregate their per-view predictions. We will add this discussion to the revision as an outlook, since this paper evaluates the principle on view filtering only.

AuthorsOpenReview ↗

Response to Reviewer MjZi — Part 2

Q1. Is Eq. 9 normalized independently within each pool using min–max normalization, as in the Robust-VGGT code? Does this include both the anchor and the seed? How much clean-view loss remains without per-pool rescaling, using a single threshold?

Yes. As in the Robust-VGGT code, each pool is min–max normalized on its own before the blend of Eq. 9. The range excludes the pool's anchor, the only difference from Robust-VGGT which includes its reference view. Under the default setting, we re-scored the stored per-pool outputs with raw scores and report numbers (CKR) as following:

ScoresCleanLowMidHigh
Per-pool normalization0.7820.9750.9940.993
Raw scores0.8990.9220.9830.988

On clean inputs, the clean-view loss falls from 21.8% to 10.1% while on noisy inputs the raw scores produce slightly worse numbers. At the same distractor rejection, raw scores keep fewer clean views and the best MCC falls from 0.949 to 0.925. Per-pool normalization gives more stable results across datasets, so we keep it as the default.

Q2. How is an unordered image collection sequentialized for the sliding-window procedure, and where are the distractors inserted in 7-Scenes?

Given an image collection, MACV first builds a graph that links each image to its mutual nearest neighbours in DINOv2 feature space. It then starts a window at an image not yet covered and grows it by breadth-first search on this graph to a fixed size. This step repeats from the next uncovered image until every image is covered. Each window therefore holds visually related images rather than a contiguous stretch of the input. Sec. 3.3 and Algorithm 1 of the revision will describe this procedure.

In 7-Scenes, each input holds N frames of the target scene and about N distractor frames, drawn equally from the other six scenes. Clean and distractor frames are randomly interleaved, and the first image (the seed) is always clean.

Q3. Are the pose errors in Tabs. 4 and 8 computed only on retained clean frames? What clean-image retention does MACV achieve at N = 150 and N = 200?

Yes. The pose metrics (ATE, RPE and RTE) in Tabs. 4 and 8 are computed over the retained clean views. The point cloud for CD and F-score in Tab. 4 is computed over all retained views, including retained distractors.

As suggested by the reviewer, we report CKR / DRR of the runs behind these tables here, averaged over the seven scenes:

MethodN = 150N = 200
MACV-VGGT0.845 / 0.9400.913 / 0.950
MACV-π³0.821 / 0.8910.856 / 0.847
MACV-MA0.756 / 0.9110.816 / 0.934
AuthorsOpenReview ↗

Response to Reviewer MjZi — Part 3

Q4. On clean ETH3D, why does unfiltered MapAnything have much worse AbsRel than the filtered variants, while the VGGT and π³ gaps are small (Tab. 6)? Is the unfiltered MapAnything pipeline otherwise identical?

No. The two pipelines differ in two ways, and both make unfiltered MapAnything look worse in Tab. 6. First, every filtered row of Tab. 6, including MACV-MA and Robust-MA, reconstructs its retained views with π³, while unfiltered MapAnything reconstructs with MapAnything. Second, MapAnything's inference applies its default output mask, which sets 6% to 13% of the depth pixels to zero. Our depth evaluation fitted scale and shift over all pixels and scored the zeroed pixels as predictions. The table below separates the two effects (AbsRel).

SettingETH3D cleanScanNet++ cleanMean, 2 datasets × 4 noise levels
Unfiltered MapAnything, as submitted (zeroed pixels scored)0.1900.1750.188
Unfiltered MapAnything, zeroed pixels excluded from fit and metric0.0640.0370.051
Unfiltered MapAnything, re-run without output masking0.0840.0380.060
MACV-MA, retained views reconstructed with π³ (as submitted)0.0170.0300.024
MACV-MA, retained views reconstructed with MapAnything, no masking0.0370.0380.044
π³ / VGGT, unfiltered (reference)0.018 / 0.0280.029 / 0.0400.027 / 0.040

With the same backbone and no masking, the gap closes on ScanNet++ (0.038 for both unfiltered MapAnything and MACV-MA). On ETH3D, the remaining gap comes from two of the eleven scenes, relief and relief_2, where unfiltered MapAnything reaches AbsRel 0.220 and 0.324. On the other nine scenes, unfiltered MapAnything and MACV-MA reach 0.042 and 0.039.

δ < 1.25 of unfiltered MapAnything changes accordingly (mean 0.756 as submitted, 0.950 without masking). Masking is post-processing, so the pose results are unaffected (mean AUC@20° 68.83 vs. 68.82).

Q5. How do the unfiltered backbones process 400-image sequences on a 64 GB MI210 in Tabs. 7–8, and could full-context Robust-X use the same memory settings?

Full-context Robust-X cannot use the same memory settings with the released Robust-VGGT code. Each unfiltered backbone processes the 398 images in one forward pass without chunking and never forms an attention map. VGGT keeps only the four intermediate layers its heads read, and MapAnything uses its memory-efficient inference mode. The released Robust-VGGT code instead keeps the tokens of intermediate layers and materializes the attention from the anchor's patches to every token of every view. Its score is computed from exactly these quantities, which the unfiltered forward discards.

On the same 64 GB GPU, the released Robust-VGGT peaks at about 4.8 + 0.34 N GiB for N images, or 61.9 GiB at 168 images. At 196 images (7-Scenes N = 100), it runs out of memory in the anchor-attention step. MACV computes the same score but bounds every forward at the pool size. Its peak stays at 15 to 27 GiB up to 400 images (Tab. 12) and at 35.0 GiB on the 1200-image ArtSci collection.

Minor. Define the "DINO retrieval threshold" reported in Tab. 17. The "best trade-off" claim for the default configuration is not fully supported.

The "DINO retrieval threshold" in Tab. 17 is τ_ver of Eq. 5, the minimum DINOv2 cosine similarity to the seed that an image needs to serve as an anchor. We will correct the labels in Tab. 17 in revision. We also agree with the reviewer on the "best trade-off" claim and have removed it.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of the Revision

We thank all reviewers for their feedback. We have uploaded a revised PDF that implements our replies with the revised text highlighted in blue. The main text still fits in 9 pages. The new tables are numbered in sequence so the appendix tables after Tab. 4 are renumbered. Below, table numbers refer to the revised PDF with the submitted number in parentheses where it changed.

1. Method presentation (Sec. 3)

  • New Algorithm 1 lists the full pipeline: anchor selection and verification, window and pool construction, per-pool scoring and the final decision (C4eH).
  • Sec. 3.2 defines a verified anchor (C4eH Q2) and simplify the notation of 2K_eff (C4eH Q1).
  • Sec. 3.3 explains how an unordered collection is covered (MjZi Q2).
  • Sec. 3.4 and Appendix A state which parts of the per-pool score follow the combined variant of Robust-VGGT, and that the normalization range excludes the pool's anchor (MjZi).
  • Sec. 3.5 notes that thresholding the best support keeps the union of the per-pool decisions (MjZi). A new paragraph bounds clean-view rejection and distractor acceptance and states two conditions (C1, C2) under which MACV is expected to succeed (C4eH W1).

2. Positioning, limitations and outlook

  • Sec. 2 relates MACV to Robust-VGGT and to cluster-wise filtering (MjZi).
  • Sec. 5 summarizes the limitations and adds an outlook on validation over contexts beyond view filtering (MjZi, C4eH W3).
  • Appendix G is rewritten as a list of limitations, including a wrong-seed experiment (C4eH W3).

3. Protocol clarifications

  • Sec. 4.1 describes how the 7-Scenes inputs are built (MjZi Q2).
  • Sec. 4.1 and Appendix B state that every filtering method reconstructs its retained views with π³, for pose and depth as well as point clouds (bee5 Q1, MjZi Q4).
  • Appendix B states that long-sequence pose errors use the retained clean views, whereas CD and F-score use all retained views (MjZi Q3).

4. Corrected results

  • Unfiltered MapAnything now runs without its default output mask, which zeroed 6% to 13% of the depth pixels (MjZi Q4).
  • Tab. 14 (formerly 11) adds two variants that score every window against the seed, with validation passes, runtime and peak memory.
  • Tab. 21 (formerly 17) is relabeled (MjZi). The former "DINO retrieval threshold" row is τ_ver.
  • The "best trade-off" claim is removed from Tab. 3 (MjZi).

5. New analyses (Appendices C, E and F)

  • New Tab. 12 compares MACV-X with unfiltered X under the same backbone (bee5 Q1).
  • New Tab. 13 reports CKR, DRR and MCC on 7-Scenes for N = 50 to 200 (MjZi Q3).
  • New Tab. 18 compares per-pool normalization with raw scores (MjZi Q1).
  • Appendix E explains why the unfiltered backbones fit 398 images while the released Robust-VGGT runs out of memory (MjZi Q5).
  • Appendix F compares Max, Mean and Majority at matched operating points (bee5 Q3). It also reports how often a distractor becomes an anchor (C4eH Q2).

6. Layout

  • To make room for Algorithm 1, the point clouds across sequence lengths (former Fig. 8) moved to Appendix D (now Fig. 14). The qualitative ablation is now Fig. 8.
  • Equations in Sec. 3 are renumbered: τ_ver is now Eq. 2 (formerly Eq. 5) and the support blend is Eq. 3 (formerly Eq. 9).
AuthorsOpenReview ↗

Final note on the method novelty

Dear Reviewers and AC,

This is our last note on the technical contribution. First, we sincerely thank you for the time and effort you have devoted to our paper and for the many suggestions that improved its quality. We really appreciate it! Based on your comments, we have improved the paper's presentation, corrected the evaluation and made the experiments more thorough. We therefore expect the technical contribution to be the main topic of the coming reviewer-AC discussion.

On this point, we would like to make one observation. We keep the validator of Robust-VGGT unchanged, so the improvements on six datasets and across three backbone networks can only come from what we add: validating each view over multiple verified contexts. The obvious ways of adding contexts do not produce these improvements, whether chunking, using the seed as the only reference, unverified anchors, or mean or majority aggregation. If the steps were obvious, one of them should have worked. Also, if our technical contribution were so limited, how could it alone account for such notable empirical improvements?

As noted in our comment "One scoping question on methodological novelty", our contribution has four parts: 1) problem analysis, 2) design principle, 3) framework implementation and 4) improved results. As far as we know, we are the first to point out the over-pruning of clean input views in view pruning, an analysis that Reviewers MjZi and bee5 list among the paper's strengths. In addition, no existing method is built on the principle that each view is validated over multiple local verified contexts before the final decision. These are our most notable conceptual contributions.

To realize them, we inherit the validator settings and the corresponding evaluation protocol of Robust-VGGT and add a few simple components on top. We emphasize that this is on purpose, and it brings two advantages. 1) Compared with more complicated alternatives, our design is simple yet still yields strong empirical gains. 2) Its simplicity helps us rule out irrelevant factors, so that the experiments focus on the importance of the key components. We believe the current results clearly bear out both advantages and show the soundness of our method, which the reviewers have acknowledged.

The ablations above also show that each proposed component is necessary. Note that the purpose of a scientific paper is to deliver solid and useful knowledge, not to present elaborate technical designs. In its current state, we believe our paper does exactly that.

Although we would very much like to discuss the technical side further with the reviewers now, we will have to wait until the final decision to learn more. By then, we hope the reviewers can share more fine-grained insights into which methodological parts need significant further improvement. Finally, thank you again for your time on this paper.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.