- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the sensitivity of feed-forward 3D reconstruction models to uncurated image collections containing irrelevant distractors. Existing test-time filtering methods such as Robust-VGGT operate within a single bounded context, running out of memory on long collections or over-pruning valid views when restricted to a local window. To overcome this, the authors propose Multi-Anchor Context Validation (MACV), a training-free input-side filtering front-end.
The framework of MACV is built on following three steps:
- Global anchor selection: Identifies and verifies seed-consistent reference anchors across the scene using DINOv2 feature retrieval and a seeded farthest-point procedure.
- Bounded context partitioning: Covers the collection with overlapping context windows and constructs anchor-local validation pools bounded by a strict capacity limit ().
- Best-support validation: Evaluates candidate views using internal backbone features and anchor-centric attention, following Robust-VGGT, and aggregates support across evaluated contexts using a max-score decision rule.
The retained subset is forwarded to an e2e 3D reconstruction backbone (e.g., VGGT, , MapAnything). Experiments across six benchmarks show that MACV preserves more valid views than single-context filtering baselines and scales to collections of up to 1,200 images.
Critical strengths
I found two main strengths in this work:
First, the conceptual motivation to frame input filtering as a "preservation-aware" problem is valuable and well-argued. In 3D reconstruction, over-pruning valid views destroys scene coverage and can be just as catastrophic as admitting distractors. Addressing the memory-robustness-coverage trade-off through repeated bounded validations is a sensible objective.
Second, the paper provides extensive empirical characterization across three modern e2e 3D reconstruction backbones (VGGT, , MapAnything) and six datasets. The ablation and diagnostic analyses in the appendix (e.g., Tables 12–16) are informative as well.
Critical weaknesses
My main concerns are the practical trade-offs and whether the comparisons sufficiently isolate the source of the reported gains:
First, the practical trade-off against no filtering needs clearer discussion. MACV improves over Robust-X in the reported averages, but this does not always translate into a benefit over using the unfiltered backbone. For example, Table 5 reports VGGT AUC@20 of 84.1 and 81.6 on the clean and small-noise inputs, compared with MACV’s preservation-aware scores of 53.0 and 76.9. Table 1 also shows that MACV removes approximately 22–28% of valid views on clean inputs, averaged across datasets. These are coverage-adjusted performance losses, not necessarily worse estimates on the retained views. The gains in noisier and longer collections remain valuable, but the paper should more clearly discuss when filtering is beneficial and when its coverage cost outweighs its benefit.
Second, the thousand-image comparison does not clearly separate target-reference availability from multi-anchor validation. Appendix E attributes the poor performance of Chunked Robust-VGGT partly to the absence of a verified target-scene reference in its chunks. Since the task already provides a target seed, a natural control is to use the same seed as the reference in every chunk. Table 3 includes a single-anchor ablation on 7-Scenes, but a corresponding seed-controlled comparison on ArtSci would make the large-scale result more convincing. The pool-size limit should be kept comparable, and the number of validation passes should be reported.
Third, the operating-point dependence of best-support aggregation needs further characterization. A max rule gives a candidate more opportunities to exceed the fixed threshold as additional anchor–context evaluations are added. This is a potential false-acceptance risk, rather than evidence that MACV necessarily admits more distractors. In fact, Table 3 reports the same DRR for Max, Mean, and Majority Vote, with higher CKR for Max at the tested setting. (Tables 16–17 also provide useful threshold analyses.) What remains unclear is whether Max retains its advantage over the other aggregation rules at matched CKR, particularly as the number of evaluations increases or the distractors become more visually ambiguous.
Questions for authors
Could the authors address the following points during the rebuttal?
-
Characterization of Operational Regimes vs. No-Filtering:
Could you characterize the tested regimes in which MACV improves over no filtering, separated by dataset, noise level, and collection length? Please distinguish cases where direct inference remains feasible from cases where filtering is needed to meet the memory budget.
-
Controlled Chunked Baseline with Global Seed:
In Table 11, does Chunked Robust-VGGT use the global target seed as its reference in every chunk? If not, could you evaluate this variant under a comparable pool-size limit, counting the seed within that limit, and report the validation-pass count and runtime?
-
Trade-off Analysis of Best-Support (Max) Aggregation:
Could you provide precision-recall curves comparing Max Support against Mean Support and Majority Voting (Table 3), or report DRR at matched CKR? How does the false-positive behavior of the max rule hold up in scenes with visually ambiguous distractors (e.g., the repetitive rooms in Figure 15)?
I am open to revising my rating if the authors address the concerns raised above.
2 comments
Response to Reviewer bee5 — Part 2
Q2. In Table 11, does Chunked Robust-VGGT use the global target seed as its reference in every chunk? If not, could you evaluate this variant under a comparable pool-size limit, counting the seed within that limit, and report the validation-pass count and runtime?
In Tab. 11, each disjoint 168-image chunk is scored against its own first view, so the target seed is the reference of the first chunk only. The Chunked Robust-VGGT variant with the same evaluation setting of MACV evaluates the ArtSci collection used in Tab. 11 and produces the following numbers. It scores each disjoint window once together with the seed:
| Method | MCC | F1 | CKR | DRR | Validation passes | Time (s) | Peak (GiB) |
|---|---|---|---|---|---|---|---|
| Seed in every window, 63 views | 0.096 | 0.644 | 0.591 | 0.510 | 20 | 480 | 19.5 |
| Seed in every window, 168 views | 0.415 | 0.769 | 0.711 | 0.725 | 8 | 621 | 43.7 |
| MACV | 0.976 | 0.992 | 0.988 | 0.993 | 380 | 1903 | 35.0 |
With 63-view windows, each pool holds at most 64 images including the seed, which equals P_max. MACV's own pools range from 2 to 130 images on this collection, and 11 of its 380 pools exceed 64. Its coverage step inserts each uncovered view into the pool of its nearest anchor and can grow a pool past P_max. We therefore also report MACV's window size of 168 views.
With 63-view windows, the variant needs 19 times fewer passes and runs 4 times faster than MACV, but its MCC is 0.096 against 0.976. With MACV's window size, its MCC rises only to 0.415, and its peak of 43.7 GiB exceeds MACV's. With the seed as the only reference, neither window size recovers the numbers. Validating views against multiple verified anchors in different contexts is what brings MCC to 0.976.
Q3. Could you provide precision-recall curves comparing Max Support against Mean Support and Majority Voting (Table 3), or report DRR at matched CKR?
We swept the threshold for the three rules (Max / Mean / Majority) and all rules use identical forwards. In the sweep, anchors are always kept and are excluded from the counts. At the CKR that Max reaches at τ = 0.4, the DRR of Max / Mean / Majority is 0.903 / 0.888 / 0.885 in Table 3's setting. Correspondingly, reaching the same DRR (0.903) at τ = 0.4, their CKR is 0.910 / 0.846 / 0.843.
We ran the same sweep on the UrbanScene3D (ArtSci) collection of Tab. 11 and Fig. 6. This collection represents large-scale scenes with visually ambiguous distractors. Of its 1200 aerial images, 400 are distractors showing another campus area (PolyTech) from the same dataset. At the CKR that Max reaches at τ = 0.4, the DRR of Max / Mean / Majority is 0.993 / 0.900 / 0.762. At the DRR that Max reaches at τ = 0.4, their CKR is 0.987 / 0.894 / 0.888. On this collection, Max rejects the most distractors at matched CKR and keeps the most clean views at matched DRR.
Response to Reviewer bee5 — Part 1
We thank the reviewer for valuing the preservation-aware framing and the evaluation across three backbones and six datasets. We reply to each point below and are open to further discussion.
Q1. Could you characterize the tested regimes in which MACV improves over no filtering, separated by dataset, noise level, and collection length? Please distinguish cases where direct inference remains feasible from cases where filtering is needed to meet the memory budget.
Based on the preservation-aware scores, the noise level (distractor ratio) matters more than the dataset or the collection length. On clean inputs, no filtering is better. MACV removes 22% to 28% of the clean views (Tab. 1), and the lost coverage outweighs the accuracy gained on the retained views. At Small noise (25% distractors), no filtering is as good or slightly better on average. At Medium and Large noise (50% and 62.5% distractors), MACV is better on all four datasets, with VGGT and with π³. On 7-Scenes (50% distractors), it is also better at every tested length from 98 to 398 images.
These conclusions rest on a comparison that uses the same backbone for filtering and reconstruction. The filtered rows of Tab. 5 reconstruct the retained views with π³, so its comparison of 84.1 / 81.6 with 53.0 / 76.9 also changes the backbone. The table below gives the preservation-aware pose AUC@20° as unfiltered → MACV, with bold marking the cells where MACV is better.
With π³, MACV is also better on each dataset at Medium and Large noise. On the views it retains, MACV-VGGT is more accurate than unfiltered VGGT at every level (87.5 vs. 84.1 on clean inputs), so the clean-input deficit is a loss of coverage. For depth, distractors have little effect and filtering does not pay off.
For the collection length, we compare unfiltered π³ → MACV-π³ on 7-Scenes sequences, with both reconstructed by π³.
MACV-π³ is better on every metric at every length. CD and F-score are computed on the whole output set, namely all retained views including retained distractors, against a ground truth built from all clean frames. They therefore penalize both removed clean views and kept distractors in a more systematic way. Unlike the pose errors, which are computed on the retained clean frames (82% to 100% of them), they need no preservation-aware correction. This geometry evaluation therefore supports the conclusions above without any correction.
Direct inference remains feasible in every setting above, since unfiltered inference fits in 64 GB. MACV's gains over no filtering are therefore accuracy gains, not a way around a memory limit. We also ran unfiltered VGGT and π³ on 698 images without running out of memory. The memory limit applies to full-context robust filtering instead. The released Robust-VGGT runs out of memory beyond about 170 images. MACV peaks at 15 to 27 GiB up to 400 images (Tab. 12) and at 35.0 GiB on the 1200-image ArtSci collection. ArtSci has no ground-truth geometry and we did not run unfiltered inference on it, so we make no claim against no filtering there.
We acknowledge that, like Robust-VGGT, MACV cannot fully avoid the coverage loss inherent to view pruning, as the clean-input results above show. Judging more accurately when view filtering should be applied in feed-forward 3D reconstruction remains future work.