- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes RCCR-MVS, a memory-efficient recurrent framework for high-resolution aerial multi-view stereo.
The core contribution is a Reverse-Context-Conditioned Regularizer (RCCR), which uses a lightweight reverse recurrent encoder to summarize evidence from current and subsequent depth hypotheses into a compressed reverse context. This context is then used to condition the gates and candidate states of a multi-scale forward recurrent encoder-decoder. The goal is to exploit bidirectional depth-hypothesis information without storing a full 4D cost volume.
The complete cascaded framework further introduces Coverage-Risk-Guided Range Expansion (CRE) to adaptively widen depth-search intervals when the true depth may fall outside the baseline range, and Spatial Evidence Support (SES) to enhance noisy matching evidence before recurrent regularization. Experiments on WHU-MVS, Luojia-MVS, München, Vaihingen, and the proposed MultiResXD-MVS dataset show strong accuracy, efficiency, cross-domain, and cross-resolution performance.
Critical strengths
-
The core RCCR architecture is a meaningful improvement over existing slice-wise recurrent regularization. The asymmetric design avoids duplicating the full forward regularizer while still allowing information from future depth hypotheses to influence current updates. This provides a clear accuracy-memory trade-off.
-
The experimental validation is strong. The paper evaluates both single-stage and coarse-to-fine variants, includes component ablations, computational efficiency analysis, cross-domain testing, and cross-resolution generalization. The results are consistently competitive and often state of the art.
Critical weaknesses
-
The broader methodological novelty is somewhat limited. Bidirectional recurrent MVS and global cross-hypothesis regularization have already been studied, and the main advance is a more efficient asymmetric way to inject compressed reverse context. While useful, the contribution remains fairly specific to slice-wise aerial MVS regularization.
-
CRE and SES appear more like effective engineering extensions than major conceptual contributions. CRE is closely related to uncertainty-guided adaptive depth-range adjustment, while SES resembles geometry-aware local cost aggregation. Their empirical benefits are clear, but they do not substantially broaden the conceptual contribution beyond the RCCR core.
Questions for authors
-
Could the authors clarify the conceptual distinction between RCCR and prior bidirectional recurrent MVS methods beyond computational asymmetry? In particular, is the main advantage the compressed reverse representation, the way it conditions the forward gates, or both?
-
How sensitive is RCCR to the reverse-context channel dimension K? Since K=4 is used throughout, it would be useful to understand whether the performance-memory trade-off is stable across different compression levels.
-
Could the authors comment on whether CRE and SES are necessary for the main contribution, or whether RCCR alone is intended to be the central transferable idea? The ablation suggests RCCR provides most of the conceptual gain, while CRE and SES mainly improve the final aerial-MVS pipeline.
-
The method achieves better accuracy but is not uniformly faster or lower-memory than all coarse-to-fine baselines. Could the authors clarify the intended operating regime where RCCR-MVS offers the strongest practical advantage?
3 comments
Response to 948J (Part 2/2): Core Contribution, Extensions, and Practical Advantages
This part addresses W2/Q3 and Q4: module roles and practical use.
[W2&Q3] RCCR Is the Core Contribution; CRE and SES Are Complementary
Question: Could the authors comment on whether CRE and SES are necessary for the main contribution, or whether RCCR alone is intended to be the central transferable idea? The ablation suggests RCCR provides most of the conceptual gain while CRE and SES mainly improve the final aerial-MVS pipeline.
RCCR is the central regularization idea and requires neither CRE nor SES. Section 3.2 explicitly establishes this hierarchy. Table 3 shows RCCR-only improves all eight metrics across WHU-MVS, Luojia-MVS, München tiles, and resized MultiResXD-MVS, including 7.50%/34.62% lower MAE on WHU/resized MultiResXD.
Table 10 also shows improvements beyond selected examples and truncated errors:
| Setting | CasREDNet MAE (m) | RCCR-only MAE (m) | Improved (%) |
|---|---|---|---|
| WHU-MVS, five views | 0.1474 | 0.1411 | 97.94 |
| MultiResXD resized 1× | 1.2621 | 0.9464 | 87.76 |
| MultiResXD native 4× | 2.3561 | 2.1768 | 95.92 |
Here MAEs are untruncated; Improved counts images with lower truncated MAE than CasREDNet. RCCR-only also improves throughout. Appendix A.15 (new Figures 18–21) adds RCCR-only posteriors, ROI errors, and depthwise-influence analyses across five aerial datasets, complementing Table 10.
CRE and SES improve distinct parts of the complete aerial pipeline. CRE adjusts hypothesis coverage before cost construction; SES refines spatial matching evidence before regularization. In Table 12, adaptive CRE reaches 0.08563 m MAE versus 0.09297 m for training-set-calibrated fixed expansion and 0.08884 m for fixed expansion. This supports spatially adaptive allocation rather than uniform widening. In Table 13, Full SES reduces edge-region MAE to 0.55749 m from 0.57013 m without geometry, supporting the value of geometric compatibility near boundaries.
The complete model further lowers final MAE and outlier rates throughout Table 10. Its lower WHU first-stage than RCCR-only (81.67% versus 82.39%) does not contradict this: posterior mass is an intermediate diagnostic, not a monotonic predictor of final error. RCCR changes regularization; CRE/SES strengthen its cascaded deployment.
Revision: Section 3.2 states the hierarchy. Section 4.3, Table 3/Figure 8 and Appendix A.12, Table 10 establish RCCR-only effectiveness. The new Appendix A.15 (Figures 18–21) extends the visual evidence. Appendix A.14, Tables 12–13 and Appendices A.3–A.4 cover the extensions.
[Q4] Where the Accuracy–Resource Trade-off Is Most Valuable
Question: The method achieves better accuracy but is not uniformly faster or lower-memory than all coarse-to-fine baselines. Could the authors clarify the intended operating regime where RCCR-MVS offers the strongest practical advantage?
RCCR-MVS targets accurate high-resolution reconstruction with bidirectional context under a practical GPU budget, rather than minimum latency or memory alone.
Under the controlled architectural protocol, D improves MAE from F's 0.0977 to 0.0913 m, with 47.63% fewer trainable parameters, 21.76% less training memory, and 15.68% shorter inference time. Inference memory is 8.70% higher. At in the separate resolution sweep, D takes 3.4563 s versus F's 6.0776 s: 43.13% less time, with 11.13% more memory. The asymmetric design therefore becomes particularly useful when duplicating full multi-scale recurrence is expensive.
Against forward-only A, D trades 59.29% more inference time for 7.50% lower MAE; against F, it improves accuracy while reducing parameters, training memory, and runtime. Each comparison uses paired measurements within one sweep.
At system level, Tables 1/8 report the lowest MAE and highest accuracy on WHU/Luojia for both view counts, and seven of eight best metrics across München/Vaihingen. Table 9 separately reports complete-model costs: at , RCCR-MVS takes 11.2907 s versus CasREDNet's 15.8728 s. Other methods can be faster; these results make the accuracy–resource trade-offs explicit.
We invite you to inspect the new comparisons in Appendix A.9, Figures 14–16: WHU/Luojia depth maps (Figure 14) and native/resized MultiResXD examples (Figures 15/16). These illustrate structural consistency across the displayed scenes and scales, alongside the existing full-image comparisons (Figures 12–13).
Revision: Section 4.3/Table 4 and Appendix A.13/Table 11 quantify core trade-offs; Section 4.2/Table 1, Appendix A.8/Table 8, and A.11/Table 9 report full-system results. Section 4.2 now cites new comparisons in Appendix A.9, Figures 14–16; the error-trend figure in A.10 is now Figure 17. Section 3.1 and Appendix A.5 clarify storage.
Thanks for the clarification. I will maintain my recommendation.
Response to 948J (Part 1/2): Conceptual Contribution and Context-Width Sensitivity
Thank you for recognizing RCCR's value. We invite you to review the latest manuscript, especially the new visuals in Appendices A.9 and A.15. This part addresses W1/Q1 and Q2; Part 2 covers module roles and practical use. References use revised numbering.
[W1&Q1] Beyond Bidirectionality: Compressed Context Conditions State Updates
Both design choices matter: compression makes reverse evidence economical, while internal conditioning makes it effective. The lightweight encoder summarizes the current and subsequent hypotheses into sd; this context enters both gates and candidate states at every scale of the forward recurrent encoder–decoder. Consequently, reverse evidence changes how recurrent memory is updated, rather than only being combined with an independently computed output.
The decoder fuses updated multi-scale states to produce a logit without writing fused features back into recurrent memory. Appendix A.2 specifies an eight-hidden-channel reverse encoder and default four-channel context, avoiding duplication of the multi-scale main regularizer.
The new controls distinguish the integration mechanism from compression alone. With discrepancy and CRE/SES disabled, internal conditioning C achieves 0.0928 m MAE, compared with 0.0976 m for decoder-only fusion of the same compressed context E and 0.0977 m for symmetric independent RED branches F. C/E preserves the reverse representation and changes its integration point; the 4.92% MAE improvement shows that the benefit depends on how reverse evidence is used. C also outperforms the larger symmetric architecture. Adding discrepancy produces D's 0.0913 m, a further 1.62% reduction.
Compact reverse evidence conditions the state transitions, rather than merely supplying output features. Figure 8 and new Appendix A.15, Figures 18–21, show responses in earlier and later logits after a slice perturbation, unlike the forward-only baseline. The added examples span WHU-MVS, MultiResXD-MVS, Luojia-MVS, München, and Vaihingen, connecting reverse information flow with improved posteriors and lower ROI errors.
Revision: Section 3.1, Eqs. (1)–(4), Figure 3, and Appendix A.2 clarify the mechanism. Section 4.3, Table 4 provides controlled comparisons. Figure 8 is now complemented by Appendix A.15, Figures 18–21, with a new cross-reference in Section 4.3.
[Q2] Accuracy–Memory Trade-offs Across Context Widths
RCCR's benefit persists across every tested width, including one-channel reverse context. Table 5 evaluates K∈{1,2,4,8,16} on the WHU-MVS five-view test set with CRE/SES disabled:
All widths improve the three accuracy metrics over forward-only A (0.0987 m/97.78%/98.31%). Accuracy improves from K=1 to K=4, but larger widths consume more memory without further accuracy gains. Relative to K=16, the selected K=4 achieves better accuracy with 31.06% lower inference memory and 7.01% lower training memory. For tighter budgets, K=1,2 retain an accuracy improvement at lower memory cost. Thus, the usefulness of reverse conditioning is not confined to a single width.
Figure 6 provides per-epoch test curves consistent with the tabulated results. Validation data alone determined K and checkpoint selection; test curves did not guide selection. Accuracy uses trained models; memory uses separate random-initialization/synthetic-input profiles: one RTX 5090, FP32 (AMP/TF32 off), batch size 1, five views, 768×384, and [48,32,8] hypotheses (Appendix A.5).
Revision: We have added the complete width sweep and epoch-wise curves to Section 4.3, Table 5 and Figure 6, together with their accuracy–memory analysis. Appendix A.5 specifies validation-only model selection and the separate resource-profiling protocol.