acceptodds
Under review as a conference paper at ICLR 2027

RCCR-MVS: Reverse Context Conditioned Regularizer for High-Resolution Aerial MVS

Abstract

While memory-efficient slice-wise recurrent regularization is essential for High-Resolution Aerial Multi-View Stereo (MVS), existing approaches either restrict state updates to unidirectional contexts or lack deep contextual integration across independent bidirectional passes. To use bidirectional evidence more effectively under controlled memory budgets, we propose a highly versatile Reverse-Context-Conditioned Regularizer (RCCR) that performs strongly across both single-stage and coarse-to-fine architectures. This asymmetric regularizer uses a lightweight reverse encoder to extract compressed context, which conditions a multi-scale forward recurrent encoder–decoder via a customized cell that uses an explicit evidence-state discrepancy mechanism for optimized sequential gating. We deploy this core mechanism using an efficient two-pass inference strategy that successfully avoids full volumetric storage. To formulate our complete RCCR-MVS, we augment its cascaded deployments with two complementary modules: Coverage-risk-guided Range Expansion (CRE) to prevent true depths from falling outside cascaded search intervals, and Spatial Evidence Support (SES) to mitigate noisy initial matching evidence. Extensive experiments demonstrate that RCCR-MVS achieves state-of-the-art performance among high-resolution aerial MVS methods on WHU-MVS and LuoJia-MVS while maintaining a favorable accuracy–resource trade-off. Furthermore, it demonstrates strong cross-domain and cross-resolution generalization on our newly constructed MultiResXD-MVS dataset and on the München and Vaihingen full-image scenarios.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Aerial Multi View Stereo, Bidirectional Context, Cost Regularization, High-Resolution Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_948JOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes RCCR-MVS, a memory-efficient recurrent framework for high-resolution aerial multi-view stereo.

The core contribution is a Reverse-Context-Conditioned Regularizer (RCCR), which uses a lightweight reverse recurrent encoder to summarize evidence from current and subsequent depth hypotheses into a compressed reverse context. This context is then used to condition the gates and candidate states of a multi-scale forward recurrent encoder-decoder. The goal is to exploit bidirectional depth-hypothesis information without storing a full 4D cost volume.

The complete cascaded framework further introduces Coverage-Risk-Guided Range Expansion (CRE) to adaptively widen depth-search intervals when the true depth may fall outside the baseline range, and Spatial Evidence Support (SES) to enhance noisy matching evidence before recurrent regularization. Experiments on WHU-MVS, Luojia-MVS, München, Vaihingen, and the proposed MultiResXD-MVS dataset show strong accuracy, efficiency, cross-domain, and cross-resolution performance.

Critical strengths
  1. The core RCCR architecture is a meaningful improvement over existing slice-wise recurrent regularization. The asymmetric design avoids duplicating the full forward regularizer while still allowing information from future depth hypotheses to influence current updates. This provides a clear accuracy-memory trade-off.

  2. The experimental validation is strong. The paper evaluates both single-stage and coarse-to-fine variants, includes component ablations, computational efficiency analysis, cross-domain testing, and cross-resolution generalization. The results are consistently competitive and often state of the art.

Critical weaknesses
  1. The broader methodological novelty is somewhat limited. Bidirectional recurrent MVS and global cross-hypothesis regularization have already been studied, and the main advance is a more efficient asymmetric way to inject compressed reverse context. While useful, the contribution remains fairly specific to slice-wise aerial MVS regularization.

  2. CRE and SES appear more like effective engineering extensions than major conceptual contributions. CRE is closely related to uncertainty-guided adaptive depth-range adjustment, while SES resembles geometry-aware local cost aggregation. Their empirical benefits are clear, but they do not substantially broaden the conceptual contribution beyond the RCCR core.

Questions for authors
  1. Could the authors clarify the conceptual distinction between RCCR and prior bidirectional recurrent MVS methods beyond computational asymmetry? In particular, is the main advantage the compressed reverse representation, the way it conditions the forward gates, or both?

  2. How sensitive is RCCR to the reverse-context channel dimension K? Since K=4 is used throughout, it would be useful to understand whether the performance-memory trade-off is stable across different compression levels.

  3. Could the authors comment on whether CRE and SES are necessary for the main contribution, or whether RCCR alone is intended to be the central transferable idea? The ablation suggests RCCR provides most of the conceptual gain, while CRE and SES mainly improve the final aerial-MVS pipeline.

  4. The method achieves better accuracy but is not uniformly faster or lower-memory than all coarse-to-fine baselines. Could the authors clarify the intended operating regime where RCCR-MVS offers the strongest practical advantage?

3 comments
AuthorsOpenReview ↗

Response to 948J (Part 1/2): Conceptual Contribution and Context-Width Sensitivity

Thank you for recognizing RCCR's value. We invite you to review the latest manuscript, especially the new visuals in Appendices A.9 and A.15. This part addresses W1/Q1 and Q2; Part 2 covers module roles and practical use. References use revised numbering.

[W1&Q1] Beyond Bidirectionality: Compressed Context Conditions State Updates

Question: Could the authors clarify the conceptual distinction between RCCR and prior bidirectional recurrent MVS methods beyond computational asymmetry? In particular, is the main advantage the compressed reverse representation, the way it conditions the forward gates, or both?

Both design choices matter: compression makes reverse evidence economical, while internal conditioning makes it effective. The lightweight encoder summarizes the current and subsequent hypotheses into sds_d; this context enters both gates and candidate states at every scale of the forward recurrent encoder–decoder. Consequently, reverse evidence changes how recurrent memory is updated, rather than only being combined with an independently computed output.

The decoder fuses updated multi-scale states to produce a logit without writing fused features back into recurrent memory. Appendix A.2 specifies an eight-hidden-channel reverse encoder and default four-channel context, avoiding duplication of the multi-scale main regularizer.

The new controls distinguish the integration mechanism from compression alone. With discrepancy and CRE/SES disabled, internal conditioning C achieves 0.0928 m MAE, compared with 0.0976 m for decoder-only fusion of the same compressed context E and 0.0977 m for symmetric independent RED branches F. C/E preserves the reverse representation and changes its integration point; the 4.92% MAE improvement shows that the benefit depends on how reverse evidence is used. C also outperforms the larger symmetric architecture. Adding discrepancy produces D's 0.0913 m, a further 1.62% reduction.

Compact reverse evidence conditions the state transitions, rather than merely supplying output features. Figure 8 and new Appendix A.15, Figures 18–21, show responses in earlier and later logits after a slice perturbation, unlike the forward-only baseline. The added examples span WHU-MVS, MultiResXD-MVS, Luojia-MVS, München, and Vaihingen, connecting reverse information flow with improved posteriors and lower ROI errors.

Revision: Section 3.1, Eqs. (1)–(4), Figure 3, and Appendix A.2 clarify the mechanism. Section 4.3, Table 4 provides controlled comparisons. Figure 8 is now complemented by Appendix A.15, Figures 18–21, with a new cross-reference in Section 4.3.

[Q2] Accuracy–Memory Trade-offs Across Context Widths

Question: How sensitive is RCCR to the reverse-context channel dimension K? Since K=4 is used throughout, it would be useful to understand whether the performance-memory trade-off is stable across different compression levels.

RCCR's benefit persists across every tested width, including one-channel reverse context. Table 5 evaluates K∈{1,2,4,8,16}K\in\{1,2,4,8,16\} on the WHU-MVS five-view test set with CRE/SES disabled:

KMAE (m)<0.6 m (%)<3 intervals (%)Train GiBInfer GiB
10.094897.9198.418.77980.2433
20.093797.9498.428.82290.2547
40.091398.0198.478.96870.2761
80.095397.9098.409.15500.3172
160.092097.9898.459.64430.4005

All widths improve the three accuracy metrics over forward-only A (0.0987 m/97.78%/98.31%). Accuracy improves from K=1K=1 to K=4K=4, but larger widths consume more memory without further accuracy gains. Relative to K=16K=16, the selected K=4K=4 achieves better accuracy with 31.06% lower inference memory and 7.01% lower training memory. For tighter budgets, K=1,2K=1,2 retain an accuracy improvement at lower memory cost. Thus, the usefulness of reverse conditioning is not confined to a single width.

Figure 6 provides per-epoch test curves consistent with the tabulated results. Validation data alone determined KK and checkpoint selection; test curves did not guide selection. Accuracy uses trained models; memory uses separate random-initialization/synthetic-input profiles: one RTX 5090, FP32 (AMP/TF32 off), batch size 1, five views, 768×384768\times384, and [48,32,8][48,32,8] hypotheses (Appendix A.5).

Revision: We have added the complete width sweep and epoch-wise curves to Section 4.3, Table 5 and Figure 6, together with their accuracy–memory analysis. Appendix A.5 specifies validation-only model selection and the separate resource-profiling protocol.

AuthorsOpenReview ↗

Response to 948J (Part 2/2): Core Contribution, Extensions, and Practical Advantages

This part addresses W2/Q3 and Q4: module roles and practical use.

[W2&Q3] RCCR Is the Core Contribution; CRE and SES Are Complementary

Question: Could the authors comment on whether CRE and SES are necessary for the main contribution, or whether RCCR alone is intended to be the central transferable idea? The ablation suggests RCCR provides most of the conceptual gain while CRE and SES mainly improve the final aerial-MVS pipeline.

RCCR is the central regularization idea and requires neither CRE nor SES. Section 3.2 explicitly establishes this hierarchy. Table 3 shows RCCR-only improves all eight metrics across WHU-MVS, Luojia-MVS, München tiles, and resized MultiResXD-MVS, including 7.50%/34.62% lower MAE on WHU/resized MultiResXD.

Table 10 also shows improvements beyond selected examples and truncated errors:

SettingCasREDNet MAE (m)RCCR-only MAE (m)Improved (%)
WHU-MVS, five views0.14740.141197.94
MultiResXD resized 1×1.26210.946487.76
MultiResXD native 4×2.35612.176895.92

Here MAEs are untruncated; Improved counts images with lower truncated MAE than CasREDNet. RCCR-only also improves PGTP_{\mathrm{GT}} throughout. Appendix A.15 (new Figures 18–21) adds RCCR-only posteriors, ROI errors, and depthwise-influence analyses across five aerial datasets, complementing Table 10.

CRE and SES improve distinct parts of the complete aerial pipeline. CRE adjusts hypothesis coverage before cost construction; SES refines spatial matching evidence before regularization. In Table 12, adaptive CRE reaches 0.08563 m MAE versus 0.09297 m for training-set-calibrated fixed expansion and 0.08884 m for fixed 1.5×1.5\times expansion. This supports spatially adaptive allocation rather than uniform widening. In Table 13, Full SES reduces edge-region MAE to 0.55749 m from 0.57013 m without geometry, supporting the value of geometric compatibility near boundaries.

The complete model further lowers final MAE and outlier rates throughout Table 10. Its lower WHU first-stage PGTP_{\mathrm{GT}} than RCCR-only (81.67% versus 82.39%) does not contradict this: posterior mass is an intermediate diagnostic, not a monotonic predictor of final error. RCCR changes regularization; CRE/SES strengthen its cascaded deployment.

Revision: Section 3.2 states the hierarchy. Section 4.3, Table 3/Figure 8 and Appendix A.12, Table 10 establish RCCR-only effectiveness. The new Appendix A.15 (Figures 18–21) extends the visual evidence. Appendix A.14, Tables 12–13 and Appendices A.3–A.4 cover the extensions.

[Q4] Where the Accuracy–Resource Trade-off Is Most Valuable

Question: The method achieves better accuracy but is not uniformly faster or lower-memory than all coarse-to-fine baselines. Could the authors clarify the intended operating regime where RCCR-MVS offers the strongest practical advantage?

RCCR-MVS targets accurate high-resolution reconstruction with bidirectional context under a practical GPU budget, rather than minimum latency or memory alone.

Under the controlled architectural protocol, D improves MAE from F's 0.0977 to 0.0913 m, with 47.63% fewer trainable parameters, 21.76% less training memory, and 15.68% shorter inference time. Inference memory is 8.70% higher. At 3072×15363072\times1536 in the separate resolution sweep, D takes 3.4563 s versus F's 6.0776 s: 43.13% less time, with 11.13% more memory. The asymmetric design therefore becomes particularly useful when duplicating full multi-scale recurrence is expensive.

Against forward-only A, D trades 59.29% more inference time for 7.50% lower MAE; against F, it improves accuracy while reducing parameters, training memory, and runtime. Each comparison uses paired measurements within one sweep.

At system level, Tables 1/8 report the lowest MAE and highest <0.6 m<0.6\,\mathrm{m} accuracy on WHU/Luojia for both view counts, and seven of eight best metrics across München/Vaihingen. Table 9 separately reports complete-model costs: at 3536×38883536\times3888, RCCR-MVS takes 11.2907 s versus CasREDNet's 15.8728 s. Other methods can be faster; these results make the accuracy–resource trade-offs explicit.

We invite you to inspect the new comparisons in Appendix A.9, Figures 14–16: WHU/Luojia depth maps (Figure 14) and native/resized MultiResXD examples (Figures 15/16). These illustrate structural consistency across the displayed scenes and scales, alongside the existing full-image comparisons (Figures 12–13).

Revision: Section 4.3/Table 4 and Appendix A.13/Table 11 quantify core trade-offs; Section 4.2/Table 1, Appendix A.8/Table 8, and A.11/Table 9 report full-system results. Section 4.2 now cites new comparisons in Appendix A.9, Figures 14–16; the error-trend figure in A.10 is now Figure 17. Section 3.1 and Appendix A.5 clarify storage.

Thanks for the clarification. I will maintain my recommendation.

Reviewer_fSV3OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes RCCR-MVS, a memory-efficient recurrent framework for high-resolution aerial multi-view stereo. Building upon slicewise RNN-based MVS architectures, the authors introduce a RCCR, which employs a lightweight reverse encoder to extract compressed contextual information and condition the hidden-state updates of a multi-scale forward recurrent network. The framework is further enhanced with CRE and SES to improve depth hypothesis coverage and spatial cost aggregation.

Extensive experiments on aerial dataset demonstrate competitive depth estimation accuracy, cross-domain generalization, and a favorable accuracy–resource trade-off.

Critical strengths
  • The paper addresses a meaningful limitation of existing recurrent MVS methods: unidirectional processing restricts information propagation across depth hypotheses. The proposed asymmetric bidirectional architecture is a reasonable and interesting approach to incorporating additional context while avoiding full cost-volume storage. The overall framework is clearly formulated and technically well motivated.
  • The method demonstrates consistent improvements over recurrent MVS baselines across multiple datasets. The experiments also cover different input resolutions, cross-domain generalization, and computational efficiency, providing substantial empirical evidence for the approach.
Critical weaknesses
  • Although the overall RCCR module is shown to be effective, the paper does not sufficiently isolate the contributions of its specific architectural choices. In particular, it remains unclear how much improvement comes from bidirectional information propagation itself, the asymmetric allocation of computation, or the proposed evidence–state discrepancy mechanism. Comparisons against symmetric bidirectional RNNs or simpler reverse-context fusion strategies would help establish the necessity and novelty of the proposed design.
  • The proposed regularization mechanism addresses a general limitation of RNN-based MVS rather than an issue specific to aerial imagery. However, all evaluations focus on aerial MVS datasets. Experiments on conventional benchmarks such as DTU or Tanks and Temples would provide stronger evidence of its general applicability.
  • Memory efficiency is a central motivation for recurrent MVS. While the proposed two-pass inference avoids storing full high-dimensional cost volumes, it introduces additional computation and cached reverse contexts. The training pipeline also differs from inference by saving and reusing matching costs. A more systematic analysis of training and inference memory consumption, particularly as image resolution and the number of depth hypotheses increase, would better substantiate the claimed efficiency advantages.
Questions for authors
  • Can the authors provide comparisons with symmetric bidirectional RNNs or alternative asymmetric configurations? In particular, would reversing the roles of the forward and backward branches lead to similar performance?
  • How much of the improvement comes specifically from the proposed evidence–state discrepancy mechanism? Would a standard ConvGRU conditioned on the same compressed reverse context achieve comparable results?
  • Figure 2 indicates that matching costs are saved and reused during training, whereas inference employs two-pass recomputation. Could the authors clarify whether the cached costs and recurrent activations are retained in GPU memory during training? How do peak training memory and inference memory scale with the number of depth hypotheses and input resolution?
  • Since the proposed regularizer is not inherently aerial-specific, can the authors provide evidence of its effectiveness on conventional MVS benchmarks, such as DTU or Tanks and Temples? Alternatively, what characteristics of aerial imagery specifically motivate the proposed architecture?
4 comments
AuthorsOpenReview ↗

Response to fSV3 (Part 1/2): Architectural Alternatives and Discrepancy Controls

Thank you for recognizing our motivation and experimental results. We invite you to review the latest revised manuscript. Part 1 jointly addresses W1 with Q1/Q2; Part 2 addresses resource scaling and aerial motivation. References use revised numbering unless explicitly marked as original.

[W1&Q1] Internal Conditioning, Symmetric Alternatives, and Scan Orientation

Question: Can the authors provide comparisons with symmetric bidirectional RNNs or alternative asymmetric configurations? In particular, would reversing the roles of the forward and backward branches lead to similar performance?

We added the requested architectural controls, and RCCR's gain is not explained by bidirectional access alone. The A–G ablation disables CRE and SES throughout. To separate context integration from discrepancy, C/E/F all omit the discrepancy term:

VariantIntegration of reverse evidenceMAE (m)Trainable params. (M)
CConditions recurrent gates and candidate states0.09281.4372
ESame compressed context, decoder-output fusion0.09761.4239
FIndependent equal-width RED branches, linear fusion0.09772.7767

C improves MAE by 4.92% over E and 5.02% over F. C/E uses the same lightweight reverse representation, directly testing internal conditioning against output-only fusion. F supplies bidirectional evidence through two full branches, yet is less accurate despite its larger parameter count. Together, these controls show why integrating reverse evidence into recurrent updates is more effective than the tested late-fusion designs. D then adds discrepancy to C and achieves the best MAE, 0.0913 m.

We also tested the opposite scan orientation. G reverses both scans while preserving D's lightweight auxiliary encoder and multi-scale main regularizer. It yields 0.0937 m, compared with 0.0913 m for D and 0.0987 m for forward-only A. Thus, both orientations improve over A, with the default giving the stronger result. D/G have identical trainable parameter counts (1.4541M) and inference peak memory (0.2761 GiB). This experiment exchanges the directions in which the two branches operate, while retaining their asymmetric capacities.

The accuracy and resource results jointly support the design: D is 6.55% more accurate in MAE than F, with 47.63% fewer trainable parameters. RCCR uses the compact branch to provide context for the main recurrence instead of duplicating the main regularizer.

Revision: We have added all A–G accuracy and efficiency results to Section 4.3, Table 4, with variant definitions in the caption and analysis under Architectural Ablation of RCCR. Section 3.1, Eqs. (1)–(4), and Appendix A.2 clarify the conditioning mechanism and asymmetric branch architecture.

[W1&Q2] The Additional Benefit of Evidence–State Discrepancy

Question: How much of the improvement comes specifically from the proposed evidence-state discrepancy mechanism? Would a standard ConvGRU conditioned on the same compressed reverse context achieve comparable results?

The conditioned ConvGRU is already effective, and discrepancy provides a further isolated improvement. C retains exactly the same compressed context in both gates and candidate states as D; only the discrepancy input is removed. The WHU five-view comparison is:

VariantMAE (m)<0.6 m (%)<3 intervals (%)
C: conditioned ConvGRU0.092897.9398.40
D: RCCR with discrepancy0.091398.0198.47

This is a 1.62% MAE reduction, with only 1.18% more trainable parameters and unchanged measured inference peak memory. The broader A/B comparison also shows a discrepancy benefit without reverse conditioning: MAE decreases from 0.0987 to 0.0966 m. Conversely, A→C improves MAE by 5.98%, showing that reverse conditioning supplies the larger gain, while discrepancy complements it.

The term ed=xd−ψe(hd−1)e_d=x_d-\psi_e(h_{d-1}) exposes the evidence–memory difference to the update/reset gates. It is a structured gate parameterization rather than an extra source of independent evidence or a supervised prediction error. The controlled results establish its practical value in the proposed recurrence, while distinguishing it from the contribution of reverse context itself.

Revision: We have clarified the discrepancy role in Section 3.1, Eqs. (2)–(4), and added the controlled A/B/C/D comparisons in Section 4.3, Table 4. The accompanying analysis separates reverse conditioning from the additional discrepancy gain.

AuthorsOpenReview ↗

Response to fSV3 (Part 2/2): Resource Scaling and Aerial Motivation

This part addresses W3/Q3 and W2/Q4.

[W3&Q3] Training Storage, Two-Pass Inference, and Resource Scaling

Question: Figure 2 indicates that matching costs are saved and reused during training, whereas inference employs two-pass recomputation. Could the authors clarify whether the cached costs and recurrent activations are retained in GPU memory during training? How do peak training memory and inference memory scale with the number of depth hypotheses and input resolution?

Yes. Training retains matching costs and recurrent activations on GPU, reusing costs across scans without activation checkpointing or CPU offloading. Inference constructs costs in two passes, retaining KDHWKDHW context elements per sample and stage, scalar logits, and hypothesis grids instead of the full high-dimensional cost volume.

Each cell reports D/F: RCCR-only/symmetric bidirectional. Both use five views, batch size 1 and FP32, with CRE/SES disabled (K=4K=4 for D).

Resolution scaling. Stage hypotheses are fixed at [48,32,8][48,32,8] (Appendix A.13, Table 11(a)).

Input resolutionTraining memory (GiB)Inference memory (GiB)Inference time (s/image)
384×1922.2538 / 2.86880.0806 / 0.07830.2095 / 0.2516
576×2884.9959 / 6.38570.1606 / 0.15090.2211 / 0.2538
768×3848.9687 / 11.46300.2761 / 0.25400.2345 / 0.2858
1024×51215.5771 / 19.93780.4743 / 0.43270.2432 / 0.3044
1536×768OOM / OOM1.0508 / 0.95130.6263 / 1.0022
2304×1152OOM / OOM2.3513 / 2.11981.2748 / 2.0723
3072×1536OOM / OOM4.1608 / 3.74413.4563 / 6.0776

D lowers training memory at every feasible resolution and inference time throughout. At 3072×15363072\times1536, inference takes 43.13% less time, with 11.13% more memory. Both reach training OOM (out of memory) at 1536×7681536\times768, while inference remains feasible at the largest tested input.

Depth-count scaling. Only D1D_1 varies at 768×384768\times384, with D2=32,D3=8D_2=32,D_3=8 (Table 11(b)).

D1D_1Training memory (GiB)Inference memory (GiB)Inference time (s/image)
248.0664 / 10.31910.2762 / 0.25490.1769 / 0.2161
488.9687 / 11.46300.2761 / 0.25400.2356 / 0.2854
9610.7702 / 13.74320.2752 / 0.25490.3477 / 0.4256
19214.3779 / 18.31210.3318 / 0.33700.5823 / 0.7003

D uses less training memory and inference time at every depth count. Training memory and runtime grow with D1D_1; the inference-memory plateau does not imply constant storage, since contexts, logits, and grids remain depth-dependent.

Appendix A.5 details synthetic-input profiling on one RTX 5090, synchronized timing, and peak-memory accounting. Architectural, resolution, and depth-count sweeps are measured separately; efficiency comparisons use paired records within each sweep.

Revision: Section 3.1 and Appendix A.5 clarify storage and profiling. Appendix A.13, Table 11(a)/(b) reports both complete sweeps. Section 4.3, Table 4 and Appendix A.11, Table 9 provide architectural and complete-system measurements separately.

[W2&Q4] Why High-Resolution Aerial Reconstruction Motivates RCCR

Question: Since the proposed regularizer is not inherently aerial-specific, can the authors provide evidence of its effectiveness on conventional MVS benchmarks, such as DTU or Tanks and Temples? Alternatively, what characteristics of aerial imagery specifically motivate the proposed architecture?

Large aerial image grids make preserving spatial context under GPU limits especially important. Building boundaries, vegetation, and broad scene coverage call for multi-scale spatial processing; duplicating that processing in two full recurrent branches is costly. RCCR instead uses a compact reverse representation to guide the main multi-scale updates.

MultiResXD-MVS separates native coverage, common-center evaluation, and fixed-coverage resizing. Under common-center evaluation, the complete model's MAE falls from 1.2656 m at 1×1\times to 0.7573 m at 4×4\times, supporting the value of context. The profiles quantify its resource cost. München/Vaihingen and MultiResXD use WHU-trained checkpoints without target-domain adaptation.

We have not added DTU or Tanks and Temples experiments. The regularization principle is not aerial-specific, but its non-aerial effectiveness remains to be evaluated. The current evidence establishes cross-domain aerial accuracy and high-resolution inference benefits, which are the paper's intended scope.

Revision: Section 1 motivates the aerial setting; Section 4.2, Table 2 and Appendix A.7 define the coverage/resolution experiments; Appendix A.13 quantifies scaling. We have updated Section 5 to explicitly identify conventional non-aerial benchmarks as requiring further evaluation.

Thank you for the detailed responses and the additional experiments. I appreciate the authors' efforts to address my concerns, particularly the new architectural ablations that clarify the contributions of reverse-context conditioning, asymmetric design, and the evidence–state discrepancy mechanism. The additional memory and computational scaling analyses are also helpful.

These results substantially strengthen the experimental justification for the proposed approach. While the generality beyond aerial MVS remains to be explored, I am satisfied with the clarifications and will maintain my positive rating.

Thank you for your thoughtful follow-up and continued positive assessment. We are pleased that the additional architectural ablations and resource-scaling analyses have addressed your concerns. Your suggestions have helped us clarify the contributions of RCCR’s design and strengthen the experimental evaluation. We agree that evaluating RCCR beyond aerial MVS remains an important direction for future work. We sincerely appreciate your time, careful review, and constructive engagement throughout the discussion.

Reviewer_jCJVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents RCCR-MVS for high-resolution aerial multi-view stereo. A lightweight reverse recurrent encoder produces compressed context that conditions a forward recurrent encoder–decoder. The cascaded system additionally incorporates coverage-risk-guided range expansion (CRE) and spatial evidence support (SES). Experiments cover standard aerial benchmarks, cross-domain evaluation, and a new dataset, MultiResXD-MVS, designed to separate changes in image resolution from changes in spatial coverage.

Critical strengths
  1. The asymmetric architecture offers a practical way to incorporate reverse-depth context. The paper explicitly accounts for the depth-dependent context cache rather than claiming constant-memory inference.

  2. The cascaded system achieves strong results across several datasets. RCCR-only also improves over the baseline across all settings in Table 3, supporting the usefulness of the core regularizer independently of CRE and SES.

  3. The evaluation is broader than a standard benchmark comparison. The fixed-center protocol is particularly useful for examining the effect of additional spatial context. The appendix further provides full-image efficiency measurements and targeted CRE/SES ablations.

Critical weaknesses
  1. The central architectural contribution is not sufficiently isolated. Table 3 evaluates RCCR as a whole, leaving unclear how much comes from reverse conditioning versus the proposed discrepancy mechanism. 2.Important details needed to reproduce and interpret the comparisons are missing. The manuscript specifies hardware and hypothesis counts, but not the optimizer, learning-rate schedule, training duration, checkpoint selection, or which training dataset/checkpoint is used for each cross-domain result. It is also unclear which baselines were retrained under the stated settings. The exact CRE target construction and several architectural hyperparameters remain unspecified. These omissions make it difficult to assess whether the reported differences arise from the regularizer under comparable training conditions.
Questions for authors

1.How does RCCR compare with a standard reverse-context-conditioned ConvGRU after removing only the discrepancy term?

2.Which checkpoints and training protocols produced the cross-domain results, and were the reproduced baselines trained comparably?

3.Can the authors reconcile the shared-input results in Table 2 and the SES results across Tables 3 and 10, and provide untruncated error statistics?

5 comments
AuthorsOpenReview ↗

Response to jCJV (Part 1/3): Isolating Reverse Conditioning and Discrepancy

Thank you for recognizing RCCR's effectiveness and our evaluation. We invite you to review the latest revised manuscript, especially the new depth-map comparisons in Appendix A.9 (Figures 14–16) and RCCR-only analyses in Appendix A.15 (Figures 18–21). This part addresses W1/Q1; Parts 2/3 cover reproducibility and evaluation consistency. References follow the latest revision unless marked as original.

[W1&Q1] Separating the Contributions of Reverse Conditioning and Discrepancy

Question: How does RCCR compare with a standard reverse-context-conditioned ConvGRU after removing only the discrepancy term?

RCCR improves over the requested reverse-context-conditioned ConvGRU even when the reverse information and forward architecture are held fixed. To address the concern that the original component ablation treated RCCR as a whole, we added controlled variants A–G. All use the same WHU-MVS five-view evaluation, with CRE and SES disabled. The relevant controls in Table 4 are:

VariantReverse conditioningDiscrepancyMAE (m)<0.6 m (%)<3 intervals (%)
ANoNo0.098797.7898.31
BNoYes0.096697.8398.34
CYesNo0.092897.9398.40
D: RCCR-onlyYesYes0.091398.0198.47

The requested comparison is C versus D. C retains the same compressed reverse context in both gates and candidate states, along with the same multi-scale forward architecture. Only the discrepancy input is removed. Restoring it reduces MAE by 1.62% and improves both threshold accuracies. Trainable parameters increase from 1.4372M to 1.4541M (1.18%), while inference peak memory remains 0.2761 GiB. Thus, the gain is obtained with a small parameter increase rather than a wider reverse branch or an additional refinement module.

The A/B/C controls also clarify the broader concern in W1. Reverse conditioning alone reduces MAE by 5.98% (A→C), whereas discrepancy alone gives 2.13% (A→B). Combining them yields the best result, 7.50% below A. These comparisons separate the larger gain from reverse-context conditioning from the complementary gain of discrepancy.

The discrepancy ed=xd−ψe(hd−1)e_d=x_d-\psi_e(h_{d-1}) explicitly presents the difference between current evidence and projected recurrent memory to the gates. It is an internal parameterization, not a separately supervised depth error. C/D therefore tests whether this representation improves the use of information already available to the conditioned recurrence.

We further compared where reverse evidence is integrated. C outperforms decoder-only fusion of the same compressed context (E: 0.0976 m) and independent symmetric bidirectional branches with linear output fusion (F: 0.0977 m). The 4.92% C/E gain supports conditioning recurrent updates rather than merely providing reverse evidence at the output. Figure 8 complements these controls with posterior and depthwise-influence visualizations, while Appendix A.12, Table 10 reports aggregate improvements beyond the selected examples.

New cross-dataset visualizations. The analyses in Appendix A.15 (Figures 18–21) extend the baseline/RCCR-only comparison to additional WHU-MVS, MultiResXD-MVS, Luojia-MVS, München, and Vaihingen scenes. The illustrated examples show increased probability mass near ground truth, lower final ROI errors, and nonzero later-to-earlier influence. The slice-26 perturbation responses demonstrate that later evidence reaches earlier outputs, complementing the controlled architectural results.

Together, the experiments identify internal reverse conditioning as the principal architectural gain and discrepancy as a lightweight, complementary improvement.

Revision: Section 4.3, Table 4 gives all A–G accuracy/resource results. Section 3.1, Eqs. (1)–(4), and Appendix A.2 specify the mechanism. Figure 8 and Appendix A.12, Table 10 provide qualitative and aggregate evidence. We have added Appendix A.15, Figures 18–21, and its cross-reference in Section 4.3 to extend the RCCR-only visual analysis across datasets.

AuthorsOpenReview ↗

Response to jCJV (Part 2/3): Reproducible Training and Implementation

This part addresses W2/Q2, including training/checkpoint provenance and implementation details. All references follow the latest revised manuscript; the training and module specifications remain in Appendices A.2–A.5.

[W2&Q2] Comparable Training, Checkpoint Provenance, and Implementation

Question: Which checkpoints and training protocols produced the cross-domain results, and were the reproduced baselines trained comparably?

The cross-domain comparisons use each method's reproduced WHU-MVS checkpoint, and RCCR-MVS and the reproduced baselines share the training protocol. No target-domain adaptation or target-domain model selection is performed. The checkpoint corresponds to the evaluated view configuration; hyperparameters and checkpoints are selected using validation data only. Test data, including the per-epoch test curves in Figure 6, do not guide either selection.

Appendix A.5 now specifies RMSprop, initial learning rate 0.001, weight decay 10−510^{-5}, batch size 2, and 16 training epochs, with learning-rate halving at epochs 10, 12, and 14. Models evaluated in-domain on WHU-MVS and Luojia-MVS are trained on their respective training sets. Results taken from original papers are explicitly marked with an asterisk and gray shading in Tables 1 and 8; they are distinguished from our reproductions.

Training uses two 32 GB RTX 5090 GPUs. Coarse-to-fine methods use [48,32,8][48,32,8] hypotheses and interval ratios [4,2,1][4,2,1]. Most single-stage baselines use spatial scale 0.5 and interval factor 2.0; R-MVSNet uses 0.25 and 1.0. Resource profiling is a separate synthetic-input experiment with batch size 1, not the source of the accuracy results.

We have also filled the architectural and supervision gaps identified in W2. Appendix A.2 specifies the eight-channel reverse ConvGRU, 3×33\times3 kernel, default context width K=4K=4, four forward spatial levels, and independent regularizers across cascade stages. Appendix A.4 specifies the single directional CNN, evidence width J=4J=4, geometric threshold γ=1.0\gamma=1.0, residual strength η=0.1\eta=0.1, and zero-initialized output projection.

For CRE, Appendix A.3, Eqs. (15)–(20), defines targets and auxiliary losses using the center cc and half-range ρ\rho of the actual unexpanded hypotheses:

ξ=max⁡(∣zGT−c∣ρ−1,0),y=1[ξ>0],a⋆=clip⁡(ξ/0.5,0,1).\xi=\max\left(\frac{|z_{\mathrm{GT}}-c|}{\rho}-1,0\right),\quad y=\mathbf{1}[\xi>0],\quad a^\star=\operatorname{clip}(\xi/0.5,0,1).

The validity mask requires the dataset mask to exceed 0.5, finite zGT,c,ρz_{\mathrm{GT}},c,\rho, and ρ>10−6\rho>10^{-6}; it is independent of predicted risk and the evaluation cutoff. Risk uses BCEWithLogits; expansion amount uses smooth L1L_1 on overflow pixels; the product papa uses smooth L1L_1 against a⋆a^\star on valid pixels. Both smooth L1L_1 terms use transition parameter 0.1. Risk/product losses balance positive and negative class means within each process-local batch. A missing class is omitted; empty losses are zero. Each auxiliary loss is averaged over stages 2/3 and weighted by 0.05, separately from depth weights (0.5,1,2)(0.5,1,2).

The head combines six uncertainty/local-variation statistics with four projected reference channels and outputs two logits. Risk/amount biases are initialized to −4/0-4/0, with output-weight standard deviation 10−310^{-3}, giving an initial expansion factor near 1.0045. Head inputs and the applied sampling expansion are detached, so the head learns through its auxiliary losses. Ground-truth-derived targets are used only in training.

The fixed-matched control in Appendix A.14, Table 12 removes spatial adaptivity without test-set calibration. Its multipliers come from the epoch-14 Adaptive CRE model in evaluation mode without augmentation, pooling valid pixels across all 4,480 WHU-MVS five-view training samples. Separate stage-2/3 averages give 1.00565207/1.01816087, held fixed throughout training and inference of the control. They match average relative expansion on the training set, not the Adaptive CRE test-set means in Table 12.

These additions specify both the common comparison conditions and the module-specific details, directly addressing whether the reported improvements can be attributed to RCCR under comparable training.

Revision: We have consolidated training, checkpoint selection, sampling settings, and profiling in Appendix A.5, referenced from Section 4.1. Appendices A.2–A.4 give architectural details; Appendix A.3, Eqs. (15)–(20), and Section 3.3, Eq. (10), specify CRE supervision. We will add the full fixed-matched calibration procedure above to Appendix A.14 beside Table 12; its training-set calibration is already stated in the caption.

AuthorsOpenReview ↗

Response to jCJV (Part 3/3): Reconciled Results and Untruncated Evaluation

This part addresses Q3: corrected results, evaluation-mask definitions, and untruncated errors. References follow the latest revised manuscript.

[Q3] Reconciling Shared-Input Results and Verifying Gains Beyond Truncation

Question: Can the authors reconcile the shared-input results in Table 2 and the SES results across Tables 3 and 10, and provide untruncated error statistics?

The native/resized 4×4\times discrepancy arose from an evaluation-mask selection error in the affected resized runs. The inputs and predictions were identical, but the old entries were scored on different valid-pixel sets. We rescored them using the protocol-defined mask shared with native 4×4\times, without changing checkpoints, predictions, or aggregation:

MethodOriginal resized 4× MAE (m)Corrected resized/native 4× MAE (m)
RED-Net0.76190.7219
EG-MVSNet0.77960.7596
AdaMVS0.90410.8641

RCCR entries are unchanged. This was an evaluation error, not a difference between the two input families at their shared maximum size. Table 2 now agrees with the shared-input definition in Appendix A.7, Table 7. The discussion also reflects the corrected ranking: RED-Net leads the single-stage group at resized 4×4\times, while RCCR leads that group at resized 1×1\times–3×3\times and all same-center scales. RCCR remains best within the coarse-to-fine group throughout Table 2.

Native/same-center 1×1\times is a different case: its scoring masks intentionally differ. Native evaluation uses that scale's valid mask; same-center evaluation intersects validity across all four native scales within the common central window. Across 49 locations, these contain 9,740,235 and 9,738,084 pixels. Identical predictions can therefore produce different MAEs because the error sum and normalization are computed over different supports. Appendix A.7 now distinguishes the fixed underlying mask from the prediction-dependent 10 m10\,\mathrm{m} exclusion. This legitimate protocol difference should not be confused with the corrected 4×4\times mask-selection error.

The SES inconsistency was an outdated aggregate entry, not a different SES configuration. Original Table 3 retained 0.0891 m/98.04%, while original Table 10 reported Full SES with RCCR enabled and CRE disabled. They represent the same ablation. Table 3 now reports 0.0868 m/98.13%, matching the 0.08681 m MAE in revised Table 13 (original Table 10) after rounding. We apologize for these inconsistencies and have aligned the aggregate and detailed results.

We also standardized relative MAE reductions using the displayed four-decimal values: 7.50%/34.62% replaces 7.57%/34.63% for RCCR-only on WHU/resized MultiResXD. Those small differences may reflect earlier calculations with unrounded MAEs; the corresponding MAE results themselves are unchanged.

The accuracy gains remain when all valid-pixel errors are included. The original MAE<10\mathrm{MAE}_{<10} excludes errors of at least 10 m rather than clipping them. Revised Table 10 adds untruncated MAE over all valid pixels and the rate of errors ≥10 m\geq10\,\mathrm{m}, computed per image and averaged with equal image weights. Entries below are untruncated MAE (m) / outlier rate (%):

SettingCasREDNetRCCR-onlyComplete RCCR-MVS
WHU-MVS, five views0.1474 / 0.220.1411 / 0.220.1256 / 0.18
MultiResXD resized 1×1.2621 / 3.820.9464 / 2.980.8904 / 2.72
MultiResXD native 4×2.3561 / 7.962.1768 / 7.962.1272 / 7.67

RCCR-only reduces untruncated MAE by 4.27%, 25.01%, and 7.61%; the complete model achieves 14.79%, 29.45%, and 9.72% reductions and lowers all three outlier rates. RCCR-only does not lower every outlier rate, but its lower untruncated MAE shows that the gain persists without the cutoff. Table 10 also reports threshold accuracy, PGTP_{\mathrm{GT}}, and per-image improvements.

For visual inspection, new depth-map comparisons in Appendix A.9, Figures 15–16 illustrate expanding coverage and fixed-coverage resizing. The existing error-trend plot is now Appendix A.10, Figure 17. These complement the aggregate results; their example-specific MAEs are not dataset averages.

Revision: Table 2/Section 4.2 and Appendix A.7 provide corrected results and mask definitions; Table 3 agrees with Appendix A.14, Table 13. Appendix A.5 defines metrics; Appendix A.12, Table 10 gives the full aggregate evaluation. New depth-map comparisons are in Appendix A.9, Figures 15–16; the trend figure is now Figure 17 in A.10. Section 4.2 points readers to both.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary and Guide to the Revised Manuscript

We thank the AC and reviewers for their careful and constructive feedback. We appreciate their recognition of RCCR's practical asymmetric design, strong aerial-MVS results, and broad experimental evaluation. Their suggestions have helped us better isolate the architectural contributions, strengthen the evidence, and improve reproducibility.

We invite all reviewers to consult the latest revised manuscript, where key revisions are highlighted in blue. The overview below maps the main changes to the corresponding concerns; all section, table, and figure numbers refer to the revised version.

Major revisions

  • Controlled architectural ablations. Added variants A–G in Section 4.3, Table 4, covering reverse conditioning, discrepancy, decoder-only fusion, symmetric bidirectionality, and reversed scans, with accuracy and resource measurements. These controls distinguish the main gain from internal conditioning from the complementary discrepancy benefit. (jCJV: W1/Q1; fSV3: W1/Q1–Q2; 948J: W1/Q1.)

  • Context-width sensitivity. Added the K=1,2,4,8,16K=1,2,4,8,16 accuracy–memory comparison and epoch-wise test curves in Section 4.3, Table 5 and Figure 6. All tested widths improve over forward-only recurrence; K=4K=4 achieves the best measured accuracy without the extra memory of larger widths. (948J: Q2.)

  • Training and inference scalability. Added resolution and depth-count sweeps in Appendix A.13, Table 11(a)/(b), complementing Table 4. Appendix A.5 specifies GPU storage, two-pass inference, synchronized timing, peak-memory accounting, and paired comparisons within each sweep. The results make RCCR's benefits and costs relative to symmetric recurrence explicit. (fSV3: W3/Q3; 948J: Q4.)

  • Aggregate and untruncated evaluation. Added Appendix A.12, Table 10, combining truncated/untruncated MAE, outlier rates, first-stage PGTP_{\mathrm{GT}}, and per-image improvements. RCCR-only lowers both MAEs in all three settings, while the complete model further lowers final errors and outlier rates. These results extend the evidence beyond selected visual examples and error truncation. (jCJV: Q3; 948J: W2/Q3.)

  • Reproducibility and implementation details. Expanded Appendices A.2–A.5 and Section 3.3 with layer configurations, initialization, CRE targets and losses, training schedules, validation-only model selection, and cross-domain checkpoint provenance. Reproduced baselines are distinguished from quoted results, and test data do not guide hyperparameter or checkpoint selection. (jCJV: W2/Q2.)

  • Seven additional visualization figures. Appendix A.9, Figures 14–16, adds WHU/Luojia depth-map comparisons and MultiResXD native/resized examples. Appendix A.15, Figures 18–21, extends the RCCR-only posterior, ROI-error, and depthwise-influence analyses across five aerial datasets. These complement Figure 8 and Table 10, with direct references from Sections 4.2–4.3. (jCJV: Q1/Q3; fSV3: Q1; 948J: Q1/Q3/Q4.)

Clarifications and corrections

  • Clarified the contribution hierarchy. Sections 3.1–3.2 and Appendix A.2 distinguish internal context conditioning from output fusion and identify RCCR as the core mechanism, independent of CRE/SES. Section 4.3, Table 3, and Appendix A.14, Tables 12–13, explain the extensions' complementary roles. (fSV3: W1; 948J: W1/Q1 and W2/Q3.)

  • Reconciled evaluation results and protocols. Corrected the affected shared-input 4×4\times entries in Table 2 after resolving the mask-selection error; Appendix A.7 clarifies the distinct native/same-center masks. Aligned the SES results in Table 3 and Appendix A.14, Table 13 (original Table 10), and standardized reported MAE reductions. The detailed correction history is provided in our response to jCJV Q3. (jCJV: Q3.)

  • Clarified storage, practical trade-offs, and scope. Section 3.1 and Appendix A.5 specify which tensors remain depth-dependent; Appendix A.11, Table 9, reports complete-system efficiency. Section 5 explicitly states that conventional non-aerial benchmarks remain to be evaluated. (fSV3: W2/Q4 and W3/Q3; 948J: Q4.)

For visual inspection, we particularly recommend Figures 14–16 for final reconstruction quality and Figures 18–21 for RCCR's internal behavior. Detailed reviewer-specific responses accompany this overview. We believe these revisions substantially strengthen the experimental support and clarify the contribution of RCCR-MVS.

Thank you again for your time and constructive engagement.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.