acceptodds
Under review as a conference paper at ICLR 2027

StrataMesh: Adaptive Allocation for Compact Meshes at Scale

Abstract

Reconstructing explicit meshes from images at scale requires balancing rendering fidelity, triangle count, and geometric quality across regions with widely varying observation support. We introduce StrataMesh, which alternates adaptive capacity allocation with quality-aware fitting to address this challenge. Its dynamic Core–Shell representation assigns geometrically well-constrained regions to a Core and retains weakly constrained content in an appearance-oriented Shell; these roles evolve as multiview evidence accumulates. To decide where faces are useful, StrataMesh estimates the conditional fitting gain of a refinement beyond what existing geometry and appearance parameters can explain. Screened splits and collapses then redistribute capacity between the two strata. After each topology change, compatible learned state is transferred to the edited mesh. Joint supervision of the indexed output and a bounded Core proxy supports fitting, while octree-organized local, hierarchical geometry optimization and validity checks make quality-aware updates practical in large scenes. Experiments on Tanks and Temples, Mip-NeRF 360, and DTU demonstrate strong reconstruction and rendering fidelity with compact, well-shaped meshes at scale. On six Tanks and Temples scenes, StrataMesh achieves the best F1-score, rendering metrics, and triangle-shape results while using 67.3% fewer faces than MILo-base.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
mesh reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_KbYzOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

StrataMesh reconstructs compact triangle meshes of large scenes from multiview images. It assigns geometrically reliable regions to a Core and weakly constrained regions to a Shell for appearance modeling. These roles evolve during training. Candidate refinements are ranked by conditional fitting gain per added face. This gain measures the benefit of new degrees of freedom beyond the existing parameters. Collapses recover faces under a scene budget. Geometry and appearance are transferred after topology edits, followed by optimization with a triangle-soup proxy, diffusion, and geometric validity checks.

Critical strengths
  1. The allocation criterion is well motivated. Conditional gain identifies errors beyond the capacity of existing geometry and appearance parameters. Removing it increases face count from 2.94M to 8.68M without improving F1, PSNR, or LPIPS, supporting its role in reducing redundant refinement.

  2. The meshes are compact with good triangle quality. On T&T, StrataMesh uses fewer faces than MILo-base. Only 0.845% of its triangles have q ≤ 0.1, compared with roughly 8% for MILo, with no observed degenerate faces. On Mip-NeRF 360, it achieves the highest PSNR and SSIM among the evaluated methods and nearly the best LPIPS with fewer faces than MILo-dense.

  3. Resource use is low and evaluation is reasonably broad. On T&T, peak GPU memory is 4.61 GiB, with lower memory use and runtime than MILo and GGGS. The paper provides per-scene results and ablations of representation roles, budget transfer, and optimization strategies.

Critical weaknesses
  1. Conditional gain lacks direct empirical validation. The paper does not test whether predicted gain per added face corresponds to actual improvement during training. The ablation's replacement criterion, candidate counts, and scoring time are unspecified.

  2. MeshSplatting is omitted. MeshSplatting (CVPR 2026) optimizes opaque meshes for standard graphics engines and compares with MILo on Mip-NeRF 360 and T&T. This closely related method is neither cited nor evaluated.

  3. Shell initialization is unclear. The main paper initializes an empty Shell, but the appendix refines it before the first explicit role transition. The initial construction step is missing. The relationship between source-face deactivation in Appendix D.1 and migration from Core to Shell also needs clarification.

  4. Key experimental settings are missing. The paper does not specify the calibration procedure used, target error, extra face reserve for conforming refinement and quality constraints, migration thresholds, persistence length, or gain damping.

  5. Data inconsistencies and missing definitions hinder verification. Figure 1 and Table 7 report different face counts for GOF (24.53M vs. 23.10M) and RaDe-GS (22.67M vs. 63.12M). Any configuration or postprocessing differences should be explained. The appendix also contains an undefined loss symbol (p. 36), three unresolved references (pp. 36 and 42), and two duplicated sentences (p. 26, lines 1366 to 1371).

Questions for authors
  1. What refinement criterion replaces conditional gain in the ablation? Can you test whether predicted gain per added face corresponds to actual improvement on a small set of candidates and report the scoring time?

  2. Can you include a comparison with MeshSplatting?

  3. How are the initial Shell geometry and appearance constructed? How is the total face budget determined in the experiments, and what values are used for the key parameters listed above?

7 comments
AuthorsOpenReview ↗

Response (1)

Thank you for pointing out the missing implementation details and for suggesting MeshSplatting as a baseline. We respond to each weakness and question below.

Critical Weaknesses

W1. Empirical validation and scoring details for conditional gain

Higher predicted gain per face leads to greater realized improvement in the Ignatius group comparison. We refine the regions ranked 1–5 and 6–10 in two separate runs from the same checkpoint at step 12,000. Each run refines several patches together because the change caused by a single split can be smaller than the loss fluctuations across training views. The groups add similar numbers of faces, and each branch receives 1,000 QAMO steps. Scoring uses fixed visibility. After the splits are applied, QAMO rerenders the edited mesh and recomputes visibility and occlusion.

Conditional-gain ranksLoss at step 12,000Loss at step 13,000 ↓Loss reduction ↑
1–50.25120.19340.0578
6–100.25120.22340.0278

The higher-ranked group achieves 2.1× the loss reduction of the lower-ranked group (Appendix Fig. 10 and Table 27). This tests whether the ranking directs a comparable face increase to regions that benefit more from refinement after nonlinear fitting. The loss is measured for each group of edits. Individual split correlations require separate trials with controlled view sampling.

  • Replacement criterion in the ablation. In the w/o conditional gain ablation (Table 5 of the main paper), we replace conditional gain with a DeMapGS-style screen-space positional gradient criterion. The gradient measures how sensitive the current fit is to updates of existing variables. Conditional gain measures the additional improvement available from a split after accounting for those existing variables. In the 2.94M-face comparison across six Tanks and Temples scenes (Appendix Table 25), conditional-gain ranking improves F1 from 0.411 to 0.498 and PSNR from 18.36 to 19.97 dB relative to gradient-based ranking.
  • Candidate count and spatial limit. Several nearby candidates can receive high scores when their neighborhoods overlap, concentrating refinement in a small patch. We rank 8,192 candidate regions per rendered image and allow at most one added triangle per 16 × 16-pixel region (experimental settings at the start of Appendix B). This limits how many faces the selector spends in the same image region. Committed edits are charged their full face increase against the shared budget, including conformity faces (Sec. 3.1.2 of the main paper and Appendix E.2.1).
  • Scoring overhead. We compute conditional gain from statistics already collected during training, so scoring reuses the existing observations and gradients. The remaining work is the local damped solve and ranking. Compatible candidates can share the existing-response basis (Appendix E.2.1), and we cache each candidate's score and face cost for selection (Sec. 3.1.2 of the main paper).
AuthorsOpenReview ↗

Response (2)

W2. Comparison with MeshSplatting

Thank you for suggesting this comparison. Section 2 of the main paper cites MeshSplatting (Held et al., CVPR 2026). Appendix B.6 evaluates both methods on Tanks and Temples.

  • Higher reconstruction fidelity with fewer faces. StrataMesh achieves higher F1 on all five scenes with ground-truth geometry (Appendix Table 23), with nearly three times the mean F1 (0.560 vs. 0.188). It uses fewer faces on four of those scenes, with 34% fewer faces on average (3.12M vs. 4.70M). On the rendered images, StrataMesh achieves lower mean LPIPS (0.296 vs. 0.353).
  • Better-shaped triangles and cleaner surfaces. StrataMesh leads on both triangle-shape metrics in each of the six scenes (Appendix Table 24). Mean triangle quality is 0.689 vs. 0.540, and the fraction of low-quality triangles (q≤0.1q \leq 0.1) is 0.766% vs. 3.971%. Appendix Fig. 8 shows paired RGB and mesh crops on Truck and Train. In the RGB crops, the truck fender and the train steps show smoother surfaces with fewer visible creases. The mesh crops show more regular triangulation around the truck wheels and cleaner boundaries around the train steps and side vents. They also illustrate how StrataMesh uses finer triangles near geometric details and coarser faces on simpler surfaces. The comparison highlights local surface continuity and triangle layout in the shown regions.
  • Lower peak GPU memory. StrataMesh uses 4.83 vs. 24.92 GiB of mean peak GPU memory across the five scenes with F1. It maintains indexed connectivity throughout fitting using local, budgeted splits and collapses with manifold checks. MeshSplatting builds connectivity through restricted Delaunay triangulation, then fine-tunes the connected opaque mesh. The peak memory numbers cover the full pipelines.

Reconstruction F1, LPIPS, and mesh resources on five scenes

SceneMethodF1 ↑LPIPS ↓Faces (M) ↓Peak GPU (GiB) ↓
TruckMeshSplatting0.290.2363.319.11
TruckStrataMesh0.620.2612.113.12
BarnMeshSplatting0.120.3574.7830.24
BarnStrataMesh0.610.2972.694.12
CaterpillarMeshSplatting0.150.4374.3930.14
CaterpillarStrataMesh0.450.3742.673.97
IgnatiusMeshSplatting0.240.3954.424.83
IgnatiusStrataMesh0.760.2585.48.79
MeetingroomMeshSplatting0.140.3386.630.26
MeetingroomStrataMesh0.360.2892.724.13
Mean (5 scenes)MeshSplatting0.1880.35264.69624.916
Mean (5 scenes)StrataMesh0.5600.2963.1184.826

Triangle quality on all six scenes

Train is included in the shape comparison. The F1 table covers the five scenes with reference geometry.

SceneMethodMean q ↑q ≤ 0.1 (%) ↓
TruckMeshSplatting0.5522.831
TruckStrataMesh0.6760.987
BarnMeshSplatting0.5016.290
BarnStrataMesh0.6790.700
CaterpillarMeshSplatting0.5444.363
CaterpillarStrataMesh0.6850.598
IgnatiusMeshSplatting0.5653.054
IgnatiusStrataMesh0.6900.784
MeetingroomMeshSplatting0.5134.912
MeetingroomStrataMesh0.7330.856
TrainMeshSplatting0.5652.376
TrainStrataMesh0.6710.672
AverageMeshSplatting0.5403.971
AverageStrataMesh0.6890.766

Both methods report zero degenerate faces under the triangle-shape criterion.

AuthorsOpenReview ↗

Response (3)

W3. Shell initialization and source-face deactivation

Thank you for pointing out the missing step in the initialization schedule and asking how deactivation relates to role migration.

  • The initial Shell is created before the first Shell refinement. At initialization, we build a colored patch in each occupied octree cell from its SfM points and their RGB values, then stitch the patches into one indexed mesh and assign all faces to Core (Sec. 3 of the main paper). We render this mesh from the calibrated training views and estimate parallax for each visible region. In the first role assignment, stronger-parallax regions remain in Core and weaker-parallax regions form the initial Shell. Shell vertex colors are initialized from the Core-only RGB renders. Panel 1 of the Ignatius walkthrough (Appendix Fig. 4) illustrates this assignment. The initial budget reallocation then refines the Shell that is already in place, followed by Shell appearance warm-up and joint Core–Shell fitting. The revised reconstruction schedule in Appendix E.4 now shows the initial assignment before the first refinement.
  • Deactivation is the Core-side step of a later Core-to-Shell migration. Once a region meets the persistent demotion criteria in Appendix E.1 and the migration is accepted, we deactivate its source faces in Core. The faces stop contributing to the Core render and receiving Core updates. The reservoir retains the source geometry for possible later promotion while it remains compatible with the current topology. If a Shell patch is nearby, we extend it to cover the region. Otherwise, we detach the selected patch from the Core connectivity and reassign it as a Shell patch. The region then receives Shell geometry and appearance updates.

W4. Experimental budget and key settings

  • Budget calibration and target error. The reported experiments use the simpler gradient-based estimate described in the revised Appendix C.1.4. This replaces the earlier rule based on calibrated regional coefficients and a prescribed target error. We identify adjacent pairs of vertices whose gradient magnitudes exceed τgrad\tau_{\rm grad}, estimate the vertex count assuming one insertion per pair, and refresh the budget estimate every few iterations.
  • Extra face reserve. We use a shared Core–Shell face budget with zero extra reserve. Each accepted refinement is charged its full face-count increase, including conformity splits on neighboring faces (Sec. 3.1.2 of the main paper and Appendix E.2.1). QAMO maintains mesh quality through regularization and geometric validity checks during fitting (Sec. 3.2 of the main paper).
  • Migration, persistence, and damping. For Core-to-Shell demotion, a face is eligible if it has fewer than two visible views or fewer than one parallax-supporting view and its mean squared residual is at or below the 35th percentile among observed Core faces. The condition must hold for two consecutive checks (Appendix E.1). Conditional-gain damping is 10−410^{-4} (Appendix E.2.1 gives the scoring equation).

W5. Face-count discrepancies and appendix corrections

Thank you for catching these inconsistencies.

  • Truck results in Figure 1. The teaser in Figure 1 of the main paper is intended to summarize the Truck evaluation reported in Table 7 in Appendix B.2. Two face-count labels were entered incorrectly when preparing the teaser. Appendix Table 7 gives the correct counts of 23.10M for GOF and 63.12M for RaDe-GS. We have corrected the teaser to use these face counts from Appendix Table 7.
  • Appendix corrections. In the revised appendix, p. 40 defines the image-loss gradient and resolves the two proof references. Appendix E.2.3 (p. 45) gives the quota derivation in place of the unresolved reference. The state-transfer passage in Appendix C.2.4 (p. 30) now contains one copy of the previously duplicated sentences.
Reviewer_HKEPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a system for reconstructing compact explicit meshes from calibrated images and sparse SfM points. The method separates scene regions into a dynamically evolving geometric Core and an appearance-oriented Shell according to multiview observability. Its Adaptive Core–Shell Allocation module updates representation roles and allocates mesh capacity using a conditional fitting gain that estimates the residual reduction enabled by newly introduced degrees of freedom beyond what existing geometry and appearance variables can explain. Faces can be reallocated across strata through screened split/collapse operations.

Critical strengths
  1. The conditional-gain formulation is the most interesting component. Instead of refining wherever the raw photometric residual is high, the method projects out image responses already available through the current geometry and appearance variables. A candidate is valuable only if its additional modes explain residual components unavailable to the current representation. This provides a principled motivation for refinement rather than a purely residual-based heuristic.
  2. The quality/complexity trade-off is strong. On the six Tanks and Temples scenes, StrataMesh achieves F1 0.498 and PSNR 19.97 dB using only 2.94M faces on average, while also providing substantially better triangle-shape statistics than the compared methods.
  3. The paper explicitly treats topology changes as part of optimization. State transfer, indexed-mesh supervision, proxy supervision, mesh-quality checks, and the octree-based geometry solve address real implementation difficulties that many adaptive-mesh papers omit.
Critical weaknesses
  1. The central claim about more efficient mesh-capacity allocation is not yet isolated as cleanly as I would like. The main ablations end with substantially different face counts: e.g. 2.94M for the full method, 8.68M without conditional gain, 6.94M for static roles, and 1.84M without cross-stratum transfer. The authors appropriately acknowledge that these experiments demonstrate end-to-end quality–size trade-offs rather than a matched-face advantage. Because adaptive allocation is the central contribution, I would find a small matched-budget/Pareto analysis highly informative. At the same face budget, does conditional-gain allocation consistently outperform residual-driven or static allocation?
  2. The conditional-gain estimate itself is not directly validated against the realized nonlinear gain. The score is computed from a local linearized model while visibility is held fixed. However, split/collapse operations can change silhouettes, visibility, and occlusion. The paper demonstrates that the complete algorithm works, but does not show whether a high predicted se/ΔFes_e/\Delta F_e actually correlates with the loss reduction achieved after applying the edit and running QAMO. This is a particularly important empirical test because the conditional-gain estimator is presented as a main conceptual contribution.
  3. The method appears dependent on recoverable initial surface support. The appendix states that promotion restores available source geometry rather than synthesizing a previously absent component. Thus, regions absent from the initial SfM-derived surface may remain unrecoverable even if later evidence would favor a Core representation.
    I think this limitation should be more prominent, particularly for low-texture or thin structures where SfM support is poor.
Questions for authors
  1. How well does the predicted conditional gain correlate with the realized nonlinear improvement after executing the topology edit and subsequent fitting? Even a scatter/rank-correlation analysis on a representative scene would help.
  2. Can the authors provide a small matched-face comparison or a quality-vs-face-count curve for the full allocator and its most important ablations? This would separate allocation quality from the final automatically selected budget.
  3. What happens when the initial SfM point cloud is substantially sparser or misses a scene component? Is the Shell able to create genuinely new surface support, or only retain/reassign existing support?
3 comments
AuthorsOpenReview ↗

Response (1)

Thank you for the thoughtful assessment and for suggesting tests that isolate allocation quality and measure nonlinear fitting gains. We respond to each weakness and question below.

Critical Weaknesses

W1. Does adaptive allocation improve quality at the same face budget?

We agree that Table 5 of the main paper compares variants that finish with different numbers of faces. To test whether StrataMesh makes better use of the same mesh capacity, we reran the two main ablations at the full method's face budget (Appendix Table 25). At the reported budget of 2.94M total faces, StrataMesh achieves better six-scene average results than both ablations on all four metrics.

The two ablations test different parts of the allocator:

  • Gradient-based ranking (w/o conditional gain) retains cross-stratum budget transfer and replaces conditional gain with DeMapGS-style screen-space gradient magnitude for ranking candidate splits. This tests the value of conditional-gain ranking when faces can still move between Core and Shell.
  • Static roles keeps the role assignment fixed and divides the face budget equally between Core and Shell. This tests the full adaptive design against a fixed role assignment and budget split.
VariantF1 ↑PSNR (dB) ↑SSIM ↑LPIPS ↓Total faces (M)
Gradient-based ranking (w/o conditional gain)0.41118.360.6840.3122.94
Static roles (fixed equal Core–Shell budget split)0.31216.330.5410.4492.94
StrataMesh0.49819.970.7070.3102.94

Results are averaged over the same six Tanks and Temples scenes. Total faces include Core and Shell.

With gradient-based ranking, a region can score highly even when adjusting existing parameters would already reduce its error. Conditional gain scores the additional improvement enabled by the split. The matched-budget result supports that choice, with +0.087 F1 and +1.61 dB PSNR over gradient-based ranking. The improvement over static roles reflects the combined benefit of updating the roles and reallocating faces between them.

These results establish the advantage at one matched operating point. The cross-stratum transfer ablations in Table 5 of the main paper remain end-to-end comparisons at their respective final face counts.


W2. Does predicted conditional gain carry over to nonlinear fitting?

We agree that the score should be tested after the selected edits are applied and the mesh is fitted again. Conditional gain ranks proposed splits using a local model with fixed visibility. We test whether that ranking helps decide where to spend a comparable number of faces when subsequent fitting can change silhouettes, visibility, and occlusion.

  • Why test groups of edits? Each split affects a small part of the scene. Even the effect of a few edits can be smaller than the fluctuations in the scene-level loss as training views change. We therefore refine several ranked patches jointly and measure the loss over a sustained fitting interval.
  • How we control the comparison. On Ignatius, both runs start from the same checkpoint at step 12,000. One run refines patches ranked 1–5 by predicted gain per added face, and the other refines patches ranked 6–10. The two groups add similar numbers of faces. After applying the splits, we run 1,000 QAMO steps in each branch. QAMO rerenders the edited meshes during fitting, allowing visibility and occlusion to change.
Conditional-gain ranksLoss at step 12,000Loss at step 13,000 ↓Loss reduction ↑
1–50.25120.19340.0578
6–100.25120.22340.0278
  • What the comparison shows. At similar face increases and the same fitting duration, the higher-ranked group achieves 2.1× the loss reduction of the lower-ranked group (Appendix Fig. 10 and Table 27). This gives group-level evidence that the score prioritizes regions where spending faces leads to greater improvement after nonlinear fitting. A per-candidate rank correlation would require resolving the effect of individual splits under controlled view sampling. The score evaluated here ranks refinement splits. Budget-recovery collapses use a separate removal cost and validity checks.
AuthorsOpenReview ↗

Response (2)

W3. Dependence on the initial surface

We agree that the method depends on the surface available at initialization. The effect of a sparser SfM point cloud depends on whether the resulting mesh still covers the structure in question.

  • If the initial surface is present but coarse, splits can add resolution and image-supervised fitting can adjust its geometry. For example, an initially coarse surface over part of a truck can be refined where the images provide enough evidence. The extent of recovery depends on the existing surface coverage and the strength of the observations.
  • If the initial surface omits a component entirely, the current method cannot create it. Shell fits appearance and geometry on the available mesh. Splits and collapses operate on existing faces. Promotion can restore a face that is inactive in Core if a compatible source face remains in the reservoir (Appendix E.1). A component with no initial surface or source faces remains missing even if the evidence accumulated during fitting favors a Core role. Recovering it requires an initialization or a surface proposal that provides the missing geometry (Appendix E.2.1).

Section 3.1.1 of the revised main paper states this limitation explicitly. It explains that thin structures and low-texture regions can remain incomplete when SfM initialization leaves them without surface support.


Questions for Authors

Q1. How well does predicted gain track realized improvement?

At the group level, higher predicted gain corresponds to greater realized loss reduction on Ignatius. Starting from the same checkpoint and running 1,000 QAMO steps with similar face increases, ranks 1–5 reduce loss by 0.0578, compared with 0.0278 for ranks 6–10. That is about 2.1× the reduction (see W2, Appendix Fig. 10, and Appendix Table 27).

We group several patches because the change in scene-level loss from a single split can be smaller than the variation across training views. The result measures how well the score guides the selection of regions to refine together. Estimating the rank correlation for individual splits requires separate trials and a fixed set of evaluation views to resolve their smaller effects.


Q2. Can we compare the allocators at the same face count?

Yes. We reran the two main allocation ablations at the full method's face budget. Appendix Table 25 reports averages over the same six Tanks and Temples scenes. All three configurations use 2.94M Core and Shell faces on average.

VariantTotal faces (M)F1 ↑PSNR (dB) ↑SSIM ↑LPIPS ↓
Static roles (fixed equal Core–Shell split)2.940.31216.330.5410.449
Gradient-based ranking (w/o conditional gain)2.940.41118.360.6840.312
Full StrataMesh2.940.49819.970.7070.310

At this matched point, StrataMesh achieves better averages on all four quality metrics. Compared with gradient-based ranking, it improves F1 by 0.087 and PSNR by 1.61 dB. Compared with static roles and an equal budget split, it improves F1 by 0.186 and PSNR by 3.64 dB. W1 explains what each comparison isolates. These results provide one matched-face comparison at the full method's reported budget.


Q3. What can Shell recover from sparse or incomplete SfM initialization?

With sparser SfM, a coarse surface that remains in the initial mesh can still be refined. If an entire component is absent from both the initial mesh and the source reservoir, Shell cannot create it and Core promotion has no source faces to restore. See W3 for the mechanism and the implications for thin structures and low-texture regions.

AuthorsOpenReview ↗

Follow-up on Our Responses to Your Concerns

Dear Reviewer HKEP,

We are following up on our point-by-point responses and the revised manuscript, which we posted several days ago.

To address your main concerns, we have conducted additional matched-budget comparisons, evaluated the relationship between predicted conditional gain and realized nonlinear improvement, and explicitly clarified the limitations of SfM-based initialization. We believe these new experiments and revisions provide stronger evidence for the effectiveness of our proposed method.

As the Author-Reviewer Discussion phase draws to a close, we would greatly appreciate your feedback on whether these results address your primary concerns or if further clarification is needed. We remain available for discussion.

Thank you again for your valuable feedback and time!

Best,

The Authors of Submission 28860

Reviewer_D5K7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes StrataMesh, a method for reconstructing compact explicit meshes from calibrated images. Based on multi-view support, parallax, and reconstruction residuals, the method partitions regions into a geometry-oriented Core and an appearance-oriented Shell. It adaptively reallocates triangles through edge splitting and edge collapsing, ranking candidate refinement operations by the additional fitting gain they can achieve relative to the existing parameters. The method then jointly supervises the indexed mesh and a bounded triangle-soup proxy attached to the Core, fitting the resulting representation through coordinated geometry updates and mesh quality checks.

The reported results are impressive, particularly the overall balance achieved between reconstruction fidelity, compactness, and triangle quality. But the paper does not make the organizing idea sufficiently clear to distinguish essential design choices from accumulated pipeline complexity.

Critical strengths
  1. Each enumerated component of the pipeline makes sense on its own. The high-level narrative of Core handling regions with high geometry-certainty while Shell handling regions with depth ambiguity makes sense. But it doesn't connect concretely to the enumerated components in the paper (see weaknesses)

  2. The numerical results are impressive.

Critical weaknesses
  1. The motivation for Core–Shell is not concrete enough: the idea of separating reliable geometry from appearance under uncertain depth is reasonable. However, both Core and Shell are explicit meshes with optimizable geometry and appearance. So from a high level, it remains unclear what problem a single adaptive-refinement mesh would exhibit that would necessitate these distinct, separate representation roles. The implementation-level differences—proxy fitting and coordinated displacement specific to Core, versus direct updates for Shell—give the split substantive content, but the relationship between these differences and geometric observability needs clearer explanation. In particular, why are these different fitting mechanisms respectively suited to their corresponding conditions?

  2. The connection between assignment and fitting is hard to follow: the introduction emphasizes the Core–Shell assignment, while Section 3.2 introduces an additional geometric object: a bounded triangle-soup proxy attached to the Core. Although proxy supervision is mentioned earlier, its purpose and its relationship to the two layers are not sufficiently set up, leaving readers unprepared to understand it. It would be very helpful for the main text to provide a concrete example that tracks a region through role selection, candidate splitting, conditional-gain scoring, and the subsequent mesh/proxy fitting. The equations describe the individual components, but the equations alone do not suffice to form a clear intuition of the complete algorithm.

Questions for authors
  1. What are the typical failure cases of a single indexed mesh with adaptive refinement that the Core–Shell split specifically resolves? It would be very helpful if role assignments and the resulting triangle allocation could be illustrated with visualization examples.

  2. Why use a bounded triangle-soup proxy specifically for the well-observation-supported Core? My understanding is that Core contains better, more accurate depth with high certainty, so the triangle-soup proxy should be allocated to regions with less certain geometry to reduce memory budget and higher rendering efficiency (2-pass rasterization sounds a bit cumbersome).

  3. The octree-local geometry update is visually emphasized as a major component of QAMO in Figure 2, but Section 3.2 provides only a brief description of its operation. The correspondence between the hierarchy in Figure 2 and the geometry-update procedure is difficult to follow. Either a revised Figure 2 or an overhaul of the structure of Section 3 would help significantly. At this point the inconsistency in the visual presentation of Figure 2 and the text of Section 3 only adds difficulty for understanding the paper.

4 comments
AuthorsOpenReview ↗

Response (1)

Thank you for the careful review and for asking us to make the motivation, fitting mechanisms, and pipeline clearer. We respond to each weakness and question below.

Critical Weaknesses

W1. Why use dynamic Core–Shell roles when both are explicit meshes?

  • We agree that a single adaptive indexed mesh can represent the scene. Both Core and Shell have optimizable geometry and appearance. The practical challenge is that the same image error can call for different updates depending on what the views constrain. On a well-observed surface, observations from sufficiently different viewpoints can guide a depth correction. On a distant roof seen from similar viewpoints, adding faces may improve the image fit while leaving several plausible depths. The Core–Shell assignment determines which geometry update each region receives. The two strata have separate connectivity and are composited in depth order to form the same rendered image.
  • Each fitting rule is suited to the evidence available in its region. In Core, local image gradients carry useful information about geometry, but neighboring faces can favor different motions for their shared vertices. A bounded proxy gives each face corner room to match the images locally. The coordinated displacement solve and validity checks then control how the connected indexed mesh moves, and a separate rendering loss supervises that indexed mesh directly. In Shell, weak parallax leaves the depth correction ambiguous. We use direct updates with displacement bounds and mesh-quality checks to fit visible content while controlling geometric distortion. The Shell ablation tests this choice with the Core update held fixed. Both variants achieve 0.61 F1 and similar rendering metrics. Direct Shell updates take 58.0 vs. 243.1 minutes and use 4.12 vs. 5.99 GiB of peak GPU memory. Appendix Table 26 reports the quantitative comparison, and Appendix Fig. 9 illustrates the update variants on Truck.
  • Regions can move between Core and Shell as the evidence changes. We recompute view support, parallax, and rendering residuals on the evolving reconstruction. A Core region becomes eligible for Shell treatment when view support or parallax is persistently weak and its residual is low. A Shell-covered region becomes eligible for Core fitting when it has sufficient view support and parallax, persistent unexplained residual, and compatible source faces that can be restored. Persistence and topology checks screen these transitions. A role change therefore changes how we fit the region in the next optimization interval. Split and collapse decisions determine where to spend faces under the shared budget. In the 2.94M-face comparison in Appendix Table 25, the full allocator improves F1 from 0.312 to 0.498 and PSNR from 16.33 to 19.97 dB over static roles with an equal Core–Shell budget split. This comparison measures the combined effect of updating roles and reallocating capacity between them.
Core updateShell updateF1 ↑PSNR ↑SSIM ↑LPIPS ↓Time (min) ↓Peak GPU (GiB) ↓Faces (M)Vertices (M)
Octree solveOctree solve0.6123.750.7470.293243.15.992.641.33
Octree solveDirect (ours)0.6123.980.7500.29758.04.122.691.35
AuthorsOpenReview ↗

Response (2)

W2. How role assignment, conditional gain, and fitting work together

Thank you for pointing out that the connection between role assignment and fitting needs to be clearer. As described in W1, the role assignment determines the fitting rule for each region. The revised main text makes the following connections explicit.

  • What does the proxy do, and how does it relate to the output? Neighboring Core faces can favor different local adjustments to a shared vertex. The proxy gives each face corner a bounded offset from its corresponding indexed vertex, allowing those faces to fit the images locally. The coordinated displacement solve governs how the shared vertices move. Both the proxy-plus-Shell and indexed Core-plus-Shell renders are supervised, so improvements in the proxy render are accompanied by a training signal on the actual indexed output. The proxy is attached to Core throughout fitting. The final mesh consists of the indexed Core and Shell.
  • How does fitting connect to allocation? The current indexed render supplies the image residual used to score possible splits. Conditional gain estimates the improvement available from a split's new variables after allowing the existing geometry and appearance variables to adjust. After the budget and validity checks accept an edit, we transfer compatible state and resume fitting on the edited mesh. The next round then uses evidence from that updated reconstruction.

Concrete example (Ignatius, Appendix Fig. 4). We follow the statue (A) through one cycle, from its role assignment to fitting after the selected split.

  1. Use multiview evidence to choose how to fit A (panel 1). The initial indexed mesh belongs entirely to Core. Across the illustrated views, the statue shows substantial parallax, so it remains in Core and receives a geometry-focused update. The distant building (C) shows weaker parallax and is assigned to Shell, where we fit its visible appearance with direct vertex updates under weak depth constraints.
  2. Fit the current mesh and identify the remaining image error (panel 2). On A, the bounded Core proxy lets neighboring face corners make local fitting adjustments. The coordinated displacement solve updates their shared indexed vertices, and the indexed mesh is also supervised directly. After this fitting interval, we use the residual from the indexed Core-plus-Shell render to ask whether A would benefit from more geometric capacity.
  3. Test whether a split can explain that error (panel 3). We compare two local fits for A with visibility fixed. In the first, the existing geometry and appearance variables can adjust, leaving an objective of 0.32 in the illustration. In the second, we also enable the virtual split's new variables, lowering the objective to 0.20. The difference of 0.12 is the predicted benefit attributable to the added variables. With two added faces, A scores 0.06 per face, ahead of the pool rim B (0.04) and the building C (0.02).
  4. Release capacity, commit the split, and fit again (panel 4 → panel 2). A screened collapse in C frees faces for A's split under the shared budget. The collapse is selected by its low removal cost. The split in A is selected by its high conditional gain per added face. B retains its current allocation in this exchange. We initialize the new split fields from the parent faces, transfer compatible state, and resume image-supervised QAMO on the edited mesh. This step fits the added variables on the actual mesh. The updated geometry and renders supply the view support, parallax, and residuals for the next round.

Section 3 of the main paper introduces the Core-bound proxy alongside the role assignment and outlines the adaptation cycle. Appendix A.1 and Appendix Fig. 4 give the full Ignatius walkthrough.

AuthorsOpenReview ↗

Response (3)

Questions for Authors

Q1. Failure modes of a single adaptive mesh and visualization

As discussed in W1, the difficulty arises when one fitting rule is used across regions with different geometric observability.

  • Direct updates can degrade triangle quality. Adjacent faces can favor conflicting motions at a shared vertex. In Table 6 of the main paper (Barn, Caterpillar, and Truck), direct vertex optimization lowers mean triangle quality from 0.680 to 0.507 and raises the fraction of low-quality triangles (q≤0.1q \leq 0.1) from 0.761% to 5.143% compared with full StrataMesh.
  • Coordinated updates in weakly observed regions can add substantial cost for similar quality. The Shell ablation in W1 reports the runtime and memory cost of extending the coordinated solve to the Shell.
  • Role assignment and face allocation. Appendix Fig. 4 (W2) shows the Core–Shell assignment in panel 1 and the resulting face reallocation in panels 3–4. The distant building releases faces for a higher-gain split on the statue.

Q2. Why attach the bounded proxy to Core?

  • We agree that Core depth is better constrained by the observations. Core assignment identifies regions with sufficient multiview support and parallax to guide depth estimation. The accuracy of the current mesh is a separate matter. Its depth can still be inaccurate, for example when the initial surface is reconstructed from sparse SfM points. The stronger observations give us a more reliable signal for correcting those errors.
  • Why place the proxy in Core? Adjacent faces can favor different local adjustments to a shared vertex. The bounded per-corner offsets give each face room to fit the images locally, and the coordinated solve controls how the connected Core moves. In Shell, weak parallax allows several depths to explain similar images. Additional per-corner freedom could improve appearance fitting while leaving the depth ambiguous. This is why we use the proxy where the observations can guide geometric corrections and fit Shell with constrained direct updates.
  • Two rasterization passes during training, one at inference. The proxy-plus-Shell render provides local fitting flexibility during training, and the indexed Core-plus-Shell render directly supervises the mesh we export. With both passes enabled, the full pipeline averages 56.17 minutes per scene and 4.61 GiB peak GPU memory across the six Tanks and Temples scenes (Table 1 of the main paper). The proxy is an auxiliary representation used during training. At inference, a single rasterization of the exported indexed Core–Shell mesh produces the image.

Q3. Octree hierarchy in Fig. 2 and Sec. 3.2

Thank you for pointing out that the original figure leaves the role of the hierarchy unclear. The octree helps solve for coordinated Core vertex motion. Fine cells reduce the residual of the displacement solve locally, coarser cells compute corrections that account for coupling across regions, and those corrections propagate back to the fine cells. The resulting motion updates the connected Core mesh.

The revised caption of Fig. 2 and Sec. 3.2 of the main paper explain how the illustrated octree levels are used in the geometry solve. Panel 2 of Appendix Fig. 4 shows where this solve occurs during mesh fitting.

AuthorsOpenReview ↗

Follow-up on Our Responses to Your Concerns

Dear Reviewer D5K7,

We are following up on our point-by-point responses and the revised manuscript, which we posted several days ago.

In particular, we have clarified the motivation behind the Core–Shell design, provided a step-by-step example illustrating the complete pipeline, and improved the explanation of the octree-based optimization. We believe these revisions substantially address your concerns regarding the clarity and presentation of our method.

As the Author-Reviewer Discussion phase draws to a close, we would greatly appreciate your feedback on whether our responses address your main concerns or if any points still require clarification. We would be happy to discuss them further.

Thank you again for your time and thoughtful review!

Best,

The Authors of Submission 28860

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions to StrataMesh

Dear Area Chair and Reviewers,

Thank you for your constructive feedback. We revised the manuscript to clarify the reconstruction pipeline, test the main design choices, and correct inconsistencies in the original submission. The main changes are marked in blue. Each item below indicates where the revision appears and which reviewer comments it addresses.

In the main paper

  1. Corrected the Truck teaser (Fig. 1). The GOF and RaDe-GS face counts now match Appendix Table 7 at 23.10M and 63.12M, respectively. (Reviewer KbYz (W5).)
  2. Added MeshSplatting to Related Work (§2). We describe its mesh construction and fitting stages and relate them to StrataMesh's approach to face allocation. (Reviewer KbYz (W2, Q2).)
  3. Clarified Core–Shell initialization and fitting (§3 and §3.1.1). We explain how colored SfM points form the initial all-Core mesh, how the first role assignment creates Shell, and why Core uses a bounded proxy with coordinated vertex updates while Shell uses constrained direct updates. (Reviewer D5K7 (W1, W2, Q1, Q2); Reviewer KbYz (W3, Q3).)
  4. Made the initialization limitation explicit (§3.1.1). Thin structures and low-texture regions can remain incomplete when SfM leaves insufficient surface support. (Reviewer HKEP (W3, Q3).)
  5. Clarified how fitting informs face allocation (§3.1.2). The current indexed render provides evidence for candidate screening, and conditional gain estimates the improvement enabled by the additional variables of a virtual split. (Reviewer D5K7 (W2, Q1); Reviewer HKEP (W2, Q1); Reviewer KbYz (W1, Q1).)
  6. Connected the QAMO update to the octree shown in Fig. 2 (Fig. 2 caption and §3.2). We explain how the illustrated levels support fine-cell relaxation, coarse-level correction, and propagation back to the fine cells. We also clarify that training supervises two renders and inference rasterizes the exported indexed mesh once. (Reviewer D5K7 (Q2, Q3).)
  7. Clarified the allocation ablation (§4.2 and Table 5). We identify the DeMapGS-style gradient criterion used in the w/o conditional gain variant and point to the new matched-budget comparison in Appendix Table 25. (Reviewer HKEP (W1, Q2); Reviewer KbYz (W1, Q1).)

In the appendix

  1. Added a worked example on Ignatius (A.1 and Fig. 4). The walkthrough follows the statue through role assignment, proxy-guided fitting, virtual split scoring, and face reallocation. (Reviewer D5K7 (W2, Q1, Q2).)
  2. Documented Shell initialization and migration (A.2, E.1, and E.4). The reconstruction schedule now shows the first role assignment and Shell color initialization before refinement. We also explain how source faces are deactivated when Shell takes over a Core region. (Reviewer KbYz (W3, Q3).)
  3. Reported the key settings and explained the scoring work (Appendix B, C.1.4, and E.1–E.2.1). We document the gradient-driven budget estimate, zero extra face reserve, full conformity-face accounting, candidate counts, migration thresholds, persistence, and damping. We explain how conditional-gain scoring reuses training observations and gradients. (Reviewer KbYz (W1, W4, Q1, Q3).)
  4. Added a MeshSplatting comparison (B.6, Tables 23–24, and Fig. 8). We report reconstruction and resource metrics on five scenes with reference geometry, triangle quality on six scenes, and paired RGB and mesh crops on Truck and Train. (Reviewer KbYz (W2, Q2).)
  5. Added matched-budget allocation and Shell-update ablations (B.7, Tables 25–26, and Fig. 9). We compare full StrataMesh with static roles and gradient-based ranking at 2.94M faces on average. We also compare direct and octree-based Shell updates with the Core solver held fixed. (Reviewer D5K7 (W1, Q1); Reviewer HKEP (W1, Q2); Reviewer KbYz (W1, Q1).)
  6. Tested conditional-gain ranking after nonlinear fitting (B.7, Table 27, and Fig. 10). Starting from the same Ignatius checkpoint, the higher-ranked group achieves 2.1× the loss reduction of the lower-ranked group after 1,000 QAMO steps. (Reviewer HKEP (W2, Q1); Reviewer KbYz (W1, Q1).)
  7. Corrected notation and references (C.2.4, p. 40, and E.2.3). We defined the image-loss gradient, repaired the broken references, and removed the duplicated sentences in the state-transfer discussion. (Reviewer KbYz (W5).)

Thank you again for your feedback. We would be happy to clarify any remaining questions.

Best regards,
The StrataMesh authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.