acceptodds
Under review as a conference paper at ICLR 2027

AncesTrA: Streamable Long-Horizon Volumetric Reconstruction via Canonical Transport Lineages and Asymmetric Basis Lifting

Abstract

Long-horizon volumetric video reconstruction requires compact and temporal modeling that remains faithful as scene evolution grows increasingly complex. Existing streamable representations either recursively propagate reconstructed states, coupling temporal continuity with accumulated errors, or rely on persistent references whose validity deteriorates as scenes diverge; symmetric Gaussian bases further limit local expressivity near discontinuities. We introduce , a streamable volumetric representation that recasts long-horizon evolution as . separates reconstruction from temporal propagation by evolving compact transport states over a shared canonical representation, enabling efficient temporal . regulates canonical validity by reanchoring evolution when transport dynamics indicate expiration, preserving long-horizon . selectively lifts symmetric Gaussian bases into asymmetric forms around geometric and appearance discontinuities, increasing local expressivity without increasing anchor density. Extensive experiments demonstrate state-of-the-art reconstruction fidelity: outperforms Recon-GS by approximately with lower storage on short sequences, and by approximately with lower storage on long-horizon sequences. The widening advantage with temporal extent substantiates canonical transport lineages as a principled alternative to recursive scene-state propagation, positioning as a substantive reformulation of how temporal evolution is organized in streamable volumetric representations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Volumetric Video, Streaming 4D Reconstruction, Long-Horizon Dynamic Scenes, 3D Gaussian Splatting, Canonical Representations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DiYpOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

AncesTrA works on streamable volumatric representation over long sequences. It is built on iFVC and proposes three designs: CiTER, TraCER, and CaBLE. CiTER aims to decouple temporal dependency from reconstructed-state propagation. TraCER periodically replaces an outdated canonical with the current frame. CaBLE turns Gaussians into learnable half-Gaussians to better fit sharp edges. AncesTrA achieves large gains compared to baselines on three benchmarks.

Critical strengths

Long-horizon volumetric video reconstruction is an important and long-standing task, and this paper offers improvements from an engineering perspective. It identifies a clear limitation of existing streamable methods: recursive frame-to-frame propagation accumulates errors, while a single persistent reference becomes increasingly outdated as the scene evolves.

Critical weaknesses
  1. It is unclear whether the method can handle substantial changes in scene content. It reuses the initial anchor pool and freezes the decoder after the first I-frame, while canonical renewal only promotes an already reconstructed frame. Newly appearing content must therefore be represented using existing capacity. Without tests of new objects entering the scene, disappearing and re-entering, or major changes in scene composition, the long-horizon results do not establish robustness to these changes.

  2. The experiments do not clearly explain where the quality gains come from. Most of the PSNR advantage over iFVC already exists in the default “w/o All” baseline, while adding CiTER alone reduces PSNR despite improving efficiency. TraCER is not compared with fixed-period or quality-triggered renewal at matched storage and computational budgets, and its bit-stabilization criterion is not shown to reliably indicate reconstruction degradation. Each renewal also gives the next predictive frame 5,000 optimization iterations rather than the usual 300, making it difficult to separate the benefit of renewal from extra optimization. CaBLE closely overlaps with the uncited 3D-HGS, including half-space splitting, bilateral opacity modulation, and anchor-conditioned prediction, yet the paper does not explain what is technically new.

Questions for authors
  1. What exactly is the “w/o All” baseline, and how does it differ from iFVC? What accounts for its substantial PSNR advantage before any proposed component is enabled?
  2. What evidence links TraCER’s bit stabilization to the need for canonical renewal? Are there comparisons with fixed-period or quality-triggered renewal at matched storage and computation that separate the criterion’s benefit from the additional optimization after renewal?
  3. What is technically new in CaBLE relative to 3D-HGS and its Scaffold-HGS variant?
Reviewer_nsauOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

AncesTrA addresses reconstruction degradation and storage costs in long-horizon, streamable volumetric video. It combines three components: CiTER reconstructs each predictive frame from a shared canonical representation while inheriting, refining, and residual-coding successive transport parameters; TraCER renews the canonical reference using a trend-to-fluctuation statistic derived from binary transport-state departure; and CaBLE introduces asymmetric Gaussian kernels through a learned splitting plane and separate opacity coefficients for the two half-spaces. The paper evaluates reconstruction quality, encoded storage, training time, and rendering throughput on N3DV and Meeting Room, as well as the longer Bar, AvatarReX ZXC, and LBN2 sequences.

Critical strengths
  1. Substantial practical gains supported by multiple long sequences. The reported results show substantial improvements in reconstruction quality, storage efficiency, and training speed over the baseline across multiple long sequences. The consistency of these gains suggests that the benefits extend beyond a single scene. If independently reproducible, these combined improvements would be valuable for practical volumetric video reconstruction.

  2. A coherent separation of scene references and transport inheritance. Within a canonical interval, reconstructing frames from a fixed scene reference while inheriting transport parameters is a sensible way to reduce direct composition of frame-wise scene transformations.

Critical weaknesses
  1. CaBLE's novelty relative to 3D-HGS and Scaffold-HGS is not established. Section 3.4, particularly Equations (18)-(20), defines complementary half-Gaussians using a learned plane through the Gaussian center, shared geometry, and distinct opacity coefficients for the two sides. These central ingredients already appear in 3D-HGS: 3D Half-Gaussian Splatting (CVPR 2025). Its Scaffold-HGS implementation also predicts the splitting normal from voxel features using an MLP and expands the opacity output to two values, making the overlap extend to the feature-conditioned implementation emphasized here. The supplied manuscript does not cite or discuss this directly relevant work, and it does not establish a substantive structural difference from Scaffold-HGS. Recovering a symmetric Gaussian when the two opacities are equal is a special case of this parameterization; it does not by itself establish a new conditional selection mechanism. This overlap materially weakens one of the paper's three stated contributions. A useful integration into a temporal framework remains possible, but the independent novelty currently attributed to CaBLE is insufficiently supported.

  2. The experiments establish the usefulness of reanchoring more clearly than the value of TraCER's particular trigger. TraCER interprets a weak departure trend relative to local fluctuations as a signal to renew the canonical reference. However, this behavior can also occur when a scene is static or its motion becomes stable. Under Equations (11)-(13), a constant departure trajectory gives a zero regime statistic and can trigger renewal once the warm-up conditions are satisfied. Appendix D.6 appropriately acknowledges that stabilization does not establish exhausted transport capacity or geometric invalidity of the reference. The remaining question is whether this operational signal selects better renewal times than simpler alternatives. The ablation does not resolve that question. Table 3 shows that enabling TraCER increases PSNR from 31.50 to 32.71 dB while also increasing storage from 16.04 to 55.47 KB per frame. This supports a benefit from allocating additional resources to renewal, but does not distinguish that benefit from better selection of renewal times. Comparisons against fixed-period renewal and reconstruction-quality-based renewal under comparable actual storage and training budgets are missing.

  3. Long-horizon comparisons with recent methods remain limited. The standard-benchmark comparisons include representative methods from 2025 and 2026. However, coverage of recent methods is narrower in the long-horizon experiments: Bar includes D-FCGS, but omits MoRGS and ClipGStream, while the additional ZXC and LBN2 comparisons contain no methods from 2026. Since sustained reconstruction over long sequences is a central contribution, this limits the evidence for the claimed advantage over recent alternatives in the setting most relevant to the paper. Comparisons with recent streamable methods, particularly MoRGS and ClipGStream, on these long sequences under matched evaluation protocols and comparable resource budgets would strengthen the claim. If such comparisons are infeasible, the authors should explain the constraints and qualify the scope of their superiority claims.

  4. The overview figures are visually cluttered and difficult to follow. Figures 1 and 2 present too much information within crowded layouts, making it difficult to identify the main ideas and follow the relationships among the proposed components. This increases the effort required to understand the method. Simplifying the layouts, establishing a clearer visual hierarchy, and separating the overall workflow from detailed mechanisms would improve readability.

  5. The ablation baseline is insufficiently specified, obscuring the attribution of performance gains. The "w/o All" configuration already substantially outperforms iFVC on Bar despite removing all three proposed modules. Appendix H identifies the disabled modules but does not adequately specify the resulting reconstruction procedure, particularly the scene-reference strategy, parameter initialization, and optimization schedule when CiTER is removed. This configuration also takes considerably longer to train than iFVC, leaving optimization budget as a potential confounding factor. Consequently, the source of its strong performance and the extent to which the overall gains arise from the proposed modules remain unclear. The authors should fully specify this baseline, explain its differences from iFVC, and provide controlled comparisons that separate the effects of the underlying representation and training budget from those of the proposed components.

  6. The CaBLE ablation does not establish its benefit under comparable resource budgets. Compared with "w/o CaBLE," the complete model provides a modest PSNR improvement but requires more storage, renders more slowly, and takes longer to train. This demonstrates a quality-cost trade-off, but does not establish whether the asymmetric parameterization offers an advantage over simply allocating more resources to conventional Gaussian primitives. Although the paper states that CaBLE does not increase anchor density, anchor density alone does not establish equivalent Gaussian counts or rendering costs. The authors should report anchor and rendered-primitive counts for both configurations and compare CaBLE against a symmetric-Gaussian baseline with increased capacity under comparable storage, rendering, and training budgets. Without these controls, a contribution from greater representation capacity or primitive count cannot be ruled out.

  7. Ablation studies are restricted to a single scene. The module ablations are conducted only on Bar. Although the complete method is evaluated on additional sequences, those comparisons do not establish that the individual module benefits generalize across scenes and motion patterns. Repeating the key ablations on additional scenes, particularly ZXC and LBN2, would help determine whether the observed quality-efficiency trade-offs and component contributions are consistent beyond Bar.

Questions for authors
  1. Novelty clarification: What substantive mathematical or training differences distinguish CaBLE from 3D-HGS and Scaffold-HGS? When integrated into the same temporal framework, how does Scaffold-HGS compare with CaBLE?

  2. Renewal mechanism: Under comparable actual encoded storage and training budgets, how does TraCER compare with fixed-period and reconstruction-quality-based renewal? Please report renewal counts, interval-length distributions, and the relationship between trigger statistics and reconstruction error, including behavior during static or stable-motion periods.

  3. Ablation baseline: What reconstruction procedure remains in "w/o All," particularly its scene-reference strategy, parameter initialization, trainable parameters, and per-frame optimization schedule? Why does it already substantially outperform iFVC while taking longer to train? Please clarify differences in representation, losses, and coding settings, and provide controlled comparisons that distinguish these factors from the contributions of the proposed modules.

  4. CaBLE's quality-cost trade-off: What explains the increased storage, reduced rendering throughput, and longer training time when CaBLE is enabled? Please report anchor and rendered-primitive counts and compare against a symmetric-Gaussian baseline with increased capacity under comparable resource budgets. Does CaBLE retain an advantage after controlling for Gaussian count and representation capacity?

  5. Experimental coverage: Can the key module ablations be repeated on additional scenes, particularly ZXC and LBN2, to establish whether their benefits generalize beyond Bar? Can MoRGS and ClipGStream also be evaluated on the long sequences under matching protocols? If these comparisons are infeasible, please explain the constraints and qualify the corresponding claims.

  6. Presentation clarity: Can Figures 1 and 2 be simplified to distinguish the overall workflow from detailed mechanisms and make the relationships among CiTER, TraCER, and CaBLE easier to follow?

Reviewer_WQCdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents AncesTrA, a streamable volumetric video representation for reconstructing dynamic scenes from synchronized multi-view videos. It aims to maintain reconstruction quality over long sequences while reducing storage. Its core component, CiTER, reconstructs each frame from a shared canonical scene using frame-specific transport parameters, while inheriting and refining the previous frame’s transport state and entropy-coding binary transport residuals. TraCER adaptively promotes the current reconstruction to a new canonical reference based on transport dynamics. CaBLE further improves local expressivity by splitting Gaussian primitives into two half-spaces with independently predicted opacities. Experiments on short- and long-sequence benchmarks report improved reconstruction quality and storage efficiency compared with existing streamable methods. The method requires per-frame optimization; its streamability does not imply real-time reconstruction.

Critical strengths
  • Clear temporal representation design. Reconstructing each frame from a shared canonical scene while inheriting transport parameters separates the reconstruction reference from temporal initialization, reducing reliance on recursive scene-state transformations.
  • Strong reported quality–storage trade-off. On the 3,540-frame Bar sequence, AncesTrA achieves 32.71 dB PSNR at 55.47 KB/frame, compared with 27.43 dB at 327.68 KB/frame for Recon-GS.
  • Substantial experimental support. The evaluation includes short and long sequences, component ablations, parameter sensitivity studies, and rate–distortion analysis, providing evidence for both reconstruction quality and compression efficiency.
Critical weaknesses
  • The benefit of adaptive reanchoring is not sufficiently isolated. The ablations, sensitivity studies, and rate–distortion analysis support the usefulness of canonical renewal. However, removing TraCER reduces both reconstruction quality and storage substantially: PSNR decreases from 32.71 to 31.50 dB, while storage decreases from 55.47 to 16.04 KB/frame. This comparison does not establish whether the proposed adaptive policy is preferable to simpler periodic renewal under comparable storage and optimization budgets.

  • The connection between transport stabilization and beneficial renewal remains insufficiently demonstrated. Appendix D.6 appropriately acknowledges that stabilization does not establish canonical invalidity or exhausted transport capacity. Nevertheless, the paper provides limited direct evidence connecting this statistic to reconstruction degradation or the improvement obtained through renewal. This leaves uncertainty about its behavior during static intervals, motion pauses, or repetitive motion.

Questions for authors
  • Could you compare TraCER with fixed-period renewal under comparable storage and optimization budgets, reporting reconstruction quality, renewal counts, and processing time? Does adaptive triggering provide a consistent advantage?

  • Could you visualize the stabilization statistic alongside reconstruction error and renewal events, including static or repetitive-motion intervals? In particular, does triggering renewal at these points improve subsequent reconstruction relative to retaining the existing canonical reference?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.