acceptodds
Under review as a conference paper at ICLR 2027

SAGE: Sparse-View Human Compression via Source-Anchored Gaussian Reconstruction

Abstract

Human multiview compression must preserve a person's appearance across camera queries while transmitting only a few captured views. A generative prior can synthesize missing viewpoints, but its inferred appearance and geometry may deviate from the captured subject. When decoded camera images and generated views jointly supervise a reconstruction, the more numerous generated views can dominate optimization and weaken captured detail. We propose SAGE, Source-Anchored Gaussian Encoding, a sparse-view human compression framework that codes selected images and geometric conditions, constructs a reusable Gaussian representation with a fixed prior, and renders later viewpoints from that representation. Its key mechanism is source anchoring: decoded captured views receive more frequent and stronger photometric supervision, while generated views extend coverage. A later reduction in generated-view weight further steers optimization toward observed appearance. On the evaluated DNA-Rendering captures, SAGE achieves the smallest average complete package and leads the compared methods on all four quality measures. On calibrated THuman2.0 scans, it improves foreground fidelity and perceptual quality over size-aligned MV-HEVC under a common Gaussian receiver. These results show how compact human-view transmission can exploit generated coverage while preserving the distinctive appearance evidence in coded observations across camera queries.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Human-Centric Compression, Sparse-View Generative Reconstruction, Gaussian Splatting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3wqCOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a new pipeline, called SAGE, for compressing multi-view human captures. Eight views are sampled from the 48 captured views. These source views are encoded together with the camera parameters, poses, and masks, and transmitted by the sender. The receiver decodes the source data and uses a multi-view diffusion network to complete the missing views with generated ones. It then reconstructs a 3D Gaussian scene from both the decoded and the generated images. The main idea of the paper is to give more weight to the source views than to the generated ones (a higher selection frequency during optimization and higher weights in the loss). Another contribution is the use of “source-cycle predictability” to assign more bits to the source views whose neighboring views the generator struggles to recreate. The authors report a slight improvement over SOTA methods in terms of both reconstruction metrics and size.

Critical strengths
  • The tackled problem is well motivated. In this setting, 40 generated views compete with 8 source views. Under uniform sampling and equal weights, the generator’s errors will considerably degrade the reconstruction. However, in the proposed approach the source views get 0.59 of the expected supervision instead of (1/6).

  • The ablations reported in the main paper and the supplement are quite extensive. They also include some mixed and negative results.

Critical weaknesses
  • The novelty of this paper is incremental. It seems to me that the core mechanism of this approach is just the loss reweighting. In Table 8 (Supplement Materials), the "Coefficient matched" case yields the greatest improvement over the "Uniform" case in PSNR. It reaches 24.85 dB, compared with 24.89 dB for full anchoring. So “source anchoring” is essentially “weight real views more than pseudo-views.”

  • The paper shows that the generated RGB does not help at all. In Table 3, the "Source-only" experiment, which ignores generated RGB entirely, yields better PSNR and DISTS than the "Source anchoring".

  • Some paragraphs are very unclear and were very hard to understand on first reading.

  • It would be more convincing to compare against recent diffusion-assisted reconstruction methods under similar reconstruction budgets and priors. The paper cites some of these methods but does not provide direct comparisons.

Questions for authors
  • How does a uniform sampler with a tuned weight ratio (a sweep over λ_s/λ_g) compare with anchoring?

  • How sensitive are the reported gains to the choice and quality of the prior (Diffuman4D) ?

  • How would the approach extend to uncalibrated settings, dynamic scenes, or non-human objects?

Reviewer_zLshOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SAGE transmits sparse human RGB views and geometric conditions, uses frozen Diffuman4D to generate complementary views, and optimizes a reusable Gaussian scene. Its main contribution is stronger supervision from decoded observations through sampling, loss weighting, and decreasing generated-view supervision; source-cycle predictability also guides QP allocation. Experiments report a 2.96 dB BBox PSNR improvement over uniform supervision on three DNA-Rendering identities, a 0.85 dB improvement over size-aligned MV-HEVC on four THuman2.0 scans, and transfer to three additional scans. Main results use direct Gaussian rendering; RGB refinement is optional.

Critical strengths
  • A consequential problem with useful controlled evidence: Preserving captured appearance while exploiting imperfect generated coverage matters for compact free-viewpoint human media. The fixed-input supervision study gains 2.96 dB BBox PSNR; excluded-camera experiments retain a 2.34 dB mean gain across repeated complete reconstructions (Tables 2 and 19).
  • Careful separation of transmission and reconstruction: Complete packages include conditions and overhead, and common-receiver comparisons separate coding from scene supervision. Coefficient matching, source-only controls, and storage/latency reporting make the practical trade-offs unusually inspectable.
Critical weaknesses
  • Insufficiently distinct methodological contribution. Diffuman4D already reconstructs Gaussians from observed and generated human views; ReconFusion explicitly separates observation and generated-view losses and linearly decays the latter from 1 to 0.1 (CVPR 2024, §3.3); ReconX already uses confidence-aware Gaussian optimization. Here, coefficient-matched uniform sampling reaches 24.85 versus 24.89 dB for anchoring (Table 8). These results support useful supervision tuning, but the paper does not establish a sufficiently distinct advance over these precedents. Source-cycle allocation is also modest: 28.76 versus 28.69 dB canvas PSNR on two development captures, with texture allocation reaching 28.77 dB (Table 11).
  • The experiments do not yet establish the proposed recipe as the preferred solution. On THuman2.0, source-only beats the original anchored rule in BBox PSNR, LPIPS, and DISTS; support masks without generated RGB improve PSNR and LPIPS (Table 3). Staged support improves fidelity (Table 6), but its advantage needs independent validation against these alternatives. There is no matched comparison to prior reliability/annealing schemes or a human-specific receiver such as GPS-Gaussian; the single general-scene MVSGaussian transfer is insufficient. GauMVC and Argus are cited but not experimentally positioned, leaving the compression claim incompletely supported.
Questions for authors
  • Clarification and decisive comparison: What methodological element goes beyond ReconFusion's separate observation loss and annealed generated-view loss, and ReconX's confidence weighting? Can an adapted comparison hold the coded inputs, frozen human prior, initialization, and reconstruction budget fixed?
  • Evidence needed to resolve the ablation trade-off: On independently held-out identities, how do source-only, support-mask-only, constant anchoring, and staged/annealed support compare at matched bytes and budgets? Please document exclusion from the prior's training set and report all four BBox metrics.
  • Baseline coverage: Can GPS-Gaussian be evaluated from the same decoded sources, with setup and query costs reported? For GauMVC/Argus, please establish which capture, temporal, or information-access assumptions prevent a meaningful comparison, or provide one where compatible.
Reviewer_mEH1OpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SAGE addresses sparse-view human compression by transmitting selected coded images and geometric conditions, then using a frozen Diffuman4D prior and iterative Gaussian reconstruction at the receiver. Its main strategy, source anchoring, increases the sampling frequency and photometric weight of decoded source views and reduces generated-view supervision later in optimization. It also allocates source coding bits using leave-one-out generative predictability. Experiments on DNA-Rendering and THuman2.0 evaluate coding frontends and supervision rules, primarily through direct Gaussian rendering, with optional image-space refinement studied separately.

Critical strengths
  1. The paper addresses a concrete failure mode: numerous generated views can overwhelm captured evidence during reconstruction. Matched-input controls demonstrate a substantial improvement over uniform supervision, including a 2.96 dB BBox PSNR gain on DNA-Rendering. The coefficient-matched and source-only controls help clarify the source of this improvement.
  2. The evaluation accounts for complete transmitted packages and distinguishes scene preparation from repeated rendering, making the trade-off between transmission cost and receiver computation more explicit.
Critical weaknesses
  1. Limited methodological novelty. Using diffusion-generated views to supplement sparse observations while preserving captured appearance is established in prior reconstruction work, including GuidedVD-3DGS and ArtiFixer. The paper does not sufficiently establish what is technically new in its source-dependent sampling, photometric weighting, and supervision schedule. Its coefficient-matched uniform sampler achieves 24.85 dB versus 24.89 dB for the joint strategy, suggesting that most of the benefit comes from changing the expected supervision coefficients. While the compression setting and source-cycle allocation should be assessed separately, the novelty of the prominently claimed reconstruction principle remains limited.

  2. Insufficient evidence of benefit over simpler alternatives. Although anchoring improves substantially over uniform supervision, Table 3 shows that source-only reconstruction outperforms the original anchored method in PSNR, LPIPS, and DISTS. The support-mask variant offers only modest and mixed improvements over source-only: +0.30 dB PSNR, nearly unchanged LPIPS, and worse DISTS. These results do not establish a consistent benefit from generated RGB supervision. Furthermore, the absence of a systematic Diffuman4D-only comparison leaves the added value of costly Gaussian optimization over direct generation unclear. The visual comparisons in Figure 5 also do not clearly substantiate the claimed improvements in captured detail.

Questions for authors
  1. What specifically distinguishes source anchoring from existing observed/generated supervision strategies? Please clarify the technical differences in sampling, weighting, and scheduling relative to relevant prior work.

  2. Can you compare Diffuman4D-only, source-only Gaussian reconstruction, SAGE Gaussian-only, and SAGE Full using the same decoded sources, conditions, and target cameras? Under what conditions does generated RGB supervision consistently outperform source-only or support-mask alternatives? Please report quality alongside preparation and query costs.

  3. Since optional RGB refinement modifies individual rendered images without updating the Gaussian scene, what evidence demonstrates that Full outputs retain cross-view consistency? Please distinguish the claims supported for Gaussian-only and Full outputs.

  4. Other presentation problems :

• Please cite DNA-Rendering and THuman2.0 when they are first mentioned in the Introduction.

• Please move specific hyperparameters and training iteration counts to Implementation Details so that the Method section focuses more clearly on the core design and technical contributions.

• The visual differences in Figure 5 are difficult to discern. Please provide additional examples with clearer comparisons to substantiate the claimed improvements.

• Although Figure 3 shows clearer facial details, noticeable artifacts remain along the arm boundaries. Please discuss these artifacts and include enlarged boundary crops for comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.