- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
GRPQ reweights rendered-view rewards when aligning a pretrained 3D diffusion model. It uses RGB edge density as a proxy for structural evidence, measures disagreement between reward and evidence rankings, and converts this disagreement into positive, normalized view weights with stopped gradients. The weighting is integrated into DDPO, ReFL, and Nabla-R2D3. Experiments use DiffSplat and GaussianCube with aesthetic and HPSv2 rewards, and assess reward gains, image-space metrics, and a proposed geometric self-consistency score, MGIS.
Critical strengths
-
The proposed intervention is simple and addresses a relevant failure mode of rendered-view reward optimization. It can be applied to several alignment objectives without introducing a learned geometry estimator. Positive weights, normalization to unit mean, and stopping gradients through the confidence computation are clearly specified. This makes the method relatively easy to understand and potentially useful in existing alignment pipelines.
-
The study spans two 3D generators, two rewards, and three optimizers. Table 1 reports MGIS improvements in all available comparisons, including sizeable gains on GaussianCube, while reward scores also improve. The main results section acknowledges that MV-FID changes are mixed. This is a better-supported, more qualified observation than the broader claim of uniform prior preservation.
Critical weaknesses
-
The key reliability assumption is insufficiently validated. Edge density depends on texture, contrast, background, and projected scale as well as geometry; disagreement with an aesthetic or preference ranking can therefore be legitimate. Appendix A shows reward-richness plots, but does not establish that rank disagreement predicts harmful updates or structural collapse, nor compare the weighting against shuffled ranks or another simple view-weighting rule. Without such evidence, it is unclear whether GRPQ detects unreliable rewards or introduces an incidental image-complexity bias. This is central to the claimed mechanism, rather than a request for broader applications.
-
The evidence for structural preservation needs a numerical and metric audit. Table 1 lists the DiffSplat/ReFL HPSv2 A-LPIPS entry as 0.035 (-0.075); under the stated additive-delta convention, this gives a negative perceptual distance. Several MV-FID entries worsen, so prior retention is not uniformly supported. MGIS is defined using depth and opacity rendered from the same asset, but no validation against human geometric judgments or controlled defects is provided. A simple flattened or oversmoothed asset can remain internally consistent, so improvement in this diagnostic alone does not establish preservation of useful 3D structure. Essential GRPQ settings, including the edge detector, number of views, and weight parameters, are also not numerically specified.
Questions for authors
-
Does rank mismatch predict independently measured structural failures after controlling for texture, object scale, and viewpoint? Can you show a matched-budget comparison with shuffled evidence ranks and a simpler view-weighting baseline, together with the exact edge detector, camera distribution, view count, and weighting hyperparameters?
-
Please correct or explain the negative implied A-LPIPS value and report absolute baseline/GRPQ scores with variation across seeds. How does MGIS rank controlled fragmentation, flattening, oversmoothing, and valid thin structures, and does it agree with independent geometric judgments? This would materially affect whether the reported improvement can be interpreted as structural preservation.
I lean toward weak rejection. The idea is practical and the reported geometric-score gains are encouraging, but the reliability proxy and the principal evaluation need stronger validation before the mechanism and preservation claims are convincing.
6 comments
Remaining questions after the rebuttal
Thank you for the corrections and additional evaluations. The Janus results and implementation details strengthen the evidence. Two checks would materially affect my assessment:
- A matched-budget comparison against shuffled evidence ranks and a simple edge-based weighting baseline, reporting variation across seeds.
- A small controlled validation of whether MGIS distinguishes flattening and oversmoothing from valid thin structures, checked against independent human judgments.
These would clarify whether the gains depend on the proposed ranking mechanism and reflect useful geometry beyond self-consistency. I retain my rating of 2 for now and would reconsider it if these checks provide supporting evidence.
Part 1
We sincerely thank the reviewer for the constructive and insightful suggestions. In particular, the requests for matched-budget comparisons and independent validation of geometric quality have been valuable in helping us strengthen our experimental evaluation. We provide the following two additional experiments to address these concerns.
1. Experiment I: Matched-Budget Comparison
1.1. Main Findings
We conduct a comprehensive matched-budget comparison across multiple random seeds, keeping all experimental settings identical except for the rank-related weighting strategy. The results indicate that GRPQ's rank-mismatch-based weighting achieves clear advantages in performance and stability over uniform view-reward averaging, richness-based weighting, and the shuffled evidence ranks control. The other three strategies exhibit relatively small performance differences, while uniform averaging demonstrates greater stability than richness-based weighting and shuffled evidence ranks.
1.2. Experimental Setup and Evaluation Metrics
Setup
- Base models: DiffSplat-PixArt- and GaussianCube.
- 2D rewards: Aesthetic Score and HPSv2.
- Edge detector: Canny edge detector with Gaussian smoothing for noise suppression.
- Training algorithm: All weighting strategies are trained using the same Nabla-based optimization procedure.
- Prompts: T3Bench, with 80% used for training and 20% for testing. All comparison groups share the same data split.
- Training steps: 256.
- View count and camera poses: Eight uniformly spaced views around each asset.
- Random seeds: Five distinct integer seeds, , are predetermined, with their exact values recorded in the implementation configuration. Within each generator–reward setting, all four weighting strategies use the same five seeds to enable paired comparisons. Shuffled evidence ranks use an independent, reproducible random permutation stream, without modifying the training or sampling budgets of the other groups.
- Evaluation sampling: Fifty assets are generated per test prompt. For each seed, all groups use the same test prompts and, wherever possible, identical generation randomness.
Baselines
Let denote the number of views, the 2D reward of view , the geometric or edge-based richness evidence computed using the same procedure as Full GRPQ, and the corresponding fusion weight. The four variants differ only in their view-weighting strategies, while sharing identical Nabla training procedures, reward evaluation budgets, view counts, and other configurations. A fixed constant is used to prevent division by zero, with its exact value determined by the implementation.
-
Uniform averaging: All views receive equal weights, .
-
Richness-based (Edge-based) weighting: View weights depend exclusively on their individual richness scores, without considering discrepancies between reward and evidence rankings:
The extraction and normalization of , including all Canny detector parameters, are identical to those used in Full GRPQ.
-
Shuffled evidence ranks: We preserve the original view rewards, richness scores, and all non-rank components. Specifically, only the evidence ranking used in the rank-mismatch computation is replaced with , where denotes a reproducible random permutation within each asset. The resulting rankings are processed using the same rank-based weighting formula as Full GRPQ. Reward rankings, images, asset identities, and non-rank features remain unchanged.
-
Full GRPQ: We use the original reward and evidence rankings to compute view weights according to the rank-mismatch weighting formula defined in the paper and implementation, without introducing additional assumptions or redefining the weighting function.
Evaluation Metrics
Reward, MV-FID, MV-CLIP, A-LPIPS, MGIS, 3DP, and Janus Proportion.
Thank you for questioning the mismatch proxy and MGIS. We added cross-detector analysis, complementary metrics, corrections, and implementation details, while stating remaining gaps.
1. Correlation Experiments Linking Mismatch and Prior Degradation
Conclusion. Appendix C provides exploratory evidence connecting atypical reward–edge relationships with low geometric consistency. It does not establish predictive validity after controlling texture, scale, or viewpoint; mismatch is a risk proxy, not a correctness test.
1.1. Experimental Results and Interpretation
We examined 10 settings: Aesthetic Score and HPSv2, each with Gaussian–Canny, Bilateral–Sobel, Guided–Scharr, Non-Local Means–Canny, or HED [1–5]. Each setting contains 50 assets × 8 views = 400 observations; views from the lowest 20% by asset-level MGIS are highlighted (Appendix C, Figs. 6–10). Some low-MGIS cases exhibit unusual reward–edge patterns, but the plots do not quantify subsequent geometric damage. Texture, contrast, background, scale, and viewpoint affect edge density.
We seek a low-cost warning signal, not a deterministic relationship. We agree that controlled tests would strengthen the evidence. Independently controlling texture, scale, and viewpoint requires carefully matched 3D assets/renderings; testing harmful updates further requires matched-budget training interventions and independent pre/post-update geometry judgments. These demanding tests remain unfinished. We plan confounder controls, shuffled-rank and simple-weighting baselines, and paired geometry measures. Current gains do not rule out image-complexity bias or prove causality.
2. Validity of MGIS and Its Supporting Conclusions
Conclusion. We agree that MGIS measures rendered geometric self-consistency, not complete 3D correctness. We added learned 3D plausibility (3DP) and expert-assessed Janus Proportion; our conclusions now concern measured improvements over unweighted alignment, not universal prior preservation.
2.1. Additional Metrics and Results
MGIS limitations. Appendix D combines reprojection-based occupancy agreement, cross-view depth consistency, and local normal regularity. Fragmented but consistently rendered parts may go undetected; flattened geometry may remain reprojection-consistent; oversmoothing may increase normal regularity despite losing detail; and valid thin structures may be unfairly penalized at sharp boundaries or occlusions. We have not tested those controlled rankings or human-geometry agreement; MGIS gains alone are insufficient.
Complementary results. A frozen MiniCPM-o evaluator measures 3DP from 12-view RGB/normal renderings, and experts label Janus artifacts. Across 20 held-out prompts × 50 assets = 1,000 per configuration, revised Table 1 shows MGIS gains in 24/24 settings and Janus reductions in 24/24 (mean 5.8 percentage points, range 1.2–11.2). In contrast, 3DP improves in 13/24; reward improves in 20/24, and MV-FID versus unweighted alignment in 21/24. Thus we claim less cross-view inconsistency, not complete geometric restoration.
Numerical audit. The earlier
0.035 (-0.075)A-LPIPS entry was erroneous: standard LPIPS is non-negative [6]. The revised table corrects deltas and restores Nabla-R2D3/Aesthetic MGIS (0.478→0.487 for DiffSplat-PixArt-Σ; 0.472→0.478 for SD1.5). Paired seeds do not replace the unreported multi-seed variability or significance estimates.3. Added Implementation Details
Conclusion. Revised Sec. 5.1 now specifies the eight-view camera orbit, Gaussian-smoothed Canny pipeline, all weighting hyperparameters, and the 80-train/20-held-out-prompt split. Paired runs match seeds and training/evaluation budgets; optimizer-specific schedules and controlled ablations remain unreported.
References
[1] Canny, A Computational Approach to Edge Detection, TPAMI 1986.
[2] Tomasi & Manduchi, Bilateral Filtering for Gray and Color Images, ICCV 1998.
[3] He et al., Guided Image Filtering, TPAMI 2013.
[4] Buades et al., A Non-Local Algorithm for Image Denoising, CVPR 2005.
[5] Xie & Tu, Holistically-Nested Edge Detection, ICCV 2015.
[6] Zhang et al., The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, CVPR 2018.