acceptodds
Under review as a conference paper at ICLR 2027

GRPQ: Structure-Preserving Alignment of 3D Diffusion Models with RGB-Only Rewards

Abstract

Aligning 3D diffusion models with specialized styles and human preferences is essential for producing usable assets, yet directly specifying such objectives in 3D remains difficult and costly. Generic 2D reward models provide abundant supervision, but optimizing their scores over rendered views can rapidly erode the structural prior of a pretrained 3D model. Existing safeguards often rely on structure-aware multi-view models or auxiliary depth and normal estimators, thereby limiting the range of applicable 2D rewards and potentially introducing estimator-specific biases. Our analysis reveals that pronounced disagreement between the cross-view rankings induced by 2D rewards and structural evidence often results in unreliable supervision and structural degradation; we refer to this phenomenon as reward–structural-evidence mismatch. Based on this finding, we introduce GRPQ, a prior-preserving alignment framework that uses the degree of mismatch as evidence for estimating the reliability of each view-level reward and assigns optimization weights accordingly. By suppressing harmful signals rather than introducing additional structural supervision, GRPQ enables generic RGB-only rewards to improve preference alignment while preserving the pretrained 3D structural prior. Extensive experiments across three alignment methods, multiple 3D diffusion models, and diverse reward models show that GRPQ improves structural integrity and prior retention while largely preserving reward gains and training efficiency. Process-level visualizations further show that GRPQ prevents optimization from favoring views that conceal geometric defects over those that reveal them, providing interpretable evidence for the proposed mechanism.

Primary area:
generative models
Keywords:
3D Generation

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_EZqsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GRPQ reweights rendered-view rewards when aligning a pretrained 3D diffusion model. It uses RGB edge density as a proxy for structural evidence, measures disagreement between reward and evidence rankings, and converts this disagreement into positive, normalized view weights with stopped gradients. The weighting is integrated into DDPO, ReFL, and Nabla-R2D3. Experiments use DiffSplat and GaussianCube with aesthetic and HPSv2 rewards, and assess reward gains, image-space metrics, and a proposed geometric self-consistency score, MGIS.

Critical strengths
  1. The proposed intervention is simple and addresses a relevant failure mode of rendered-view reward optimization. It can be applied to several alignment objectives without introducing a learned geometry estimator. Positive weights, normalization to unit mean, and stopping gradients through the confidence computation are clearly specified. This makes the method relatively easy to understand and potentially useful in existing alignment pipelines.

  2. The study spans two 3D generators, two rewards, and three optimizers. Table 1 reports MGIS improvements in all available comparisons, including sizeable gains on GaussianCube, while reward scores also improve. The main results section acknowledges that MV-FID changes are mixed. This is a better-supported, more qualified observation than the broader claim of uniform prior preservation.

Critical weaknesses
  1. The key reliability assumption is insufficiently validated. Edge density depends on texture, contrast, background, and projected scale as well as geometry; disagreement with an aesthetic or preference ranking can therefore be legitimate. Appendix A shows reward-richness plots, but does not establish that rank disagreement predicts harmful updates or structural collapse, nor compare the weighting against shuffled ranks or another simple view-weighting rule. Without such evidence, it is unclear whether GRPQ detects unreliable rewards or introduces an incidental image-complexity bias. This is central to the claimed mechanism, rather than a request for broader applications.

  2. The evidence for structural preservation needs a numerical and metric audit. Table 1 lists the DiffSplat/ReFL HPSv2 A-LPIPS entry as 0.035 (-0.075); under the stated additive-delta convention, this gives a negative perceptual distance. Several MV-FID entries worsen, so prior retention is not uniformly supported. MGIS is defined using depth and opacity rendered from the same asset, but no validation against human geometric judgments or controlled defects is provided. A simple flattened or oversmoothed asset can remain internally consistent, so improvement in this diagnostic alone does not establish preservation of useful 3D structure. Essential GRPQ settings, including the edge detector, number of views, and weight parameters, are also not numerically specified.

Questions for authors
  1. Does rank mismatch predict independently measured structural failures after controlling for texture, object scale, and viewpoint? Can you show a matched-budget comparison with shuffled evidence ranks and a simpler view-weighting baseline, together with the exact edge detector, camera distribution, view count, and weighting hyperparameters?

  2. Please correct or explain the negative implied A-LPIPS value and report absolute baseline/GRPQ scores with variation across seeds. How does MGIS rank controlled fragmentation, flattening, oversmoothing, and valid thin structures, and does it agree with independent geometric judgments? This would materially affect whether the reported improvement can be interpreted as structural preservation.

I lean toward weak rejection. The idea is practical and the reported geometric-score gains are encouraging, but the reliability proxy and the principal evaluation need stronger validation before the mechanism and preservation claims are convincing.

6 comments

Thank you for questioning the mismatch proxy and MGIS. We added cross-detector analysis, complementary metrics, corrections, and implementation details, while stating remaining gaps.

1. Correlation Experiments Linking Mismatch and Prior Degradation

Conclusion. Appendix C provides exploratory evidence connecting atypical reward–edge relationships with low geometric consistency. It does not establish predictive validity after controlling texture, scale, or viewpoint; mismatch is a risk proxy, not a correctness test.

1.1. Experimental Results and Interpretation

We examined 10 settings: Aesthetic Score and HPSv2, each with Gaussian–Canny, Bilateral–Sobel, Guided–Scharr, Non-Local Means–Canny, or HED [1–5]. Each setting contains 50 assets × 8 views = 400 observations; views from the lowest 20% by asset-level MGIS are highlighted (Appendix C, Figs. 6–10). Some low-MGIS cases exhibit unusual reward–edge patterns, but the plots do not quantify subsequent geometric damage. Texture, contrast, background, scale, and viewpoint affect edge density.

We seek a low-cost warning signal, not a deterministic relationship. We agree that controlled tests would strengthen the evidence. Independently controlling texture, scale, and viewpoint requires carefully matched 3D assets/renderings; testing harmful updates further requires matched-budget training interventions and independent pre/post-update geometry judgments. These demanding tests remain unfinished. We plan confounder controls, shuffled-rank and simple-weighting baselines, and paired geometry measures. Current gains do not rule out image-complexity bias or prove causality.

2. Validity of MGIS and Its Supporting Conclusions

Conclusion. We agree that MGIS measures rendered geometric self-consistency, not complete 3D correctness. We added learned 3D plausibility (3DP) and expert-assessed Janus Proportion; our conclusions now concern measured improvements over unweighted alignment, not universal prior preservation.

2.1. Additional Metrics and Results

MGIS limitations. Appendix D combines reprojection-based occupancy agreement, cross-view depth consistency, and local normal regularity. Fragmented but consistently rendered parts may go undetected; flattened geometry may remain reprojection-consistent; oversmoothing may increase normal regularity despite losing detail; and valid thin structures may be unfairly penalized at sharp boundaries or occlusions. We have not tested those controlled rankings or human-geometry agreement; MGIS gains alone are insufficient.

Complementary results. A frozen MiniCPM-o evaluator measures 3DP from 12-view RGB/normal renderings, and experts label Janus artifacts. Across 20 held-out prompts × 50 assets = 1,000 per configuration, revised Table 1 shows MGIS gains in 24/24 settings and Janus reductions in 24/24 (mean 5.8 percentage points, range 1.2–11.2). In contrast, 3DP improves in 13/24; reward improves in 20/24, and MV-FID versus unweighted alignment in 21/24. Thus we claim less cross-view inconsistency, not complete geometric restoration.

Numerical audit. The earlier 0.035 (-0.075) A-LPIPS entry was erroneous: standard LPIPS is non-negative [6]. The revised table corrects deltas and restores Nabla-R2D3/Aesthetic MGIS (0.478→0.487 for DiffSplat-PixArt-Σ; 0.472→0.478 for SD1.5). Paired seeds do not replace the unreported multi-seed variability or significance estimates.

3. Added Implementation Details

Conclusion. Revised Sec. 5.1 now specifies the eight-view camera orbit, Gaussian-smoothed Canny pipeline, all weighting hyperparameters, and the 80-train/20-held-out-prompt split. Paired runs match seeds and training/evaluation budgets; optimizer-specific schedules and controlled ablations remain unreported.

References

[1] Canny, A Computational Approach to Edge Detection, TPAMI 1986.
[2] Tomasi & Manduchi, Bilateral Filtering for Gray and Color Images, ICCV 1998.
[3] He et al., Guided Image Filtering, TPAMI 2013.
[4] Buades et al., A Non-Local Algorithm for Image Denoising, CVPR 2005.
[5] Xie & Tu, Holistically-Nested Edge Detection, ICCV 2015.
[6] Zhang et al., The Unreasonable Effectiveness of Deep Features as a Perceptual Metric, CVPR 2018.

Reviewer_EZqsOpenReview ↗

Remaining questions after the rebuttal

Thank you for the corrections and additional evaluations. The Janus results and implementation details strengthen the evidence. Two checks would materially affect my assessment:

  1. A matched-budget comparison against shuffled evidence ranks and a simple edge-based weighting baseline, reporting variation across seeds.
  2. A small controlled validation of whether MGIS distinguishes flattening and oversmoothing from valid thin structures, checked against independent human judgments.

These would clarify whether the gains depend on the proposed ranking mechanism and reflect useful geometry beyond self-consistency. I retain my rating of 2 for now and would reconsider it if these checks provide supporting evidence.

We sincerely thank the reviewer for the constructive and insightful suggestions. In particular, the requests for matched-budget comparisons and independent validation of geometric quality have been valuable in helping us strengthen our experimental evaluation. We provide the following two additional experiments to address these concerns.

1. Experiment I: Matched-Budget Comparison

1.1. Main Findings

We conduct a comprehensive matched-budget comparison across multiple random seeds, keeping all experimental settings identical except for the rank-related weighting strategy. The results indicate that GRPQ's rank-mismatch-based weighting achieves clear advantages in performance and stability over uniform view-reward averaging, richness-based weighting, and the shuffled evidence ranks control. The other three strategies exhibit relatively small performance differences, while uniform averaging demonstrates greater stability than richness-based weighting and shuffled evidence ranks.

1.2. Experimental Setup and Evaluation Metrics

Setup

  • Base models: DiffSplat-PixArt-Σ\Sigma and GaussianCube.
  • 2D rewards: Aesthetic Score and HPSv2.
  • Edge detector: Canny edge detector with Gaussian smoothing for noise suppression.
  • Training algorithm: All weighting strategies are trained using the same Nabla-based optimization procedure.
  • Prompts: T3Bench, with 80% used for training and 20% for testing. All comparison groups share the same data split.
  • Training steps: 256.
  • View count and camera poses: Eight uniformly spaced views around each asset.
  • Random seeds: Five distinct integer seeds, s1,…,s5s_1,\ldots,s_5, are predetermined, with their exact values recorded in the implementation configuration. Within each generator–reward setting, all four weighting strategies use the same five seeds to enable paired comparisons. Shuffled evidence ranks use an independent, reproducible random permutation stream, without modifying the training or sampling budgets of the other groups.
  • Evaluation sampling: Fifty assets are generated per test prompt. For each seed, all groups use the same test prompts and, wherever possible, identical generation randomness.

Baselines

Let K=8K=8 denote the number of views, rir_i the 2D reward of view ii, gi≥0g_i\geq 0 the geometric or edge-based richness evidence computed using the same procedure as Full GRPQ, and wiw_i the corresponding fusion weight. The four variants differ only in their view-weighting strategies, while sharing identical Nabla training procedures, reward evaluation budgets, view counts, and other configurations. A fixed constant ε>0\varepsilon>0 is used to prevent division by zero, with its exact value determined by the implementation.

  • Uniform averaging: All views receive equal weights, wiavg=1/Kw_i^{\mathrm{avg}}=1/K.

  • Richness-based (Edge-based) weighting: View weights depend exclusively on their individual richness scores, without considering discrepancies between reward and evidence rankings:

    wiedge=gi+ε∑j=1K(gj+ε).w_i^{\mathrm{edge}}=\frac{g_i+\varepsilon}{\sum_{j=1}^{K}(g_j+\varepsilon)}.

    The extraction and normalization of gig_i, including all Canny detector parameters, are identical to those used in Full GRPQ.

  • Shuffled evidence ranks: We preserve the original view rewards, richness scores, and all non-rank components. Specifically, only the evidence ranking πg\pi_g used in the rank-mismatch computation is replaced with π~g=σ(πg)\widetilde{\pi}_g=\sigma(\pi_g), where σ\sigma denotes a reproducible random permutation within each asset. The resulting rankings are processed using the same rank-based weighting formula as Full GRPQ. Reward rankings, images, asset identities, and non-rank features remain unchanged.

  • Full GRPQ: We use the original reward and evidence rankings to compute view weights according to the rank-mismatch weighting formula defined in the paper and implementation, without introducing additional assumptions or redefining the weighting function.

Evaluation Metrics

Reward, MV-FID, MV-CLIP, A-LPIPS, MGIS, 3DP, and Janus Proportion.

Reviewer_8XvUOpenReview ↗
Rating
4(+1)
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The central problem addressed in this paper is how to prevent the degradation of 3D structural quality when fine-tuning 3D generative models using 2D image-based rewards, even as reward scores improve. I believe this is an important problem with significant practical relevance. The proposed method is also relatively lightweight, and the experimental evaluation is sufficiently comprehensive. However, I would like the authors to provide further evidence or analysis to justify why inconsistencies in rankings necessarily indicate that the reward signal is unreliable.

Critical strengths

(1) 2D rewards evaluate rendered images, whereas the ultimate goal is to generate a plausible and geometrically consistent 3D object. There may be a gap between these two objectives: an object that looks more visually appealing from one viewpoint does not necessarily have better visual quality from other viewpoints, nor does it necessarily exhibit more reasonable topology or geometric structure. Rather than focusing solely on the overall reward score, this paper investigates whether supervision signals from different viewpoints are equally reliable. I believe this is a meaningful and worthwhile research question.

(2) The authors provide extensive experimental evidence to support their claims.

Critical weaknesses

(1) I would like the authors to provide further justification for why the reliability of the reward signal should be reduced when the reward ranking is inconsistent with the richness ranking.

(2) I am not sure whether my understanding is correct. Standard LPIPS is a non-negative distance metric, and the original LPIPS paper explicitly imposes non-negativity constraints on the weights. Therefore, if A-LPIPS here refers to the conventional average LPIPS distance, a negative value would be mathematically inconsistent with its definition. The authors should either correct the reported value or clearly explain whether they are using a different definition of A-LPIPS.

(3) The MGIS score for the DiffSplat / Nabla-R2D3 / Aesthetic configuration is not reported, and no explanation is provided for its omission. Since this configuration achieves a relatively high reward score, the missing MGIS result makes it difficult to fully assess the trade-off between reward optimization and structural quality. (Page 8, Table 1)

Questions for authors

(1) Could the authors provide empirical evidence demonstrating that larger ranking discrepancies are indeed more likely to correspond to harmful geometric updates?

(2) Whether an additional object-level suppression mechanism is necessary.

2 comments

Thank you for questioning the reliability assumption and identifying reporting errors. We clarify the mechanism, evidence, and limitations.

1. Mechanism: How Rank Mismatch Determines the Weights

Conclusion. Reward and structural-evidence rankings need not agree. GRPQ treats substantial disagreement as a potential structural-risk signal, not proof of an incorrect reward. Its rationale is conditional; downstream gains do not guarantee per-view correctness.

1.1. Intuitive Example

A high-scoring projection may conceal missing geometry, while a view revealing structural complexity scores lower due to clutter. Uniform optimization may favor the misleading view. Yet simple views can be valid: edge density is neither a geometry label nor an optimization target (Figs. 1, 4).

1.2. Theoretical Analysis

Let gig_i be the update contributed by view ii, Q(θ)Q(\theta) a hypothetical structural-quality objective, and hi=⟨∇θQ,gi⟩h_i=\langle\nabla_\theta Q,g_i\rangle. Consider the explicit assumption E[hi∣m]=a−bmi\mathbb E[h_i\mid\mathbf m]=a-bm_i, b≥0b\ge0, with rank displacement mim_i. GRPQ assigns qi=Nwi/∑jwjq_i=Nw_i/\sum_jw_j, where wi=wmin⁡+(1−wmin⁡)e−κumiw_i=w_{\min}+(1-w_{\min})e^{-\kappa u m_i}. Thus ∑iqi=N\sum_iq_i=N and qiq_i decreases with mim_i. For additive view updates,

=-b\operatorname{Cov}_N(\mathbf q,\mathbf m)\ge0.$$ Hence down-weighting improves the **expected first-order structural direction only if greater mismatch predicts less helpful updates**. This does **not** prove causality or final-training guarantees; rank displacement follows Spearman's footrule [1]. ### 1.3. Evidence of Improvement Revised **Table 1** shows increased **MGIS in 24/24** paired settings, decreased **Janus in 24/24** (mean **5.8 percentage points**), and increased reward in **20/24**. For DiffSplat-SD1.5/Nabla-R2D3/Aesthetic, Janus drops **41.0%→29.8%** and MGIS rises **0.472→0.478**. These results support the weighting rule, **not direct attribution of damage to individual gradients**. ### 1.4. Exploratory Results and Interpretation **Appendix C** examines **two rewards × five edge extractors**, each with **50 assets/400 views**, highlighting the lowest-MGIS 20%. Some low-MGIS assets exhibit atypical reward–edge patterns; these plots neither control confounders nor establish harmful updates. We seek a **cheap risk signal**, not a deterministic law. We agree that stronger tests require matched assets/renderings, **budget-matched optimization interventions**, and independent before/after geometry assessment. These demanding experiments **were not completed**. We plan controlled comparisons, shuffled-rank/simple weighting baselines, and paired geometry measurements. Current results do not validate causality. ## 2. Correction of Two Reporting Errors and Updated Results **Conclusion.** We apologize for the incorrect A-LPIPS delta and missing MGIS entry. Standard LPIPS is **non-negative** [2]; the former `0.035 (-0.075)` was a reporting error, not a different definition. Revised **Table 1** gives absolute unweighted values and signed GRPQ changes, and restores Nabla-R2D3/Aesthetic MGIS: **0.478→0.487** (DiffSplat-PixArt-Σ) and **0.472→0.478** (DiffSplat-SD1.5). Both remain below pretrained MGIS (**0.693**, **0.718**): we claim **mitigation**, not full restoration. ## 3. Necessity of Object-Level Suppression **Conclusion.** A separate asset-wide gate is **not needed for GRPQ's relative view reweighting**, though it may help in untested settings. ### 3.1. Theoretical and Practical Analysis GRPQ combines asset-level mismatch **prevalence and maximum severity** into $u$ (Eq. 6), which adjusts view weights while preserving $\sum_iq_i=N$. This preserves useful views, whereas an asset-wide gate attenuates *all* rewards. Large mismatches may also reflect legitimate complexity, viewpoint asymmetry, or preferences. However, **if every $m_i$ is identical, normalization forces $q_i=1$**, so GRPQ cannot attenuate that entire asset. A calibrated object-level gate might be complementary; **we have not tested it**. ### References [1] Diaconis & Graham, *Spearman's Footrule as a Measure of Disarray*, JRSS-B 1977. [2] Zhang et al., *The Unreasonable Effectiveness of Deep Features as a Perceptual Metric*, CVPR 2018.
Reviewer_8XvUOpenReview ↗

Keep my score

Thanks for the authors responce, especially the detailed experiment analysis.

I agree that some high-level experiments are not neccessary in this paper, and my main concerns all soulved in the rebuttle. I decide to keep my score and think this is a good paper desevered to be accepted.

Comments on the paper

8 comments
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank you for your invaluable feedback and the constructive discussion period. We have prepared a summary of our responses and outlined how we have addressed the reviewers' concerns. Please refer to our final official comment, General Response, for details.

We are encouraged that our rebuttal has addressed several of the reviewers' primary concerns and received positive reassessments from the review team. For your convenience, we summarize our responses below.


General Response

We would like to express our sincere gratitude for your valuable feedback and constructive suggestions. For your convenience, we summarize our responses and explain how we have addressed the reviewers' concerns. We hope this summary will facilitate your assessment and make the review process more efficient.

We have received encouraging comments from the reviewers, including "I believe this is a meaningful and worthwhile research question." and "easy to understand and potentially useful in existing alignment pipelines."

We outline our responses to the concerns raised by each reviewer below.

Common Concerns

Reviewers 87CF and 8XvU — Concern 1: Insufficient details in the experimental setup, along with missing results and typographical errors in some reported findings.

First, we made every effort to provide comprehensive experimental details, including both those explicitly requested by the reviewers and those not previously specified, such as hyperparameters, the number of cameras and camera poses, edge-detector configurations, and training and test dataset details.

Regarding the experimental results, we added DiffSplat-PixArt-Σ as an additional generator configuration and DRaFT as a comparison baseline. We also introduced two additional evaluation metrics: 3D Plausibility (3DP) and the Janus rate, the latter assessed by expert evaluators. These metrics complement, rather than replace, MGIS, reward scores, MV-FID, MV-CLIP, and A-LPIPS.

Reviewer 87CF

Concern 2: Why is diffusion model alignment necessary for style adaptation, rather than using alternatives such as agent-based generation models or feed-forward networks?

We clarified that the central contribution of our method is a way to express preferences with a lower risk of degrading the pretrained model, rather than a technique tied to any particular model architecture. We further explained that diffusion-based, feed-forward, and agent-based generation are not directly interchangeable approaches and cannot be treated as simple substitutes for one another. We supported this distinction with our experiments and results from existing studies.

The reviewer agreed with our explanation: "I agree that diffusion-based, feed-forward, and agent-based 3D generation have complementary roles."

Follow-up Concern 3: Does our method remain effective for image-to-3D diffusion models?

We conducted image-to-3D experiments on a scale comparable to that of our main experiments and demonstrated the effectiveness of our approach. We also explained that our method works by preserving the model's pretrained geometric generation capabilities during alignment, a mechanism that does not depend on the input modality.

We are awaiting the reviewer's final response.

Reviewer 8XvU

Concern 4: What is the mechanism behind our method's downweighting of mismatched rewards? Should suppression or downweighting instead be applied at the object level?

We explained the underlying mechanism of our method through intuitive examples, theoretical analysis, exploratory experiments, and the motivations behind our design. We also clarified why we do not adopt object-level downweighting.

The reviewer agreed with our explanation and increased both their score and confidence.

Reviewer EZqs

Concern 5: Controlled experiments are needed to verify the effectiveness of the mismatch-based insight, along with a small controlled study to assess MGIS's ability to evaluate geometric quality.

We conducted correlation experiments using 10 different combinations of reward models and edge detectors, providing evidence for the broad applicability of our insight. We also added two new geometry-related evaluation metrics to further substantiate our conclusions.

In subsequent supplementary responses, we completed a matched-budget comparison and a small controlled experiment on MGIS. These experiments supported the effectiveness of our approach while revealing limitations of MGIS in a small subset of cases. These findings motivated us to introduce two additional metrics to strengthen our evaluation framework.

We are awaiting the reviewer's final response.

AuthorsOpenReview ↗

Response to Reviewer 87CF — Part I (1/2)

Summary. GRPQ improves multi-view RGB reward reliability; it is not diffusion-specific. Diffusion, feed-forward systems, and agents have complementary roles. We separate style-supervision alternatives from generator architectures, compare workflows, outline application needs, and improve reproducibility. Only diffusion alignment is validated; cross-architecture use remains future work.

1. What GRPQ Addresses and Its Alternatives

A target 3D style can be expressed through language, reference images, preferences, or rewards. GRPQ aggregates 2D reward observations across rendered views into an asset-level objective; a single RGB reward model can provide all observations.

Each 2D score describes one view, omitting occluded and full 3D spatial evidence. A high-scoring projection may therefore conceal geometric defects. GRPQ uses reward–structural-evidence rank mismatch as a risk indicator and down-weights potentially misleading views. In our diffusion tests, this reduces structural degradation, including Janus artifacts, while largely retaining reward gains. Mismatch does not necessarily mean an incorrect reward, nor does GRPQ eliminate all defects.

Thus, our direct alternatives are other ways to specify and supervise style—text descriptions, reference images, native 3D rewards/preferences, and structural or multimodal supervision—rather than particular generator architectures. The principle is mathematically architecture-agnostic, but transfers require validation.

2. Generation Architectures and Complementarity

Conclusion. Diffusion/flow-based models learn detailed geometry and appearance; agents interpret open-ended instructions, plan, and use tools; feed-forward models offer fast reconstruction and decoding. These approaches can coexist. Rewards offer an explicit, repeatable style criterion alongside text or examples, especially for consistent adaptation across many assets. Their use with agents or feed-forward models remains a proposed extension, not a demonstrated GRPQ result.

2.1. Comparative Analysis and Relationship to GRPQ

Diffusion/flow-based 3D generators synthesize complex shapes and materials at a sampling cost. Feed-forward reconstruction is fast but relies on visual inputs rather than open-ended style reasoning. Agents turn language into modeling steps but may reproduce nuanced geometry and materials inconsistently.

Text alone may underspecify subjective or fine-grained styles. A calibrated reward supplies a measurable optimization or selection criterion, although it remains a proxy for intended 3D quality. Appendix. A, Fig. 2 and Table 1 show the effect of correcting view-level rewards in diffusion alignment; Appendix B includes an agent in a supplementary preference comparison. Neither establishes that reward conditioning causally outperforms text conditioning. Potential combinations include reward-guided agent candidate selection/refinement and reward-supervised feed-forward adaptation; both need direct testing.

2.2. Generation Efficiency and Quality

On 100 single-object T³Bench prompts [1], we compared five workflows. Image-conditioned methods shared one text-to-image reference pipeline; MV-CLIP used consistent cameras, lighting, and rendering resolution. Times are average end-to-end generation latencies.

WorkflowMV-CLIP ↑Time (s) ↓
TRELLIS.20.253243.1
Hunyuan3D0.244983.6
TripoSR0.22410.62
Stable Fast 3D0.22630.94
GPT-6 Astra + Blender0.2631931.4

The reconstruction systems are faster; the agent achieves the highest MV-CLIP at far higher latency. This is not a controlled style-adaptation or total-cost comparison: customization data, manual intervention, hardware expense, and diffusion fine-tuning costs are not matched. We use it to identify workflow trade-offs, not universal quality or cost superiority.

2.3. Human Preference Survey

Appendix B compares fine-tuned DiffSplat-SD1.5, DiffSplat-PixArt-Σ, GaussianCube, and GPT-6 Astra + Blender on 100 prompts (one asset per prompt and method). Participants viewed blinded, randomized 360° rotations and chose the best aesthetic quality and overall preference, or Neither; HPSv2 was computed separately. The appendix plots preferences and specifies a procedure for 95% intervals.

The survey adds perceptual evidence but does not isolate conditioning effects or assess geometry or cost. It also includes all 100 prompts, not just the 20 held-out prompts, so it cannot independently establish unseen-prompt generalization. Rater counts and numerical confidence intervals remain incompletely reported.

Reference: [1] He et al., T³Bench: Benchmarking Current Progress in Text-to-3D Generation (2023), arXiv:2310.02977.

AuthorsOpenReview ↗

Response to Reviewer 87CF — Part II (2/2)

2.4. Limitations of Agent-Based 3D Creation

Agentic reasoning does not yet ensure valid, editable 3D output. In BlenderWorld [13], the leading agent scores 79.9/100, but fully meets task requirements in only 42.2% of 223 cases. A CAD study [14] finds geometric-validity, parameter-dependency, and part-relation errors. 3DVerifierBench [15] reports a 21.6-percentage-point gap from human 3D verification. Frontier3DBench [16] finds reference-fidelity and multistep-interaction problems; 3DCodeBench [17] and BlenderGym [18] document failures in procedural modeling and graphics editing.

These are separate benchmarks, not head-to-head comparisons with GRPQ. They explain why agents' semantic and tool-use strengths can complement, rather than replace, generative priors and reliable reward evaluation.

3. Demand for Diverse Style Adaptation

3.1. Style Requirements

3D production spans photorealistic products, cinematic looks, painting-inspired or historical styles, pixel/voxel aesthetics, and cartoon rendering. It also demands hand-painted textures, PBR metals or glass, ceramics, and domain-specific identity, geometry, physical, or factual accuracy in branding, architecture, scientific illustration, and heritage. Consistency across views and assets matters beyond the appearance of one rendering.

Appendix A categorizes these needs by application, visual/geometric cues, evaluation criteria, and failure risks. It illustrates possible use cases; it neither measures market size nor claims that GRPQ was tested on every listed style.

3.2. Diffusion Fine-Tuning: Generality and Applications

Studies demonstrate few-shot adaptation to new styles [1,2] and efficient personalization, style–content separation, and composition [3–5]. Applications span traditional art [6], fashion [7], branding [8], artistic stylization [9], and video [10]; recent 3D systems support controllable asset generation and materials [11,12]. For rare styles applied to many assets, fine-tuning can adapt the output distribution after a one-time training investment. This motivates specialized 3D alignment, but most cited results concern 2D/video; they do not prove universal 3D quality/cost superiority or GRPQ effectiveness in every style.

4. Added Implementation Details

Conclusion. We accept that the original protocol was insufficiently specified and that prior preservation was overstated. Sec. 5.1 now gives the GRPQ and evaluation settings. The correct claim is mitigating degradation relative to unweighted alignment, not restoring the pretrained prior.

We use 8 azimuthal views over 360° (45° intervals; fixed elevation, radius, and intrinsics), Gaussian-smoothed Canny edges [19] (5×5, σ=1.0, thresholds 50/150), and τ_m=0.25, w_min=0.25, κ=3, λ_cnt=0.5, λ_sev=0.99. Of 100 T³Bench prompts, 80 train / 20 test; each test prompt yields 50 assets. Comparisons match seeds, rendering, sampling, training budget, reward evaluation, and checkpoint selection. Optimizer-specific learning rates, batch sizes, and step counts still need a consolidated table.

For DiffSplat-SD1.5 / Nabla-R2D3 / Aesthetic, MGIS rises 0.472→0.478 and Janus falls 41.0%→29.8% with GRPQ; pretrained MGIS is 0.718. This quantifies meaningful mitigation without implying full recovery.

References (Part II)

  • [1] Lu et al., Specialist Diffusion, CVPR 2023.
  • [2] Everaert et al., Diffusion in Style, ICCV 2023.
  • [3] Frenkel et al., Implicit Style-Content Separation Using B-LoRA, ECCV 2024.
  • [4] Shah et al., ZipLoRA, ECCV 2024.
  • [5] Jones et al., Customizing Text-to-Image Models with a Single Image Pair, SIGGRAPH Asia 2024.
  • [6] Lee et al., Style Composition within Distinct LoRA Modules for Traditional Art, ICCVW 2025.
  • [7] Chen and Ma, Fashion Design Combining Attribute Knowledge and Stable Diffusion, Textile Research Journal 2025.
  • [8] Zhao and Liang, Diffusion Models for Brand Logo Style Transformation, JOEUC 2026.
  • [9] Chen et al., Art Creator: Steering Styles in Diffusion Model, Neurocomputing 2025.
  • [10] Liu et al., StyleCrafter, ACM TOG 2024.
  • [11] Li et al., Step1X-3D, arXiv:2505.07747.
  • [12] Tencent Hunyuan3D Team, Hunyuan3D 2.1, arXiv:2506.15442.
  • [13] Fang et al., BlenderWorld, arXiv:2610.23986.
  • [14] Zou et al., Assessing the CAD Capabilities of General-Purpose AI Models, arXiv:2610.24451.
  • [15] Li et al., Grounding Generative Verification in 3D Content, arXiv:2610.22360.
  • [16] Lin et al., Frontier3DBench, arXiv:2611.09017.
  • [17] Gao et al., 3DCodeBench, arXiv:2606.01057.
  • [18] Gu et al., BlenderGym, CVPR 2025.
  • [19] Canny, A Computational Approach to Edge Detection, IEEE TPAMI 1986.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.