- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper improves the coarse voxel structure generated in the first stage of image-to-3D generation. It addresses two problems: the output may have an unwanted orientation, and its shape may not match the input images. The method uses the first image to set the object’s azimuth while keeping it centered, normalized in scale, and upright. It also connects a VGGT-based image tower with a TRELLIS2-based geometry tower at each layer and trains them together for reconstruction and generation. During training, ground-truth geometry guides the attention between voxel and image tokens. Results on GSO and Toys4K show better coarse structures. Replacing the coarse stage in three existing pipelines also improves their final geometry without retraining the later stages.
Critical strengths
-
The results on GSO and Toys4K are strong. Replacing only the coarse stage improves the final geometry in all three tested pipelines, while keeping the refinement models unchanged.
-
The spatial-prior-guided attention provides a reasonable way to encourage pixel-to-voxel correspondence during training, and the ablation results show that it improves structure quality.
Critical weaknesses
-
The dual-tower design has limited novelty. ReconViaGen and Mix3R already combine reconstruction features with 3D generation.
-
Aligning the output with the input view has already been explored by Pixal3D. Coarse3D aligns only the azimuth and keeps the object upright, centered, and normalized in scale. This may be useful, but the paper does not directly test whether it is better than full view alignment.
-
Coarse3D uses about one million private training assets in addition to public data. The comparison does not control for this extra data, so it is hard to tell whether the gains come from the method or the training data. The paper also has no reproducibility statement and does not say whether the code, model weights, or private data will be released.
Questions for authors
-
Which cross-attention layers use the spatial prior loss? Is it applied to every attention head and every denoising timestep?
-
Can the authors compare their method with the baselines using the same public training data? This would help show how much of the improvement comes from the method rather than the extra private data.
-
Does mixing the spatial prior with the teacher’s attention work better than using either one alone, given the same training budget? How much do the results change with different values of β, and why was 0.2 chosen?
5 comments
W3 / Q2. Private data, matched comparisons, and reproducibility
Removing private assets. We trained a variant using only approximately 1M public assets from TRELLIS-500K and the filtered TexVerse subset, excluding the approximately 1M in-house assets from our training stages.
| Training assets | GSO IoU ↑ | GSO CD ↓ | Toys4K IoU ↑ | Toys4K CD ↓ |
|---|---|---|---|---|
| Public only, ~1M | 0.4943 | 0.0137 | 0.6192 | 0.0108 |
| Public + in-house, ~2M | 0.5101 | 0.0124 | 0.6315 | 0.0091 |
The public-only model still outperforms all compared baselines in coarse IoU and CD, including Pixal3D with ground-truth poses. The observed advantage therefore does not require the additional in-house assets.
This controls private-data use within Coarse3D, but does not retrain every baseline on identical data or match their capacity and pretraining. “Public-only” refers to our training-stage assets; both variants retain TRELLIS2/VGGT initialization.
Reproducibility. We recognize the importance of public access for reproducibility. While company policy prevents us from sharing the approximately 1M in-house assets, we are actively pursuing internal disclosure review and open-source approval for the inference code and model weights. Their release remains subject to approval, and we are not yet able to commit to a release date.
Q1. Which layers, heads, and timesteps are supervised?
The submitted model applies spatial-prior supervision to layers 16–30 (1-based), all 12 heads, and every sampled training timestep. These details are documented in Sec. 3.3 and Appendix A.2.3, pp. 15–16.
Layer selection
Starting from the same checkpoint, all variants receive 100K additional training steps:
| Supervised layers | IoU ↑ | CD ↓ | Peak memory ↓ |
|---|---|---|---|
| None | 0.5599 | 0.0103 | ~50 GB |
| 1–30 | 0.6318 | 0.0089 | ~110 GB |
| 11–30 | 0.6433 | 0.0087 | ~94 GB |
| 16–30 (default) | 0.6392 | 0.0087 | ~86 GB |
All tested ranges improve over the unguided control. Our default achieves the same reported CD as layers 11–30, with only 0.0041 lower IoU and approximately 8.5% less memory, making it a practical quality–memory trade-off.
The attention visualizations in Fig. 6, p. 16 show more localized voxel–image correspondence in deeper layers, motivating this choice. upervised layers explicitly compute attention maps; other layers retain FlashAttention. Memory measurements use gradient checkpointing and microbatch size 1 with four accumulation steps.
Head selection
| Supervised heads | IoU ↑ | CD ↓ |
|---|---|---|
| First 6 of 12 | 0.6253 | 0.0092 |
| All 12 | 0.6392 | 0.0087 |
All-head supervision performs better. Each head’s target includes its own teacher attention, retaining head-specific associations rather than imposing an identical purely geometric target.
Timestep coverage
The loss is applied at every sampled training timestep, without restricting the noise range. Our attention-map inspection did not reveal a consistent timestep-dependent pattern that would justify limiting supervision to a particular noise range. We therefore apply spatial guidance throughout the sampled denoising trajectory.
At inference, all layers use FlashAttention, with no spatial-prior construction or attention-supervision overhead.
Q3. Mixed targets and sensitivity to β
We compare targets under the same training schedule: the same 200K-asset subset and evaluation split as Table 4, the same starting checkpoint, and 100K fine-tuning steps at a learning rate of 1e-5. Only β changes:
Attention target = (1 − β) × teacher attention + β × spatial prior
| β | Target | IoU ↑ | CD ↓ |
|---|---|---|---|
| 0.0 | Teacher only | 0.5639 | 0.0103 |
| 0.1 | Mixed | 0.6215 | 0.0092 |
| 0.2 | Mixed (default) | 0.6392 | 0.0087 |
| 0.5 | Mixed | 0.6384 | 0.0089 |
| 1.0 | Spatial only | 0.6177 | 0.0094 |
Mixing outperforms either target alone. The default reduces CD by 15.5% versus teacher-only and 7.4% versus spatial-only supervision. The spatial prior provides geometric guidance, while teacher attention retains head-specific associations. Note that β = 0 still applies teacher-attention supervision; it is not the no-attention-loss control.
Why β = 0.2? It was the best tested value in our pre-submission ablation. However, β = 0.5 performs similarly, and all tested mixtures outperform both endpoints. Thus, the benefit is not restricted to a single precisely tuned value; we do not claim that 0.2 is uniquely optimal.
Dear Reviewer w6Ku,
Thank you again for your thoughtful questions about spatial-prior-guided attention. The newly uploaded supplement (pp. 15–17) addresses layer selection, head selection, timestep coverage, and sensitivity to β, including compute-matched comparisons of mixed and unmixed attention targets. We hope these additions clarify the design and its effectiveness.
As the discussion period is nearing its end, we would appreciate your feedback and would be happy to clarify any remaining questions.
Best regards,
The Authors
We thank the reviewer for recognizing the strong results, compatibility with frozen refinement stages, and effectiveness of spatial-prior-guided attention. We address the concerns below, combining W3 and Q2 on training data. Revisions are marked in red in the updated PDF.
W1. Novelty of the dual-tower design
We agree that combining reconstruction features with generation is not itself new. We have added the previously omitted Mix3R to the related work and distinguish our specific design as follows:
We added two ablations to test these choices.
Interaction direction
We compare our unidirectional conditioning with a Dual-DiT variant that adds reverse cross-attention, feeding geometry features back into the image tower.
Unidirectional conditioning improves both generation and reconstruction: CD decreases by 12.0%, RRE from 7.39 to 5.88, and RTE from 0.0805 to 0.0692. These results support keeping image conditioning independent of noisy geometry in our framework. Note that this is an interaction-direction ablation within Dual-DiT, not a reproduction of or direct comparison with Mix3R’s full MoT architecture.
Reconstruction supervision. With architecture, initialization, coordinate space, data, and schedule fixed, enabling the reconstruction loss gives:
These results support the value of explicit reconstruction supervision beyond simply adding a pretrained image tower (updated Table 4).
W2. Azimuth anchoring versus full view alignment
We agree that Pixal3D already explores input-view-aligned generation. Our distinction is azimuth-only anchoring, not view alignment itself. Retaining an upright, centered, scale-normalized representation provides:
In our evaluated multi-view configuration, Coarse3D also outperforms Pixal3D with ground-truth poses while using unposed inputs: GSO IoU/CD are 0.5101/0.0124, versus 0.4419/0.0181.
However, this end-to-end comparison does not isolate azimuth-only versus full alignment. We present our coordinate convention as a design suited to upright coarse-to-fine generation, not as proven universally superior to full camera alignment.