acceptodds
Under review as a conference paper at ICLR 2027

Backdrop: Enabling Any-Reference 3D Generation with One Line of Code

Abstract

We propose Backdrop, a one-line-of-code intervention that enables controllable 3D generation from arbitrary image references. Existing image-conditioned 3D generators provide a convenient interface for reference-based control, but this interface implicitly assumes object-centric images. When the reference is instead a texture, scene, painting, or material photograph, existing stylization methods can severely distort or fragment the generated geometry. We trace this failure to a mismatch in conditioning structure: object-centric references contain a large population of background patch tokens that absorb substantial cross-attention in deeper generator blocks, while full-frame references do not. Removing these tokens from object references reproduces the failure, while restoring them to non-object references repairs it. Backdrop exploits this observation by appending a cached set of patch tokens from a single blank image to the reference condition. This restores the generator's expected fallback population without changing the reference, retraining the model, optimizing per instance, or altering the underlying control method. Across diverse textures, scenes and artistic images, and with different 3D stylization methods, consistently improves geometric integrity while preserving strong reference transfer and content shape, making arbitrary images reliable controls for 3D generation. Results and code are available at https://backdrop-3d.pages.dev/.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D generation, 3D stylization, style transfer, image-conditioned 3D generation, reference-based control, cross-attention, attention sinks, training-free

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_dqbsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents a method called Backdrop which aims to provide 3D stylization via a full-frame texture images. The authors make the observations that in SoTA 3D generation methods such as Trellis 2, while object reference images work, full texture images result in artifacts. They observe that this is because the background patches in object reference images observe substantial cross attention, and without this attention destablizes geometry.

Backdrop appends additional patch tokens from a blank image to overcome this challenge. It's training free and the method is shown on top of Trellis 2 with Style Sculptor and MorpAny 3D.

Critical strengths
  • Interesting observation that the redundant background tokens play a crucial role for stylization. The paper does various experiments (e.g., removing background tokens from object references also result in artifacts) to show this.
  • The method is training free and is adapted to both Style Sculptor and Morph Anyhting 3D.
Critical weaknesses
  • Important closely related work is missing: Oztas et al., 3D Stylization via Large Reconstruction Model, SIGGRAPH 2025. That work already demonstrates training-free 3D appearance stylization by injecting reference-image features into selected cross-attention layers of a pretrained reconstruction model. It also examines how layer selection affects geometry preservation. This work is directly relevant to the attention-blending baseline and the broader claim of enabling reference-guided stylization in frozen 3D models.

  • The pipeline is shown successfully on top of a single generator Trellis 2. The experiments on Trellis show that is behaviour is different and Backdrop may not be as necessary. And for Hunyuan3D, the proposed solution is not as effective. While these are interesting observations, it also means the solution is not general and more architecture specific.

  • The authors already mention that addition of blank image tokens, in a way, is adjusting the attention scale of the reference tokens. The zero-value-token variant and fixed attention scaling ablations already are effective. This finding is also related to the above mentioned related work, Oztas et al. Hence, the tecnhical contribution is not justified.

  • The evaluation section would benefit from choosing metrics that separately evaluate stylization quality, geometric quality.

Questions for authors
  • How does your attention-blending baseline relate to Oztas et al.’s work? Would restricting style injection to selected layers, as studied there, mitigate collapse on TRELLIS.2 without appended tokens?

  • How sensitive are the results to the number of appended tokens? Can a substantially smaller subset retain the benefits, and how does the required number relate to reference composition or style strength?

-Which architectural properties determine whether BACKDROP is needed and effective? Can you demonstrate a successful repair on another generator, rather than only an attention-allocation similarity or a different unresolved failure?

  • Can you evaluate geometric-style transfer separately from texture and color transfer, ideally with human judgments?
5 comments

Thank you for the review and for highlighting Öztaş et al. We would like to clarify several important distinctions regarding our contributions and provide additional experimental evidence addressing the concerns. We have uploaded a revised PDF with changes marked in blue.

W1, Q1: Related work and layer-selective injection.

We have included Öztaş et al. in the revision (Section 2). However, their work pursues a different technical objective from BACKDROP. They propose a stylization method that injects reference features into the last 4 of 16 cross-attention blocks to transfer appearance while preserving geometry. In contrast, BACKDROP is not another stylization method. Our contribution is identifying why existing methods fail with non-object references and resolving this failure to enable both appearance and geometric stylization.

Our attention-blending baseline follows the same weighted combination of content and reference cross-attention outputs as Eq. 2 of Öztaş et al., but applies reference conditioning throughout the shape blocks to enable geometric as well as appearance stylization.

To directly investigate the suggested alternative, we conducted new layer-selection experiments on TRELLIS.2. Without BACKDROP, restricting reference injection from all shape blocks to the late half and final quarter reduces collapses from 418/432 to 226/432 and 30/432, respectively. However, this substantially weakens stylization: the final-quarter setting achieves CSD 0.27 versus 0.33 with BACKDROP, and geometric-style similarity 0.12 versus 0.25. BACKDROP preserves 430 of 432 outputs while allowing reference injection throughout the network, achieving stronger appearance and geometric stylization without sacrificing structural integrity. These results directly demonstrate why restricting injection layers does not replace our contribution.

W2, Q3: Applicability beyond TRELLIS.2.

Our cross-backbone experiments provide evidence that the importance of background conditioning extends beyond TRELLIS.2. On Hunyuan3D 2.1, deeper blocks also allocate substantial attention to background tokens (Fig. 8), and removing these tokens during direct generation degrades geometry, including structural failures (Fig. 12). This independently confirms that background conditioning influences the generated geometry across different 3D generators.

Our analysis suggests two conditions under which BACKDROP is particularly relevant: (1) the generator's deeper blocks allocate substantial attention to background tokens, and (2) the underlying stylization method preserves content geometry when using object references. These conditions explain the observed architectural differences. TRELLIS (v1) predominantly attends to global CLS and register tokens, which remain available for non-object references, and consequently does not exhibit the same collapse. On Hunyuan3D 2.1, the tested stylization implementation does not reliably preserve content geometry even with object references, preventing a direct demonstration of stylization repair in that setting.

These findings establish that the underlying background-attention phenomenon is not unique to TRELLIS.2. BACKDROP provides a simple, training-free intervention motivated by this broader mechanism, with its effectiveness demonstrated across three stylization methods and diverse reference types on TRELLIS.2. We have clarified this experimental scope in the revision by removing the overgeneralized wording "and generators" from Figure 1 and the TL;DR, and explicitly identifying TRELLIS.2 in the abstract.

W3: Technical contribution.

We respectfully disagree that the technical contribution is unjustified. The proposed method is derived from a causal investigation of geometric collapse, rather than an arbitrary adjustment of attention strength. Our experiments distinguish BACKDROP from the suggested alternatives:

  1. Zero-value tokens (Table 5) still require BACKDROP's blank-image keys to compete for attention. This validates our mechanism rather than replacing it. Although this variant achieves comparable structural integrity, it requires modifying the internal cross-attention operations, whereas BACKDROP works directly at the conditioning interface with a single line of code.
  2. Fixed attention scaling (Table 6) uses factors measured from BACKDROP's own attention distributions, yet still produces 28 collapsed outputs versus only 2 with BACKDROP.
  3. Global style-weight adjustment (Table 10) cannot achieve the same combination of strong stylization and geometric integrity under the evaluated settings.

Crucially, BACKDROP enables query-dependent attention redistribution through the generator's own softmax, naturally adapting across queries, heads, blocks, and sampling steps without additional learned parameters or per-instance tuning. Our contribution therefore includes both identifying and causally validating a previously overlooked conditioning failure and developing a simple, effective intervention that resolves it. The one-line implementation reflects this mechanistic insight, rather than a lack of technical contribution.

W4, Q4: Separate evaluation of geometric stylization.

Our existing evaluation already distinguishes three objectives: style similarity (CSD), structural integrity (SIS), and content-shape preservation (F@2%). To further isolate geometric-style transfer from texture and color, we conducted additional evaluations using untextured surface-normal renders across three stylization methods.

Using a VGG-16 patch-statistics score normalized relative to the unstylized base (0) and reference (1), BACKDROP achieves 0.25 for attention blending and 0.28 for StyleSculptor, improving by +0.097 [0.055, 0.139] and +0.127 [0.083, 0.171], respectively, over their unaugmented counterparts.

CSD computed on the same normal renders independently supports these findings. All three methods with BACKDROP outperform the unstylized base, while attention blending and StyleSculptor also significantly outperform their corresponding unaugmented versions, by +0.028 [0.005, 0.054] and +0.022 [0.0003, 0.048], respectively. For MorphAny3D, the improvement over its unaugmented version remains statistically unresolved, consistent with the original CSD evaluation.

These additional experiments provide quantitative evidence that BACKDROP supports geometric-style transfer beyond appearance similarity and structural preservation, directly addressing the concern about insufficient separation of evaluation objectives. A human study could not be completed within the discussion period due to the required ethics approval.

Q2: Token-count sensitivity.

The choice of 1,024 tokens was not tuned; it corresponds to the complete 32 × 32 patch grid of a blank image at the encoder's native 512-pixel resolution. We additionally evaluated blank-image encodings producing 64–2,304 tokens (full table in our response to Reviewer enmS).

The results reveal a clear relationship between attention competition and structural integrity. At α = 0.85, where 418/432 outputs collapse without BACKDROP, 576 tokens already reduce collapses to 14/432, while 1,024 tokens reduce them to only 2/432. Increasing the count to 2,304 does not improve reliability (14/432 collapses). The required count also depends on style strength: at α = 0.55, only 256 tokens already reduce collapses to 6/432, versus 0/432 with 1,024 tokens. Reference composition also matters: at 576 tokens, collapses occur in 6/96 artistic, 8/276 texture, and 0/60 scene pairs.

These results directly validate the attention-competition mechanism and demonstrate that the original, untuned 1,024-token choice provides the strongest structural reliability among the tested configurations, while maintaining strong style transfer.

Dear Reviewer dqbs, as the discussion period ends soon, we'd be grateful if you could check whether our response addresses your concerns. In particular, we added Öztaş et al., the layer-selection experiment you suggested, and a geometric-style evaluation. Thank you!

Reviewer_enmSOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

BACKDROP addresses geometric collapse when non-object images are used as style references in TRELLIS.2-based 3D stylization. The authors identify a dependence on background tokens and propose appending cached patch tokens from a blank image to reference conditioning, without training. Experiments on 432 content–reference pairs across three stylization methods, together with token-removal and replacement ablations, demonstrate substantial improvements in geometric integrity while retaining reference style.

Critical strengths
  1. Elegant and effective intervention. Appending cached blank-image tokens substantially improves geometric integrity with modest overhead and no training. The simplicity and compatibility with existing stylization methods make the contribution practically valuable.

  2. Compelling diagnostic evidence. Background-token removal reproduces geometric collapse, while replacement restores coherent geometry. Additional ablations, particularly the zero-value experiment, provide useful insight into the role of attention competition.

Critical weaknesses

The choice of appending 1,024 blank-image tokens is not sufficiently justified through sensitivity analysis. Since these tokens compete with the reference for attention, their number may affect the balance between geometric integrity and style transfer. A token-count ablation would strengthen confidence in the robustness of this design choice.

Questions for authors
  1. Design clarification: Why append exactly 1,024 blank-image tokens, given that the analyzed object references contain 277–664 background patches? Evidence about sensitivity to token count would help assess the robustness of this fixed choice and the risk of suppressing reference style.

  2. Comparative evaluation: Can the image-border alternatives achieve comparable CSD–SIS trade-offs after adjusting their style weights? Comparing only at the same nominal weight may not fully establish the advantage of separately encoded tokens.

  3. I am curious about the applicability of BACKDROP beyond TRELLIS.2. Given the different behaviors observed on TRELLIS and Hunyuan3D 2.1, could the authors discuss under what conditions they would expect the intervention to be beneficial?

3 comments

Thank you for the positive review and for recognizing both the practical effectiveness of BACKDROP and the strength of our mechanistic analysis. We address the questions with additional experiments and clarifications below. We have uploaded a revised PDF with changes marked in blue.

W1 & Q1: Choice and sensitivity of 1,024 tokens.

The number of tokens was not tuned. It follows directly from encoding a complete 512 × 512 blank image into a 32 × 32 grid of patch tokens. Importantly, the relevant quantity is not the exact number of tokens, but the attention share they attract (Eq. 3). Although object references contain 277–664 background patches, they also contain only 360–747 foreground patches, whereas non-object references have 1,024 content patches. Appending 1,024 blank-image tokens therefore provides a comparable background-to-content ratio and restores the final-block attention allocation close to that of object references (71% versus 74%; Fig. 3).

To directly address the sensitivity concern, we conducted a new experiment varying the blank-image resolution from 128 to 768 pixels, producing 64–2,304 tokens. All results use 432 non-object pairs at α = 0.85; the 0- and 1,024-token results are from the original paper.

Blank-image tokens0642565761,0242,304
Attention share (final shape block)–17%38%56%71%80%
Collapses / 43241840925114214
SIS ↑0.130.150.470.920.970.95
CSD ↑0.250.270.320.350.330.32

The results confirm the predicted role of attention competition and support our original choice. Too few tokens receive insufficient attention to prevent collapse. Increasing the count substantially improves structural integrity, while 1,024 tokens achieve the fewest collapses (2/432) and the highest SIS among the tested settings. Although 576 tokens yield slightly higher CSD, they also produce seven times as many collapses. Further increasing the count to 2,304 provides no additional benefit. Thus, the untuned choice of 1,024 tokens offers a strong balance between geometric integrity and style transfer, without requiring additional token-count tuning.

Q2: Black-border comparison.

We additionally evaluated both border constructions across style weights, reporting CSD (SIS in parentheses):

αBACKDROPShrunken-reference borderLarger-canvas border
0.550.21 (0.99)0.19 (1.00)0.19 (1.00)
0.700.28 (0.99)0.25 (0.99)0.26 (0.99)
0.850.33 (0.97)0.30 (0.99)0.31 (0.98)
1.000.35 (0.93)0.33 (0.98)0.34 (0.96)

BACKDROP achieves higher CSD at every matched style weight while maintaining high structural integrity. Increasing the border weights can produce comparable CSD–SIS trade-offs, confirming that image-space background padding is also an effective intervention, consistent with our diagnosis. The key distinction is that BACKDROP restores the missing background-token population without modifying or resizing the original reference image, preserving its encoded features and requiring only a single conditioning-level operation.

Q3: Applicability beyond TRELLIS.2.

Our experiments on Hunyuan3D 2.1 provide additional evidence that the importance of background conditioning extends beyond TRELLIS.2. Its deeper blocks allocate substantial attention to background tokens, exhibiting a pattern similar to TRELLIS.2 (Fig. 8). More importantly, removing background tokens during direct generation degrades geometry: among 12 object references, eight produce coarser shapes, three lose structure, and one collapses (Fig. 12). These results independently demonstrate that background tokens play an important role in shaping the generated geometry across different 3D generators.

Our cross-backbone analysis suggests two conditions under which BACKDROP is particularly beneficial: (1) the generator's deeper blocks allocate substantial attention to background tokens, and (2) the underlying stylization method preserves content geometry with object references. TRELLIS (v1) predominantly attends to global CLS and register tokens, which remain available for non-object references, and consequently does not exhibit the same collapse. For Hunyuan3D 2.1, the tested stylization implementation does not reliably preserve content geometry even with object references, preventing a direct demonstration of stylization repair in that setting.

Together, these findings support the broader relevance of background-token conditioning in 3D generation. BACKDROP provides a simple, training-free intervention motivated by this mechanism, with its effectiveness demonstrated across three stylization methods on TRELLIS.2. The additional cross-backbone experiments further establish that the underlying phenomenon is not unique to TRELLIS.2.

Reviewer_enmSOpenReview ↗

Response to Authors’ Rebuttal

Thank you to the authors for the detailed response and the additional experiments. The token-count analysis addresses my main concern well. I will maintain my original score.

Thank you for reading our response and for your feedback. We're glad the token-count analysis addressed your main concern.

Reviewer_EJFhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a one-line code method that appends black-image patch tokens to non-object image references as stylization conditioning. Authors find that by removing background conditioning, TRELLIS 2 can be stablilized, while restoring these black-image patch tokens would repair geometry.

Experiments are conducted on 3 stylization methods, comparing directly to 3D stylization and chained pipeline to stylize in 2D first then generate 3D. This paper shows clear experimental gains with the proposed method.

Critical strengths

Clear practical improvements with little implementation effort. The whole changes that this paper propose is to add black color background tokens together with reference non-object images, and proved in experiment that is works well.

Critical weaknesses

Lack of a root-cause investigation.

Despite presenting a simple and effective approach to address stylization 3D with non-object reference, this paper does not fully analyze why this approach works, and why other alternatives would not. The paper does show some experimental results on other alternative ways, to provide more observations in general, but it does not go further downwards into the mechanism and give more insights about the problem. This is more like an empirical study instead of a complete explanation.

From this perspective, a simple method that requires one-line code change seems like a trick to use, rather than an informative research and insightful study.

Questions for authors
  1. Does the Backdrop method retain its CSD advantage over black-border conditioning when the border baselines has their style weights adjusted to achieve structural integrity?
5 comments

Thank you for the review. We have uploaded a revised PDF with changes marked in blue.

W1: Why BACKDROP works.

We respectfully disagree with the characterization of BACKDROP as an empirical trick lacking a mechanistic explanation. This assessment overlooks the extensive causal investigations already presented in our paper. Identifying the root cause of geometric collapse is a central contribution of our work, and the one-line implementation follows directly from this analysis. Specifically, we provide four complementary investigations:

  1. Identifying the mechanism (Fig. 3). TRELLIS.2 allocates 74% of its final shape block's reference attention to background tokens. Non-object references lack this token population, causing excessive attention to concentrate on reference-content tokens.
  2. Establishing causality (Fig. 4, Table 4). Removing only background tokens from valid object references causes 143/144 outputs to collapse. Restoring them with independently encoded blank-image tokens repairs all 144 outputs, directly demonstrating the causal role of missing background conditioning.
  3. Isolating attention competition (Table 5). Zeroing the blank-image tokens' values still prevents collapse, demonstrating that attention competition, rather than their value contributions, is sufficient. Conversely, replacing them with identical mean tokens causes 294/432 outputs to collapse, showing that simply adding tokens is insufficient.
  4. Validating the explanation (Table 6). Directly attenuating reference attention without appending tokens recovers 90% of BACKDROP's SIS improvement. However, this fixed attenuation still leaves 28 collapsed outputs versus only 2 with BACKDROP.

These controlled interventions establish a causal explanation, not merely an empirical correlation: without background-token competition, reference-content tokens receive excessive attention and destabilize geometry. BACKDROP restores this competition while preserving the original reference features. The simplicity of the one-line solution is the result of this mechanistic understanding, not evidence of a lack of scientific depth.

Q1: Borders with adjusted weights.

Black borders also prevent geometric collapse, further supporting our diagnosis that the missing background-token population causes the failure. We additionally evaluate both border baselines across style weights, reporting CSD (SIS in parentheses):

αBACKDROPShrunken-reference borderLarger-canvas border
0.550.21 (0.99)0.19 (1.00)0.19 (1.00)
0.700.28 (0.99)0.25 (0.99)0.26 (0.99)
0.850.33 (0.97)0.30 (0.99)0.31 (0.98)
1.000.35 (0.93)0.33 (0.98)0.34 (0.96)

BACKDROP achieves higher CSD at every matched style weight while maintaining high structural integrity. Increasing the border weights narrows the gap: at α = 1.0, the shrunken-reference border reaches the same rounded CSD as BACKDROP at α = 0.85, with comparable SIS. This confirms that image-space background padding is also an effective repair, consistent with our identified mechanism. The distinction is that borders modify the reference image before encoding, whereas BACKDROP preserves its original encoded features and restores the background-token population independently. Thus, BACKDROP provides a simple conditioning-level intervention with stronger style transfer at matched weights, without altering or resizing the reference image.

Dear Reviewer EJFh, as the discussion period ends soon, we'd be grateful if you could check whether our response addresses your concerns. In particular, we outlined the causal experiments behind the collapse and added the border comparison across style weights you asked about. Thank you!

Thank you for providing further clarifications about the positioning and depth of this research work. I would agree these further analysis revealed some characteristics about (mostly) TRELLIS.2, and that the solution is based on well understanding of it.

Could you additionally use some very brief sentences to summarize the insights and how this mechanism understanding could potentially transfer to other research for the community? It would help me understand whether these observed mechanisms are something unique on one or two particular models (e.g. TRELLIS.2), or something that has broader meaning?

Comments on the paper

1 comment

Dear Reviewers, as the discussion deadline approaches, we’d like to note that we have responded to all comments and believe your concerns have been addressed. We’re happy to clarify anything further. Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.