acceptodds
Under review as a conference paper at ICLR 2027

SPIN: Scale-aware Position-unbound Identity Injection for Consistent Visual Autoregressive Generation

Abstract

Visual autoregressive (VAR) models generate images from coarse to fine over a schedule of scales, yet existing methods for subject-consistent generation improve identity consistency across prompts at the expense of pose and layout diversity. Methods that expose source activations to the target also expose their positional information. In VAR models, this locks the target's composition at the coarse scales where layout is determined, tying gains in identity consistency to losses in diversity. We present SPIN, a training-free method that relaxes this trade-off. Its premise is that the source-target state gap is a single object that a transformer block admits through exactly three interfaces, each capable of carrying identity while shedding positional information. State Read Transfer moves this gap into what a layer reads. RoPE-Rewound Subject Attention allows the target to attend to source keys whose rotary phase is rewound on the band that carries position. Pre-Render Write Transfer transfers the per-layer increment, in which everything accumulated before the layer cancels out. Each transfer is placed according to scale- and layer-dependent positional effects. On a 191-group benchmark, SPIN reduces DreamSim on Infinity-2B from 0.235 to 0.182 while retaining 89% of the backbone's pose diversity. It lies above the identity-diversity trend of twelve prior methods, and the same placement recipe improves identity across five tested next-scale backbones.

Primary area:
generative models
Keywords:
visual autoregressive models, Consistent character generation, Subject-consistent text-to-image generation, Residual stream

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_zx8zOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes SPIN, a training-free method for preserving subject identity in next-scale VAR models. The starting observation is that exposing source activations to the target carries positional information along with identity, so the layout gets copied at coarse scales. The method puts a module at each of three different parts of a Transformer block. SRT pulls the attention input toward the source, R$^2$SA adds source K/V with rewound RoPE phase to the target softmax, and PWT mixes the difference of per-layer increments instead of the accumulated state. On Infinity-2B, the method lowers DreamSim while keeping about 90% of the backbone's pose diversity, and the same recipe transfers to five backbones. My concerns are that, apart from the RoPE rewind, the analysis behind each design choice is thin, that the result may be just another point on the existing trade-off curve rather than an improvement of it, and that the source of the Tab. 1 baseline numbers is unclear.

Critical strengths
  1. The problem framing is reasonable since the paper decomposes diversity loss, which identity-preservation work tends to ignore, along the structural axes of VAR (scale, layer, rotary band). RoPE rewind is the one part where the design actually follows from measurement, as App. G shows that same-position preference concentrates in the high-frequency band and the band ablation agrees.

  2. Practicality is compelling, as the method is training-free, one placement rule transfers to five next-scale backbones, and prompt fidelity is mostly preserved.

Critical weaknesses
  1. The central claim of the paper, that the trade-off is improved, is not established. Rather, it looks like the method only moves along that trade-off. SPIN has more diversity than Infinite Story but less identity, and turning R$^2$SA on from scale 2 moves it to almost exactly the point of Infinite Story. Relative to its backbone, CoDi retains pose diversity better at a similar identity gain (about 98% vs. 89%). The "above the trend" claim in Fig. 3a mixes absolute numbers across backbones, so it is confounded by each backbone's diversity ceiling. The SDXL-family pose values behind it are also 20 to 30% higher than the CoDi paper's own reproduction, so it is not even clear that the trend is on a comparable metric. So one can still argue for using Infinite Story when identity matters and CoDi when diversity matters, and the paper does not answer this. Since that is the reason for this method to exist, this is the decisive issue.

  2. The design comes from tuning rather than analysis, and the ablation does not check it. SRT has only a verbal rationale with no measurement. The claim that dzdz carries less positional information than hh is measured on the source dzdz only, not on what is actually injected, and there is no end-metric comparison of mixing hh instead of dzdz. Tab. 2(a) follows a single cumulative order (PWT, then R$^2$SA, then all together including SRT), so the individual contribution and necessity of each module cannot be read, and since every step trades diversity for identity, it is impossible to tell a better trade-off from a stronger setting. The placement logic is also confusing. R$^2$SA is withheld at scales 1 and 2 because they are layout-committing, yet PWT and SRT run on the full grid at those same scales, and the gate that supposedly opens after composition is settled is effectively open from scale 1.

Questions for authors
  1. Were the SDXL-family baselines in Tab. 1 reproduced or quoted? The pose values differ from the CoDi paper by 20 to 30% (e.g., vanilla SDXL 0.0967 here vs. 0.0772 there, and similarly for the other three SDXL-family rows), and only the 1Prompt1Story pose value matches. Which keypoint estimator and confidence threshold were used for pose diversity, how are animal and object subjects handled, and, if reproduced, what resolution and seeds were used? App. A only covers the Tab. 3 re-implementation.

  2. Can you report each method relative to its backbone, and compare the Pareto curves from strength sweeps of SPIN and Infinite Story?

  3. Can you add each module alone, leave-one-out ablations from the final configuration, and a variant with PWT turned off at the layout-committing scales? The current table follows one cumulative order, and the representation-level complementarity in App. D does not substitute for it.

  4. What happens when hh is mixed instead of dzdz at the same layers and strength?

  5. Were the constants tuned on a separate set from the one used for evaluation?

9 comments
AuthorsOpenReview ↗

Response to zx8z : W1 & Q2

We thank the reviewer for raising the central question of whether SPIN improves the identity–diversity trade-off rather than selecting another point on the same curve.

W1 / Q2. Identity–diversity trade-off

We agree that Fig. 3a alone cannot establish a better trade-off from absolute diversity values across backbones, which may have different diversity ceilings. We therefore swept SPIN and Infinite Story on the same Infinity-2B backbone. All other settings and module placements were fixed: SPIN varied only λ\lambda, jointly scaling SRT/PWT at their original ratio (λ=1\lambda=1 is the default), and Infinite Story varied its strength ww.

MethodStrengthDreamSim ↓CLIP-I ↑Pose ↑Centroid ↑
SPINλ=0.5\lambda=0.50.19910.90680.07790.0997
SPINλ=1\lambda=10.18210.91420.07500.0945
SPINλ=2\lambda=20.16590.91880.06260.0817
SPINλ=3\lambda=30.14150.92830.05590.0713
Infinite Storyw=0.25w=0.250.19550.90380.07060.0829
Infinite Storyw=0.5w=0.50.17800.91220.06690.0792
Infinite Storyw=0.7w=0.70.16750.91580.05970.0736
Infinite Storyw=0.85w=0.850.15960.91900.05970.0662

At comparable identity, SPIN (λ=2\lambda=2) achieves lower DreamSim than Infinite Story (w=0.7w=0.7) (0.1659 vs. 0.1675), with 4.9% higher pose (0.0626 vs. 0.0597) and 11.0% higher centroid diversity (0.0817 vs. 0.0736). Near the defaults, SPIN (λ=1\lambda=1) has slightly higher DreamSim than Infinite Story (w=0.5w=0.5) (0.1821 vs. 0.1780), but 12.1% higher pose diversity (0.0750 vs. 0.0669).

We also compared the methods at matched DreamSim, using linear interpolation of the measured sweep points:

DreamSimMethodStrengthPose ↑Centroid ↑
0.18SPINλ≈1.130\lambda\approx1.1300.07340.0928
0.18Infinite Storyw≈0.471w\approx0.4710.06730.0796
0.17SPINλ≈1.747\lambda\approx1.7470.06570.0849
0.17Infinite Storyw≈0.652w\approx0.6520.06140.0749

At DreamSim = 0.18, SPIN retains 9.0% higher pose and 16.6% higher centroid diversity; at 0.17, the gains are 7.0% and 13.4%, respectively. Relative to the same vanilla Infinity-2B (DreamSim 0.2348; pose 0.0842), SPIN (λ=2\lambda=2) improves DreamSim by 29.3% and retains 74.3% of pose diversity; Infinite Story (w=0.7w=0.7) improves DreamSim by 28.7% and retains 70.9%.

These sweeps support a more favorable trade-off over the evaluated strength range, rather than only a different operating point. In particular, the measured λ=2\lambda=2 versus w=0.7w=0.7 comparison improves DreamSim and both diversity metrics simultaneously.

We also acknowledge the reviewer's observation that activating R²SA from scale 2 moves the variant toward an Infinite Story-like operating point. This suggests that sharing source K/V too early can reduce the diversity advantage. We distinguish this variant from the default, which starts R²SA at scale 3.

CoDi retains about 98% of pose diversity relative to its own backbone, versus about 89% for default SPIN; however, the identity gains differ. When ported to the same Infinity-2B, CoDi improves DreamSim by about 2.8%, versus 22.4% for SPIN. Thus, pose retention alone does not establish a better trade-off at matched identity. We will clarify Fig. 3a's cross-backbone limitation and report backbone-relative metrics alongside same-backbone strength-sweep curves.

AuthorsOpenReview ↗

Response to zx8z : Q1, W2 & Q3/Q4

Q1. Source and evaluation protocol for Table 1

The SDXL-family results in Table 1 were re-evaluated on generated images using our own pipeline, not copied from the CoDi paper. CoDi's released code does not include its pose-diversity evaluation, so we reimplemented the protocol described in the paper and applied it uniformly to all methods:

  • Keypoints: RT-DETR (detection threshold 0.3) and ViTPose-base (17 COCO keypoints, confidence threshold 0.7).
  • Pose distance: Procrustes-aligned distance for image pairs with at least 5 shared valid keypoints.
  • Subjects: 74 human/humanoid groups; animals and non-humanoid objects excluded.
  • Generation: 1024 × 1024 resolution, seed 100.

The difference from CoDi's reported values (e.g., vanilla SDXL 0.0967 vs. 0.0772) may arise from differences in keypoint detection, evaluated groups, or distance implementation. As the original evaluation code is unavailable, we cannot identify the exact cause. We will label Table 1 "Re-evaluated Using Our Unified Protocol" to distinguish our measurements from the original paper's results.

W2-1 / W2-3 / Q3. SRT and full-factorial ablation

To assess each module beyond the original sequential ablation, we ran all 23=82^3=8 combinations, including individual modules and leave-one-out variants.

ConfigurationCLIP-I ↑DreamSim ↓Pose ↑Centroid ↑
Backbone0.89450.23480.08420.1052
R²SA0.90000.21580.08160.1036
PWT0.90230.21130.08240.0944
SRT0.89920.21370.08130.1041
R²SA + PWT0.90700.19790.07910.0942
R²SA + SRT0.90370.20640.07670.1021
SRT + PWT0.90560.19860.07320.0919
Full SPIN0.91420.18210.07500.0945

SRT alone improves DreamSim from 0.2348 to 0.2137, retaining 96.6% of baseline pose diversity. Adding SRT to R²SA+PWT further improves DreamSim from 0.1979 to 0.1821. Removing any one module from Full SPIN worsens DreamSim: 0.1979 without SRT, 0.1986 without R²SA, and 0.2064 without PWT, versus 0.1821 for Full SPIN. This supports each module's additional contribution to identity preservation when the other two are present.

The modules act at different points: SRT modifies attention inputs, R²SA supplies source K/V with reduced rotary positional bias, and PWT transfers layer increments instead of accumulated residual states. Because identity gains can accompany diversity losses, we separately evaluated the strength-sweep trade-off in W1/Q2.

W2-2 / Q4. Increment versus state transfer

We clarify that the original positional-share (η2\eta^2) measurements were made on source representations, not directly on injected source–target differences. Positional share measures the fraction of feature variation explained by grid position. On Infinity-2B, token-weighted pooled η2\eta^2 is 0.092 for layer increments versus 0.155 for residual states, a 40.6% reduction.

To test the effect of the actual transferred representation on generation, we compared write-gap (source–target layer-increment difference) and state-gap (source–target accumulated-state difference) injection, controlling the layers, scales, and injection norm.

MetricWrite gapState gap
DreamSim improvement10.0%21.5%
Pose-diversity loss7.0%18.2%
Centroid-diversity loss7.5%22.9%

State-gap transfer gives a larger identity improvement, but 2.6× greater pose-diversity loss (18.2% vs. 7.0%) and 3.1× greater centroid-diversity loss (22.9% vs. 7.5%). Thus, the choice of transferred difference affects the generated identity–diversity balance. We do not equate the measured source-representation η2\eta^2 with the η2\eta^2 of the injected difference; these are separate analyses.

PWT alone improves DreamSim from 0.2348 to 0.2113. It also overlaps with R²SA at scales 3–5 (R²SA: 3–12; PWT: 1–5), rather than operating only where R²SA is inactive. Adding PWT to R²SA improves DreamSim from 0.2158 to 0.1979 (8.3%), supporting its distinct role in transferring a less position-dependent layer increment.

AuthorsOpenReview ↗

Response to zx8z : W2-4, W3 & Q5

W2-4 / Q5. Hyperparameter selection and possible benchmark overfitting

We acknowledge the concern that hyperparameter choices may depend too strongly on the evaluation benchmark. We further analyzed whether the main module placements can be explained by backbone structure and internal representations rather than benchmark scores alone. Specifically: (i) the PWT layer band is selected using render NCC and positional share (η2\eta^2); (ii) R²SA starts at the first scale where the token count exceeds the source-bank size (ns>Bn_s>B); and (iii) PWT scales follow a common rule covering the small-token region (ns≤144n_s\leq144) and up to two scales after R²SA starts.

BackboneR²SA startPWT scalesAnalysis layers
Infinity-2B31–54–10
Switti51–712–15
HART51–73–8
BackboneMean η2\eta^2 (inc./state)Pooled η2\eta^2 (inc./state)
Infinity-2B0.235 / 0.3040.092 / 0.155
Switti0.136 / 0.2860.086 / 0.177
HART0.115 / 0.1860.110 / 0.181

The analysis bands above are distinct from the Infinity-2B PWT injection layers (4–9). At the selected analysis bands, increments have lower positional share than states in all three backbones. For Infinity-2B, pooled η2\eta^2 is 40.6% lower for increments. The directly adjusted backbone-specific setting was the R²SA start scale; PWT ranges were determined by the same structural rule, without separate backbone-specific performance searches. In Table 6, identity consistency improved on all five backbones without additional backbone-specific performance tuning.

With unchanged settings across seeds 100, 101, 102 on Infinity-2B, DreamSim ranged from 0.1821–0.1857, pose diversity from 0.0713–0.0750, and CLIP-I from 0.9140–0.9142. In a separate 10-style shift evaluation, DreamSim improved from 0.3453 to 0.2890 (16.3%). These results support the structural explanation of the main placements and show improvements across the tested seeds and styles. They do not, however, replace an evaluation on independent tuning/test prompt sets.

W3 / Q3. Early-scale PWT/SRT placement and adaptive gating

R²SA and PWT/SRT use different transfer paths at early layout scales. R²SA directly adds source K/V to attention, which can exert greater influence when few target tokens are present and may copy source layout. In contrast, PWT transfers a less position-dependent layer increment rather than the accumulated residual state; SRT modifies the attention input without directly adding source state to the residual stream. Thus, using PWT/SRT at early scales while delaying R²SA reflects the differences in what is transferred.

In a separate early-scale ablation, removing PWT at scales 1–2 worsened DreamSim from 0.1857 to 0.1993, indicating an identity contribution from early-scale PWT. The 0.1857 value is the reference for that ablation; the main Full SPIN result remains 0.1821. The norm-controlled write-gap/state-gap comparison in W2-2/Q4 further examines the effect of the transferred representation: state-gap injection caused 2.6× greater pose and 3.1× greater centroid-diversity loss.

The adaptive gate is not a hard switch that blocks injection until composition is complete. It continuously modulates injection strength according to target-specific similarity, including at early scales. At scale 1, the gate is fully closed for 15% of targets, and the bottom 25% have gate values of 0.52 or lower. Its role is therefore target-dependent modulation, not an identical on/off schedule for every target.

To assess this choice, we compared adaptive and fixed gates at an equal 1.519% injection dose; the adaptive results were interpolated from the strength sweep.

GateCLIP-I ↑DreamSim ↓CLIP-T ↑VQA ↑Pose ↑Centroid ↑
Adaptive (interp.)0.91400.18190.30380.82220.07110.0942
Fixed0.91450.17740.30360.82080.07050.0900

At the same dose, the fixed gate has lower DreamSim (0.1774 vs. 0.1819), whereas the adaptive gate retains higher pose (0.0711 vs. 0.0705) and centroid diversity (0.0942 vs. 0.0900; 4.7% higher). We do not claim that the adaptive gate is better on all metrics. Fixed gating is a valid simplification; adaptive gating is a design choice that emphasizes diversity preservation. We will add the early-scale PWT ablation, gate distributions, and matched-dose comparison, and revise wording suggesting that early-scale layout copying is completely prevented.

Reviewer_ej1LOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SPIN, a training-free framework for subject-consistent visual autoregressive generation. It addresses the trade-off between identity consistency and pose diversity by reducing the transfer of source positional information.This is achieved by three feature transfer mechanisms operating on attention inputs, source keys and values, and layer-wise residual increments. By applying these mechanisms at carefully selected scales and transformer layers, SPIN improves subject identity consistency while preserving more pose diversity than existing approaches.

Critical strengths
  • Identity consistency is an important problem in image generation with substantial practical value, particularly for applications requiring consistent subjects across different scenes and poses.
  • The proposed feature injection mechanisms are carefully designed to balance identity consistency and pose diversity. The scale- and layer-aware strategies are supported by insightful analyses of positional information in transformer features.
  • The paper provides extensive experimental evaluations, ablation studies, and visualizations that effectively support the proposed hypotheses and demonstrate the effectiveness of the method.
Critical weaknesses
  • The paper primarily investigates the disentanglement of identity from pose and spatial layout under different actions and scenes. However, identity preservation under changes in visual style is also an important challenge in consistent image generation. It remains unclear whether the proposed method can maintain identity consistency when the target images involve substantial style variations.
  • The proposed method is specifically designed for scale-wise visual autoregressive models, leveraging their coarse-to-fine generation process. It is unclear whether the underlying principles and feature injection strategies can be generalized to other autoregressive architectures, such as spatial-wise autoregressive models.
  • Although the paper provides quantitative analyses of the proposed feature injection and positional encoding modifications, their respective effects on the generated images remain insufficiently illustrated. Additional visualizations comparing attention maps and corresponding generation results, particularly before and after feature injection and RoPE modification, would help clarify how these operations influence identity preservation and spatial layout.
  • The computational overhead of the proposed method is relatively substantial. In particular, the inference time of SPIN on Infinity-2B already exceeds that of the vanilla Infinity-8B backbone. Considering this additional computational cost, the trade-off between the achieved performance improvements and inference efficiency is not entirely convincing.
Questions for authors

Please refer to the weaknesses part.

4 comments
AuthorsOpenReview ↗

Response to ej1L : W1 & W2

We thank the reviewer for asking whether identity is preserved under style changes and whether SPIN transfers beyond scale-wise VAR. We conducted additional evaluations for both questions.

W1. Identity consistency under style shifts

We kept the source fixed and changed only the target prompt's style across 10 styles, including photo, watercolor, anime, and pixel art, while keeping subject attributes, action, and background conditions unchanged. Vanilla DreamSim increased from 0.2277 without the style change to 0.3453 with style changes. Under the latter condition:

MetricVanillaSPIN
DreamSim ↓0.34530.2890
CLIP-I ↑0.83980.8652
Pose diversity ↑0.08430.0740

SPIN reduced DreamSim by 16.3% (0.3453 → 0.2890), improved CLIP-I (0.8398 → 0.8652), and retained 87.8% of vanilla pose diversity (0.0740 vs. 0.0843). These results support improved source similarity under varied target styles, in addition to changes in action and scene. Image similarity alone cannot fully separate identity preservation from copying source style. To examine target-style adherence more directly, we will add side-by-side source, vanilla, and SPIN generations for photo, watercolor, anime, and pixel-art prompts.

W2. Transfer to spatial-wise next-token AR

We applied the full SRT + R²SA + PWT configuration to LlamaGen-512, a spatial-wise next-token AR model. DreamSim improved by 5.3% (0.4006 → 0.3795), while retaining 95.8% of pose diversity and 99.4% of centroid diversity relative to vanilla. Because LlamaGen has no explicit coarse-to-fine scales or layout-commitment boundary, we retained the three modules' functions while mapping scale-dependent decisions to raster token steps.

SRT and PWT. SRT changes the attention input toward a corresponding source representation without directly adding to the residual stream. PWT transfers the source–target difference in the layer write gap (block output minus input) at the block output:

hl←hl+awτ(Δzsrc(j)−Δztgt).h_l \leftarrow h_l + aw\tau\bigl(\Delta z_{\mathrm{src}}^{(j)}-\Delta z_{\mathrm{tgt}}\bigr).

We used layers 5–10 of 36, mapped by relative depth from Infinity-2B's layers 4–9 of 32, with a=0.2a=0.2 and w=9/6=1.5w=9/6=1.5. Infinity reduces injection strength across scales, whereas an analogous decay across raster steps would disproportionately affect the upper image region. We therefore used a constant base strength and a token-wise gate τ\tau. PWT is active at all 1,024 raster token steps, modulated by τ\tau.

R²SA. We append a 32-token source K/V bank to self-attention: 12 (37.5%) tokens from the source subject region and 20 sampled spatially across the image. Bank use is skipped with 30% probability at each layer/step. With 2D axial RoPE and head dimension 64, we rewind the high-frequency rotary pairs 0–7 and 16–23 in source bank keys while retaining lower-frequency pairs. Unlike Infinity's scale-wise rebuilt bank, the raster bank is constructed once and reused. R²SA starts at cell 32, adapting Infinity's bank-size-based start condition to token generation.

Token-wise correspondence and gate. The adaptations are:

Infinity (scale-wise)LlamaGen-512 (raster-wise)
Strength decays across scalesConstant base strength
PWT at scales 1–5All tokens, gated by τ\tau
R²SA from scale 3From cell 32
Scale-wise NCC gateNot used
Text-attention subject selectionSource Grounded-SAM2 mask and token similarity
Same-grid-cell matchingContent-based matching

At each target token and layer, we choose source subject token jj by the highest cosine similarity of attention inputs. The gate τ\tau normalizes this maximum similarity using the layer-wise source distribution, mapping the background median to 0 and subject median to 1. For CFG, SRT/PWT add the same amount to conditional and unconditional branches; each branch attends to its corresponding source bank. Layer band and base strength were not tuned on LlamaGen metrics. We adjusted only the gate threshold: the initial background-q90/subject-q50 threshold made τ=0\tau=0 in every layer (disabling SRT/PWT), so we changed it to background-q50/subject-q50, without metric-based tuning.

The improvement on LlamaGen (5.3%) is smaller than on Infinity-2B (22.4%), potentially because LlamaGen lacks the coarse-to-fine scale structure used for SPIN's placement. We interpret this as initial evidence of identity improvement in a different AR architecture, not as establishing identical gains across AR structures.

AuthorsOpenReview ↗

Response to ej1L : W3 & W4

W3. Visualizing injection and RoPE rewind

We agree that the mechanism analyses should be connected more directly to generated images. Existing Figs. 7–9 show that RoPE rewind reduces same-position source attention bias from 12.1× to 3.1× (a 74.4% reduction), while Fig. 6 visualizes how PWT transfers a less position-dependent layer increment rather than the accumulated residual state. These analyses focus on internal attention and representations, rather than a direct side-by-side connection with final image identity and layout.

We will therefore add visual comparisons using the same prompt and seed: (1) generated images as SRT, R²SA, and PWT are enabled sequentially; (2) source attention maps with versus without RoPE rewind, alongside their generated images; and (3) SRT/PWT injection locations and strength maps alongside their outputs. For the rewind comparison, we will hold the other modules, layer, attention head, prompt, and seed fixed. These visualizations are intended to make the relation between identity preservation and source-layout copying more direct.

W4. Inference cost and practical trade-offs

We acknowledge SPIN's additional inference cost. Across the two evaluated backbones, the absolute increase was similar (about 1.3 s), while the relative overhead decreased from 2.02× on Infinity-2B to 1.51× on Infinity-8B.

BackboneVanillaSPINOverhead
Infinity-2B1.32 s2.66 s2.02×
Infinity-8B2.54 s3.84 s1.51×

At similar inference times, SPIN-2B (2.66 s) achieves lower DreamSim than vanilla Infinity-8B (2.54 s): 0.1821 vs. 0.2299, a 20.8% reduction, while using approximately 55% less GPU memory.

A matched-identity strength-sweep comparison shows that at DreamSim = 0.18, SPIN retains higher pose diversity than StoryDiffusion (0.0734 vs. 0.0661, about 11.0% higher):

Method (interpolated)DreamSim ↓Pose diversity ↑
SPIN0.18000.0734
StoryDiffusion-style port0.18000.0661

The runtime and matched-DreamSim results are separate comparisons. Thus, although SPIN increases computation over the same vanilla backbone, these results show advantages in identity consistency and memory use at approximately the runtime of a larger vanilla backbone, and in inference efficiency relative to the evaluated consistency baseline.

AuthorsOpenReview ↗

Pointers to the Revised Manuscript: Style-shift evaluation, LlamaGen adaptation, and visual ablations

Thank you again for your thoughtful review. As the discussion period is nearing its end, we would like to gently follow up on our response above. For your convenience, here is a brief summary of where each of the four weaknesses you raised is addressed in the revised paper:

  • Identity under style shifts. App. K and Table 14 (p. 27) report the ten-style evaluation with the source and non-style prompt attributes fixed. Fig. 11 (p. 28) shows the source, vanilla Infinity-2B, and SPIN side by side, including photo, watercolor, anime, and pixel-art targets.

  • Adaptation beyond next-scale models. App. I (pp. 25–26) reports the LlamaGen-512 results. Table 13 (p. 26) summarizes the changes to placement, token correspondence, and gating for raster-order generation.

  • Controlled visual comparisons. Fig. 10 (p. 24) contains the three requested views. Panel (a) adds SRT, R$^2$SA, and PWT in sequence. Panel (b) pairs attention maps and outputs with and without RoPE rewind, using the same source, target prompt, seed, layer, and head. Panel (c) overlays the per-token SRT and PWT update strengths on the generated images. Fig. 9 (p. 23) includes additional rewind examples. Fig. 2 (p. 6) and App. O (Figs. 13–16, pp. 32–35) show further qualitative comparisons.

  • Inference cost and practical trade-offs. Sec. 4 (p. 10) and App. M (Table 16, p. 30) give runtime and peak memory, including the comparison between SPIN-2B and vanilla Infinity-8B at similar runtimes. The matched-DreamSim diversity comparison with the StoryDiffusion port is in Table 3(b) (p. 8).

We would greatly appreciate it if you could let us know whether these additions address your concerns. If anything remains unclear, we would be glad to provide further clarification in the remaining time.

Dear authors,

I would like to thank for the rebuttal. Most of my concerns are addressed. Therefore I would like to raise my score.

The reviewer

Reviewer_SiPwOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper rests on two observations, both of which are simple and well supported:

(1) Positional information has an identifiable carrier. In 2D axial RoPE, the fine “same-grid-cell” preference in attention lives in the high-frequency rotary pairs. Removing the source-side phase on that band alone drops the same-position preference from 12.1× to 3.1× while leaving identity, which is spread across all pairs, intact (App. G).

(2) Layout commitment has an architectural boundary. In next-scale VAR models, placement, size, and coarse pose are fixed at the first few scales (1×1 to 4×4). “When to inject” is therefore a scale index, not a tuned timestep. → Implies a method with ~3 settings: shared source K/V attention, band-selective rewind, start scale.

However, I think the method as built by three injection modules (SRT, R²SA, PWT) doing the same thing at three points in the block with ~12 tuned constants plus a gate that collapses to a fixed schedule. Layer band (4–9) found by intersecting three measurements; all other placements picked by ablation on the same benchmark, single seed. The second claim (increments carry less position than states) is weak on the 2B model where results are reported; PWT mainly fills the scales where R²SA is off.

Critical strengths

1: Explicit identity–diversity trade-off, measured on both axes for all 12 baselines.

2: First to use VAR’s scale boundary for consistency: no attention sharing at layout scales.

3: Band-selective RoPE rewind: cheap, verified (same-position bias 12× → 3×)

4: Thorough mechanism analysis (η², render NCC, injection gain) and same-backbone baselines, paired tests, five backbones, user study.

Critical weaknesses

The paper’s two principles would imply a method with three settings: shared attention over source K/V, a band-selective rewind, and a start scale. The deployed method is far from that.

My main concern is practical: is this method actually usable?In short, the method is too complex and too engineered: three modules, each with its own layers, scales, schedules, and gates, and a dozen or so numbers tuned by hand on Infinity-2B. I could accept that complexity if the recipe transferred to other AR models rigorously and without re-tuning. The paper claims it does, but the evidence is thin: the same prompts, a single seed, and only one parameter changed per backbone. If the authors can show the full recipe moving to new backbones cleanly, with a clear rule for every setting, the complexity becomes a feature rather than a cost.

Questions for authors

1: For Tab. 3: at which scales and layers was each ported method active, and were any of its hyper-parameters tuned on Infinity? Can you give a strength sweep for the ported StoryDiffusion and for Infinite Story so that both appear as curves in Fig. 3b?

5 comments
AuthorsOpenReview ↗

Response to SiPw : W1 & W3

We thank the reviewer for questioning whether SPIN's complexity is justified.

Our three modules address distinct aspects of source information transfer:

  • SRT: Controls the information entering attention.
  • R²SA: Preserves identity through source K/V without imposing source rotary positions.
  • PWT: Transfers Layer Increments rather than accumulated Residual States.

Although all three transfer source information, they operate through different pathways: Attention Input, Position-aware Attention, and Layer Increments. We therefore conducted additional experiments to verify their complementary contributions beyond cumulative ablations.

W1 — Complementarity Among Modules

We evaluated all 2³ = 8 module combinations.

ConfigurationCLIP-I ↑DreamSim ↓Pose ↑Centroid ↑
Backbone.8945.2348.0842.1052
R²SA.9000.2158.0816.1036
PWT.9023.2113.0824.0944
SRT.8992.2137.0813.1041
R²SA + PWT.9070.1979.0791.0942
R²SA + SRT.9037.2064.0767.1021
SRT + PWT.9056.1986.0732.0919
Full SPIN.9142.1821.0750.0945

Compared with the reviewer-suggested R²SA-only configuration, Full SPIN improves DreamSim by 15.6% (.2158 → .1821).

Adding the third module to any two-module combination consistently improves DreamSim:

  • R²SA + PWT: .1979 → .1821
  • R²SA + SRT: .2064 → .1821
  • SRT + PWT: .1986 → .1821

No two-module combination matches Full SPIN's identity preservation, supporting the complementary contributions of all three modules rather than redundant source injection.

W1 — Adaptive Gate vs. Fixed Gate

We compared Adaptive and Fixed Gates at the same Injection Dose (1.519%), interpolating the Adaptive results from the Strength-Sweep curve.

GateCLIP-I ↑DreamSim ↓CLIP-T ↑VQA ↑Pose ↑Centroid ↑
Adaptive (Interpolated).9140.1819.3038.8222.0711.0942
Fixed.9145.1774.3036.8208.0705.0900

At the same Injection Dose, the Adaptive Gate maintains higher Pose Diversity (.0711 vs. .0705) and Centroid Diversity (.0942 vs. .0900) than the Fixed Gate, with approximately 4.7% higher Centroid Diversity. In contrast, the Fixed Gate achieves a lower DreamSim score (.1774 vs. .1819).

These results demonstrate that the injection strategy can influence the Identity–Diversity trade-off even when the total Injection Dose is held constant. The Adaptive Gate achieves competitive identity preservation while maintaining relatively higher diversity.

W3 — Why Layer Increments Instead of Residual States?

To validate the lower positional dependence of Layer Increments, we measured Positional Share (η²), the proportion of feature variance explained by grid positions.

Since the high η² values at the 2×2 Scale (.910/.960) disproportionately affected the previous scale-wise average, we computed token-count-weighted Pooled η².

Infinity-2B results:

RepresentationPooled η²
Layer Increment.092
Residual State.155

Layer Increments exhibit 40.6% lower positional dependence.

We further compared both representations while keeping Layer, Scale, and Injection Norm identical. Replacing Layer Increments with Residual States increased the reduction in Centroid Diversity by 3.1× and Pose Diversity by 2.6×. Thus, representation choice matters independently of injection location and magnitude.

PWT is also not merely a substitute for R²SA at early Scales, for two reasons.

First, their operating Scales overlap. In Infinity-2B, R²SA operates at Scales 3–12 and PWT at Scales 1–5, meaning both operate simultaneously at Scales 3–5.

Second, PWT contributes even alongside R²SA. Adding PWT to R²SA improves DreamSim from .2158 to .1979 (8.3%). Independently, PWT improves DreamSim from .2348 to .2113, with Pose Diversity changing only from .0842 to .0824.

These results support PWT as a distinct mechanism that transfers representations with lower positional dependence, rather than merely compensating for inactive R²SA Scales.

AuthorsOpenReview ↗

Response to SiPw : W2, W4 & Q1

W2, W4, & Q1: Placement rules, backbone transfer, and baseline fairness

W2/W4: Placement and transfer. To address tuning and portability, we analyzed structural rules for the principal placements: (i) select the PWT layer band using render NCC and positional share (η2\eta^2); (ii) start R$^2$SA at the first scale whose token count exceeds the source-bank size (ns>Bn_s>B); and (iii) apply PWT over the small-token region (ns≤144n_s\leq144) and up to two scales after R$^2$SA begins. Applying these same criteria gives:

BackboneR$^2$SA startPWT scalesAnalysis layersη2\eta^2 pooled (inc./state)
Infinity-2B31–54–100.092/0.155
Switti51–712–150.086/0.177
HART51–73–80.110/0.181

The analysis layers in this table are distinct from the actual PWT injection layers (4–9 on Infinity-2B). For completeness, the corresponding scale-averaged η2\eta^2 values (increment/state) are 0.235/0.304, 0.136/0.286, and 0.115/0.186, respectively. In every model, increments have lower positional share. The directly adjusted backbone-specific parameter was the R$^2$SA start scale; PWT ranges follow the same structural rule without a separate backbone-specific performance search. As reported in Table 6, identity consistency improved on all five tested backbones without further backbone-specific performance tuning.

We also evaluated Infinity-2B at seeds 100, 101, and 102 with unchanged settings. DreamSim ranged from 0.1821–0.1857, pose diversity from 0.0713–0.0750, and CLIP-I from 0.9140–0.9142. This supports seed stability on that benchmark, but does not by itself exclude prompt-set overfitting or establish multi-seed transfer on every backbone.

Q1: Ported baselines and strength sweeps. We retained each baseline's main published settings and mapped only the required scale/layer locations to Infinity. Infinite Story uses its official Infinity-2B implementation (not a separate port).

MethodLayersScales / placement
SPINSRT: 4,6,9,16,24,31; PWT: 4–9; R$^2$SA: allSRT: 1–12; PWT: 1–5; R$^2$SA: 3–12
Infinite StoryASI/SGA: allASI/SGA: 2–3 (SearlyS_{\mathrm{early}}); IPR: text embedding
StoryDiffusion-style portConsistent self-attention: all0–4: always paired; 5–12: paired with keep probability 0.7

All placements were fixed during sweeps, with one strength variable per method: SPIN's λ\lambda (scaling SRT/PWT together at their original ratio; λ=1\lambda=1 is default), Infinite Story's ww, and StoryDiffusion's reference-token bank size. For StoryDiffusion, we retained the original sampling rate of 0.7 and swept bank size 4–96 (default 12). All sweep ranges were set in advance, without point-wise retuning.

Matched-DreamSim comparison. Entries below use linear interpolation between measured sweep points; fractional bank sizes denote interpolated effective strengths, not actual noninteger banks.

DreamSimMethodStrengthPose ↑\uparrowCentroid ↑\uparrow
0.18SPINλ≈1.130\lambda\approx1.1300.07340.0928
0.18Infinite Storyw≈0.471w\approx0.4710.06730.0796
0.18StoryDiffusionbank ≈5.5\approx5.50.06610.0833
0.17SPINλ≈1.747\lambda\approx1.7470.06570.0849
0.17Infinite Storyw≈0.652w\approx0.6520.06140.0749
0.17StoryDiffusionbank ≈8.8\approx8.80.06160.0763
0.14SPINλ≈3.12\lambda\approx3.120.05510.0705
0.14Infinite StoryN/A––
0.14StoryDiffusionbank = 240.05310.0606

At DreamSim = 0.18, SPIN maintains higher pose diversity than Infinite Story (0.0734 vs. 0.0673) and StoryDiffusion (0.0734 vs. 0.0661); its centroid diversity is also higher (0.0928 vs. 0.0796/0.0833). The advantages persist at DreamSim = 0.17, and at 0.14 SPIN retains higher diversity than StoryDiffusion within the evaluated ranges. N/A means the target DreamSim was not attained in the tested strength range. We will add all measured sweep curves to Fig. 3b and distinguish measured points from interpolated comparisons.

I thank the authors for the extensive additional experiments.

1; The LlamaGen port (response to ej1L) required redesigning most components, including a gate threshold changed after it disabled SRT/PWT, and gave a 5.3% gain versus 22.4% on Infinity-2B. This suggests the recipe does not transfer cleanly.

2; Gate. At matched dose, the fixed gate gives better DreamSim (0.1774 vs. 0.1819) with nearly identical pose diversity (0.0705 vs. 0.0711). As the authors acknowledge, the adaptive gate can be removed.

I would encourage the authors to simplify the method to the components supported by the analysis.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.