- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper proposes SPIN, a training-free method for preserving subject identity in next-scale VAR models. The starting observation is that exposing source activations to the target carries positional information along with identity, so the layout gets copied at coarse scales. The method puts a module at each of three different parts of a Transformer block. SRT pulls the attention input toward the source, R$^2$SA adds source K/V with rewound RoPE phase to the target softmax, and PWT mixes the difference of per-layer increments instead of the accumulated state. On Infinity-2B, the method lowers DreamSim while keeping about 90% of the backbone's pose diversity, and the same recipe transfers to five backbones. My concerns are that, apart from the RoPE rewind, the analysis behind each design choice is thin, that the result may be just another point on the existing trade-off curve rather than an improvement of it, and that the source of the Tab. 1 baseline numbers is unclear.
Critical strengths
-
The problem framing is reasonable since the paper decomposes diversity loss, which identity-preservation work tends to ignore, along the structural axes of VAR (scale, layer, rotary band). RoPE rewind is the one part where the design actually follows from measurement, as App. G shows that same-position preference concentrates in the high-frequency band and the band ablation agrees.
-
Practicality is compelling, as the method is training-free, one placement rule transfers to five next-scale backbones, and prompt fidelity is mostly preserved.
Critical weaknesses
-
The central claim of the paper, that the trade-off is improved, is not established. Rather, it looks like the method only moves along that trade-off. SPIN has more diversity than Infinite Story but less identity, and turning R$^2$SA on from scale 2 moves it to almost exactly the point of Infinite Story. Relative to its backbone, CoDi retains pose diversity better at a similar identity gain (about 98% vs. 89%). The "above the trend" claim in Fig. 3a mixes absolute numbers across backbones, so it is confounded by each backbone's diversity ceiling. The SDXL-family pose values behind it are also 20 to 30% higher than the CoDi paper's own reproduction, so it is not even clear that the trend is on a comparable metric. So one can still argue for using Infinite Story when identity matters and CoDi when diversity matters, and the paper does not answer this. Since that is the reason for this method to exist, this is the decisive issue.
-
The design comes from tuning rather than analysis, and the ablation does not check it. SRT has only a verbal rationale with no measurement. The claim that carries less positional information than is measured on the source only, not on what is actually injected, and there is no end-metric comparison of mixing instead of . Tab. 2(a) follows a single cumulative order (PWT, then R$^2$SA, then all together including SRT), so the individual contribution and necessity of each module cannot be read, and since every step trades diversity for identity, it is impossible to tell a better trade-off from a stronger setting. The placement logic is also confusing. R$^2$SA is withheld at scales 1 and 2 because they are layout-committing, yet PWT and SRT run on the full grid at those same scales, and the gate that supposedly opens after composition is settled is effectively open from scale 1.
Questions for authors
-
Were the SDXL-family baselines in Tab. 1 reproduced or quoted? The pose values differ from the CoDi paper by 20 to 30% (e.g., vanilla SDXL 0.0967 here vs. 0.0772 there, and similarly for the other three SDXL-family rows), and only the 1Prompt1Story pose value matches. Which keypoint estimator and confidence threshold were used for pose diversity, how are animal and object subjects handled, and, if reproduced, what resolution and seeds were used? App. A only covers the Tab. 3 re-implementation.
-
Can you report each method relative to its backbone, and compare the Pareto curves from strength sweeps of SPIN and Infinite Story?
-
Can you add each module alone, leave-one-out ablations from the final configuration, and a variant with PWT turned off at the layout-committing scales? The current table follows one cumulative order, and the representation-level complementarity in App. D does not substitute for it.
-
What happens when is mixed instead of at the same layers and strength?
-
Were the constants tuned on a separate set from the one used for evaluation?
9 comments
Response to zx8z : Q1, W2 & Q3/Q4
Q1. Source and evaluation protocol for Table 1
The SDXL-family results in Table 1 were re-evaluated on generated images using our own pipeline, not copied from the CoDi paper. CoDi's released code does not include its pose-diversity evaluation, so we reimplemented the protocol described in the paper and applied it uniformly to all methods:
- Keypoints: RT-DETR (detection threshold 0.3) and ViTPose-base (17 COCO keypoints, confidence threshold 0.7).
- Pose distance: Procrustes-aligned distance for image pairs with at least 5 shared valid keypoints.
- Subjects: 74 human/humanoid groups; animals and non-humanoid objects excluded.
- Generation: 1024 × 1024 resolution, seed 100.
The difference from CoDi's reported values (e.g., vanilla SDXL 0.0967 vs. 0.0772) may arise from differences in keypoint detection, evaluated groups, or distance implementation. As the original evaluation code is unavailable, we cannot identify the exact cause. We will label Table 1 "Re-evaluated Using Our Unified Protocol" to distinguish our measurements from the original paper's results.
W2-1 / W2-3 / Q3. SRT and full-factorial ablation
To assess each module beyond the original sequential ablation, we ran all combinations, including individual modules and leave-one-out variants.
| Configuration | CLIP-I ↑ | DreamSim ↓ | Pose ↑ | Centroid ↑ |
|---|---|---|---|---|
| Backbone | 0.8945 | 0.2348 | 0.0842 | 0.1052 |
| R²SA | 0.9000 | 0.2158 | 0.0816 | 0.1036 |
| PWT | 0.9023 | 0.2113 | 0.0824 | 0.0944 |
| SRT | 0.8992 | 0.2137 | 0.0813 | 0.1041 |
| R²SA + PWT | 0.9070 | 0.1979 | 0.0791 | 0.0942 |
| R²SA + SRT | 0.9037 | 0.2064 | 0.0767 | 0.1021 |
| SRT + PWT | 0.9056 | 0.1986 | 0.0732 | 0.0919 |
| Full SPIN | 0.9142 | 0.1821 | 0.0750 | 0.0945 |
SRT alone improves DreamSim from 0.2348 to 0.2137, retaining 96.6% of baseline pose diversity. Adding SRT to R²SA+PWT further improves DreamSim from 0.1979 to 0.1821. Removing any one module from Full SPIN worsens DreamSim: 0.1979 without SRT, 0.1986 without R²SA, and 0.2064 without PWT, versus 0.1821 for Full SPIN. This supports each module's additional contribution to identity preservation when the other two are present.
The modules act at different points: SRT modifies attention inputs, R²SA supplies source K/V with reduced rotary positional bias, and PWT transfers layer increments instead of accumulated residual states. Because identity gains can accompany diversity losses, we separately evaluated the strength-sweep trade-off in W1/Q2.
W2-2 / Q4. Increment versus state transfer
We clarify that the original positional-share () measurements were made on source representations, not directly on injected source–target differences. Positional share measures the fraction of feature variation explained by grid position. On Infinity-2B, token-weighted pooled is 0.092 for layer increments versus 0.155 for residual states, a 40.6% reduction.
To test the effect of the actual transferred representation on generation, we compared write-gap (source–target layer-increment difference) and state-gap (source–target accumulated-state difference) injection, controlling the layers, scales, and injection norm.
| Metric | Write gap | State gap |
|---|---|---|
| DreamSim improvement | 10.0% | 21.5% |
| Pose-diversity loss | 7.0% | 18.2% |
| Centroid-diversity loss | 7.5% | 22.9% |
State-gap transfer gives a larger identity improvement, but 2.6× greater pose-diversity loss (18.2% vs. 7.0%) and 3.1× greater centroid-diversity loss (22.9% vs. 7.5%). Thus, the choice of transferred difference affects the generated identity–diversity balance. We do not equate the measured source-representation with the of the injected difference; these are separate analyses.
PWT alone improves DreamSim from 0.2348 to 0.2113. It also overlaps with R²SA at scales 3–5 (R²SA: 3–12; PWT: 1–5), rather than operating only where R²SA is inactive. Adding PWT to R²SA improves DreamSim from 0.2158 to 0.1979 (8.3%), supporting its distinct role in transferring a less position-dependent layer increment.
Response to zx8z : W2-4, W3 & Q5
W2-4 / Q5. Hyperparameter selection and possible benchmark overfitting
We acknowledge the concern that hyperparameter choices may depend too strongly on the evaluation benchmark. We further analyzed whether the main module placements can be explained by backbone structure and internal representations rather than benchmark scores alone. Specifically: (i) the PWT layer band is selected using render NCC and positional share (); (ii) R²SA starts at the first scale where the token count exceeds the source-bank size (); and (iii) PWT scales follow a common rule covering the small-token region () and up to two scales after R²SA starts.
| Backbone | R²SA start | PWT scales | Analysis layers |
|---|---|---|---|
| Infinity-2B | 3 | 1–5 | 4–10 |
| Switti | 5 | 1–7 | 12–15 |
| HART | 5 | 1–7 | 3–8 |
| Backbone | Mean (inc./state) | Pooled (inc./state) |
|---|---|---|
| Infinity-2B | 0.235 / 0.304 | 0.092 / 0.155 |
| Switti | 0.136 / 0.286 | 0.086 / 0.177 |
| HART | 0.115 / 0.186 | 0.110 / 0.181 |
The analysis bands above are distinct from the Infinity-2B PWT injection layers (4–9). At the selected analysis bands, increments have lower positional share than states in all three backbones. For Infinity-2B, pooled is 40.6% lower for increments. The directly adjusted backbone-specific setting was the R²SA start scale; PWT ranges were determined by the same structural rule, without separate backbone-specific performance searches. In Table 6, identity consistency improved on all five backbones without additional backbone-specific performance tuning.
With unchanged settings across seeds 100, 101, 102 on Infinity-2B, DreamSim ranged from 0.1821–0.1857, pose diversity from 0.0713–0.0750, and CLIP-I from 0.9140–0.9142. In a separate 10-style shift evaluation, DreamSim improved from 0.3453 to 0.2890 (16.3%). These results support the structural explanation of the main placements and show improvements across the tested seeds and styles. They do not, however, replace an evaluation on independent tuning/test prompt sets.
W3 / Q3. Early-scale PWT/SRT placement and adaptive gating
R²SA and PWT/SRT use different transfer paths at early layout scales. R²SA directly adds source K/V to attention, which can exert greater influence when few target tokens are present and may copy source layout. In contrast, PWT transfers a less position-dependent layer increment rather than the accumulated residual state; SRT modifies the attention input without directly adding source state to the residual stream. Thus, using PWT/SRT at early scales while delaying R²SA reflects the differences in what is transferred.
In a separate early-scale ablation, removing PWT at scales 1–2 worsened DreamSim from 0.1857 to 0.1993, indicating an identity contribution from early-scale PWT. The 0.1857 value is the reference for that ablation; the main Full SPIN result remains 0.1821. The norm-controlled write-gap/state-gap comparison in W2-2/Q4 further examines the effect of the transferred representation: state-gap injection caused 2.6× greater pose and 3.1× greater centroid-diversity loss.
The adaptive gate is not a hard switch that blocks injection until composition is complete. It continuously modulates injection strength according to target-specific similarity, including at early scales. At scale 1, the gate is fully closed for 15% of targets, and the bottom 25% have gate values of 0.52 or lower. Its role is therefore target-dependent modulation, not an identical on/off schedule for every target.
To assess this choice, we compared adaptive and fixed gates at an equal 1.519% injection dose; the adaptive results were interpolated from the strength sweep.
| Gate | CLIP-I ↑ | DreamSim ↓ | CLIP-T ↑ | VQA ↑ | Pose ↑ | Centroid ↑ |
|---|---|---|---|---|---|---|
| Adaptive (interp.) | 0.9140 | 0.1819 | 0.3038 | 0.8222 | 0.0711 | 0.0942 |
| Fixed | 0.9145 | 0.1774 | 0.3036 | 0.8208 | 0.0705 | 0.0900 |
At the same dose, the fixed gate has lower DreamSim (0.1774 vs. 0.1819), whereas the adaptive gate retains higher pose (0.0711 vs. 0.0705) and centroid diversity (0.0942 vs. 0.0900; 4.7% higher). We do not claim that the adaptive gate is better on all metrics. Fixed gating is a valid simplification; adaptive gating is a design choice that emphasizes diversity preservation. We will add the early-scale PWT ablation, gate distributions, and matched-dose comparison, and revise wording suggesting that early-scale layout copying is completely prevented.
Response to zx8z : W1 & Q2
We thank the reviewer for raising the central question of whether SPIN improves the identity–diversity trade-off rather than selecting another point on the same curve.
W1 / Q2. Identity–diversity trade-off
We agree that Fig. 3a alone cannot establish a better trade-off from absolute diversity values across backbones, which may have different diversity ceilings. We therefore swept SPIN and Infinite Story on the same Infinity-2B backbone. All other settings and module placements were fixed: SPIN varied only λ, jointly scaling SRT/PWT at their original ratio (λ=1 is the default), and Infinite Story varied its strength w.
At comparable identity, SPIN (λ=2) achieves lower DreamSim than Infinite Story (w=0.7) (0.1659 vs. 0.1675), with 4.9% higher pose (0.0626 vs. 0.0597) and 11.0% higher centroid diversity (0.0817 vs. 0.0736). Near the defaults, SPIN (λ=1) has slightly higher DreamSim than Infinite Story (w=0.5) (0.1821 vs. 0.1780), but 12.1% higher pose diversity (0.0750 vs. 0.0669).
We also compared the methods at matched DreamSim, using linear interpolation of the measured sweep points:
At DreamSim = 0.18, SPIN retains 9.0% higher pose and 16.6% higher centroid diversity; at 0.17, the gains are 7.0% and 13.4%, respectively. Relative to the same vanilla Infinity-2B (DreamSim 0.2348; pose 0.0842), SPIN (λ=2) improves DreamSim by 29.3% and retains 74.3% of pose diversity; Infinite Story (w=0.7) improves DreamSim by 28.7% and retains 70.9%.
These sweeps support a more favorable trade-off over the evaluated strength range, rather than only a different operating point. In particular, the measured λ=2 versus w=0.7 comparison improves DreamSim and both diversity metrics simultaneously.
We also acknowledge the reviewer's observation that activating R²SA from scale 2 moves the variant toward an Infinite Story-like operating point. This suggests that sharing source K/V too early can reduce the diversity advantage. We distinguish this variant from the default, which starts R²SA at scale 3.
CoDi retains about 98% of pose diversity relative to its own backbone, versus about 89% for default SPIN; however, the identity gains differ. When ported to the same Infinity-2B, CoDi improves DreamSim by about 2.8%, versus 22.4% for SPIN. Thus, pose retention alone does not establish a better trade-off at matched identity. We will clarify Fig. 3a's cross-backbone limitation and report backbone-relative metrics alongside same-backbone strength-sweep curves.