- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
SPECTRA is a training-free controller for source-preserving image editing. At every sampling step, it obtains clean predictions from a source-reference branch and a target-conditioned branch, decomposes them into multiscale directional subbands, and uses source statistics plus attention-based spatial gating to decide how much source information to retain. The fused clean prediction is then converted into the update expected by the backbone's native DDIM or FlowMatch scheduler. Across the four main benchmark panels, SPECTRA gives the strongest structural and appearance-preservation scores, but several baselines have higher CLIP-T, making the preservation-versus-editability trade-off central to the result.
Critical strengths
- Framing preservation as allocation across scale, direction, space, and sampling time is intuitive, and the clean-prediction interface gives a concrete way to use the same controller with both diffusion and flow backbones.
- The paper evaluates nine baselines, four backbones, two editing objectives, and several complementary preservation metrics. The scale-depth, direction-assignment, transform-family, and DCRCI ablations are more informative than a single end-to-end comparison.
- Reproducibility is handled carefully. The paper fixes task manifests and paired settings, records model and scheduler configurations, and provides detailed pseudocode and diagnostic logging for the mechanism.
Critical weaknesses
- SPECTRA has lower CLIP-T than several baselines in every main benchmark panel. This is consistent with stronger source retention, but it also means that part of the apparent gain may come from making a weaker edit. More direct human evaluation of edit success at matched preservation levels would help establish a better trade-off rather than a different operating point.
- The cross-backbone results are not controlled component comparisons: SD3 and FLUX use strength 0.4 for SPECTRA and 0.6 for Vanilla, while the U-Net variants additionally use Self Attn-Conv. These results support portability of the interface, but not the size of the controller's causal contribution.
- Within-scale direction adaptation produces modest DINO improvements, while Edge, CSD, and CLIP-T are mixed. The paper acknowledges this, but it makes the broad Scale × Direction framing stronger than the isolated evidence for the Direction component.
- The method evaluates separate source and target branches and applies directional transforms at every selected step. GPU specifications are provided, but runtime, memory overhead, and the cost relative to the baselines are not.
Questions for authors
Please refer to the Weaknesses section.
4 comments
- The cross-backbone results are not controlled component comparisons: SD3 and FLUX use strength 0.4 for SPECTRA and 0.6 for Vanilla, while the U-Net variants additionally use Self Attn-Conv. These results support portability of the interface, but not the size of the controller's causal contribution.
Matched img2img strength on SD3 and FLUX. We agree that the original strength mismatch complicated the comparison. We reran native SD3-Medium and FLUX.1-dev img2img at strength 0.4, matching the submitted SPECTRA runs within each backbone on the same benchmark cases.
Matched-strength comparisons from Table 3. All rows use img2img strength 0.4. DINO is scaled by 100. Bold marks the better result within each backbone pair.
(a) Color40 (n = 40).
| Backbone | Method | DINO ↓ | Edge ↑ | CLIP-T ↑ |
|---|---|---|---|---|
| SD3-Medium | Native img2img | 0.23280 | 0.95803 | 0.30236 |
| SD3-Medium | SPECTRA | 0.1845 | 0.963370 | 0.299517 |
| FLUX.1-dev | Native img2img | 0.21492 | 0.95865 | 0.29401 |
| FLUX.1-dev | SPECTRA | 0.1870 | 0.965861 | 0.293674 |
(b) Style-9 (n = 80).
| Backbone | Method | DINO ↓ | Edge ↑ | CLIP-T ↑ | CSD ↑ | ΔE₀₀ ↓ | LPIPS ↓ |
|---|---|---|---|---|---|---|---|
| SD3-Medium | Native img2img | 0.25694 | 0.95636 | 0.30829 | 0.80077 | 4.92691 | 0.22454 |
| SD3-Medium | SPECTRA | 0.1646 | 0.969802 | 0.296483 | 0.873107 | 3.054036 | 0.159466 |
| FLUX.1-dev | Native img2img | 0.22416 | 0.95783 | 0.29171 | 0.88156 | 4.77811 | 0.22328 |
| FLUX.1-dev | SPECTRA | 0.1586 | 0.970501 | 0.292088 | 0.923750 | 3.243154 | 0.170879 |
At matched strength, SPECTRA improves every reported preservation metric on both backbones. DINO decreases by 20.7% on SD3 and 13.0% on FLUX for Color40, and by 35.9% and 29.2%, respectively, for Style-9. CLIP-T is lower in three comparisons and slightly higher for FLUX on Style-9. Table 3 in Section 4.4 reports these comparisons and specifies the shared strength in its caption.
Role of Self Attn-Conv and controller attribution. The U-Net and flow variants share SPECTRA’s core controller, which combines source-derived scale–direction weighting with SSMO-guided fusion of clean predictions. Self Attn-Conv is an additional source-injection mechanism in the U-Net variants. The matched flow results show that SPECTRA improves preservation on both tasks without this injection.
To isolate the controller’s contribution on a fixed backbone, our matched SD2.1 2×2 ablation varies the controller and Self Attn-Conv independently. With injection held on, adding the controller reduces DINO by 25.6% on Color40 and 31.7% on Style-9, improves Edge on both, and lowers CLIP-T by 0.00608 and 0.00811, respectively. The controller also improves DINO and Edge with injection held off. Section 4.8 summarizes this ablation, and Appendix L.1 (“Full-controller factorial ablation”) reports the full results in Table 17.
- Within-scale direction adaptation produces modest DINO improvements, while Edge, CSD, and CLIP-T are mixed. The paper acknowledges this, but it makes the broad Scale × Direction framing stronger than the isolated evidence for the Direction component.
Scope of the Scale × Direction contribution. We agree that the isolated benefit of within-scale directional adaptation is modest. Tables 9–11 show small DINO improvements, while Edge, CSD, and CLIP-T results are mixed. In the full Color40 configuration (Table 13), adding directional adaptation to Scale-only slightly improves mean DINO and Edge, with both paired 95% confidence intervals spanning zero. Assigning the same weights to their corresponding bands improves DINO and Edge over shuffling them, with both intervals excluding zero and a small decrease in CLIP-T. Together, these results support directional adaptation as a modest, conditional refinement.
SPECTRA’s contribution is a controller that allocates source preservation across scales, directions, spatial locations, and denoising steps. The components play complementary roles. Scale weighting determines how much source information to retain at each resolution, directional weighting adjusts that contribution among bands within a scale, and SSMO uses target attention to determine where the contribution is applied at each step. Sections 4.6 and 4.7 describe the directional benefit at the magnitude supported by the isolated ablations.
Matched 2×2 controller ablation on SD2.1. To quantify the contribution of the controller as a whole, we independently toggle joint self-attention and convolutional-feature injection (I) and the complete controller (C), with 0 denoting off and 1 denoting on. The controller combines low-frequency fusion, adaptive scale–direction weighting, SSMO gating, and spectral reconstruction. All four conditions share the same source construction and sampling settings. AdaIN and DCRCI are off throughout Color40 and on throughout Style-9. When enabled, injection uses . Values are dataset means, and bold marks the best mean in each column.
Table 17(a). Color40, appearance changing (n = 40).
| Condition | DINO ×100 ↓ | Edge ↑ | CLIP-T ↑ |
|---|---|---|---|
| Neither (I = 0, C = 0) | 0.73496 | 0.88620 | 0.33317 |
| Injection only (I = 1, C = 0) | 0.19938 | 0.96319 | 0.31689 |
| Controller only (I = 0, C = 1) | 0.58096 | 0.90362 | 0.32918 |
| Full SPECTRA (I = 1, C = 1) | 0.14832 | 0.97060 | 0.31081 |
Table 17(b). Style-9, appearance preservation (n = 80).
| Condition | DINO ×100 ↓ | Edge ↑ | CLIP-T ↑ | CSD ↑ | ΔE₀₀ ↓ | LPIPS ↓ |
|---|---|---|---|---|---|---|
| Neither (I = 0, C = 0) | 0.55899 | 0.91749 | 0.33194 | 0.60665 | 8.95588 | 0.39912 |
| Injection only (I = 1, C = 0) | 0.22486 | 0.96430 | 0.31175 | 0.76792 | 5.16615 | 0.21300 |
| Controller only (I = 0, C = 1) | 0.41180 | 0.93488 | 0.32177 | 0.69717 | 6.51298 | 0.30725 |
| Full SPECTRA (I = 1, C = 1) | 0.15350 | 0.97138 | 0.30364 | 0.83014 | 4.30659 | 0.16164 |
Adding the complete controller to injection lowers mean DINO by 25.6% on Color40 and 31.7% on Style-9 and improves Edge on both datasets, with CLIP-T decreasing by 0.00608 and 0.00811, respectively. The controller also improves DINO and Edge with injection disabled. These gains quantify the contribution of the controller as a whole. The isolated direction ablations characterize the smaller, conditional effect of directional adaptation within it.
Section 4.8 summarizes the controller results, and Appendix L.1 (“Full-controller factorial ablation”) reports both panels in Table 17. Table H.1 reports the paired directional-assignment intervals directly below Table 13 in Appendix H.
- The method evaluates separate source and target branches and applies directional transforms at every selected step. GPU specifications are provided, but runtime, memory overhead, and the cost relative to the baselines are not.
Runtime and peak GPU memory. To quantify the cost of evaluating separate source and target branches and applying directional transforms during denoising, we profiled SPECTRA and the baselines on an RTX 3090 at batch size 1. The within-backbone comparisons with Vanilla measure the combined overhead of the SPECTRA pipeline. First-edit latency includes source preparation, editing, and final decoding, excluding model loading and image writing. For SD2.1, we also report inversion time and the latency of a subsequent edit that reuses the cached inversion. A dash indicates that a timing breakdown is unavailable.
Table 15. Computational cost on PIE-Bench Color40. Timings and per-case peak VRAM are averaged over 40 cases. FIA-Edit uses its native SD3.5-Medium backbone at the same resolution as the SD3-Medium runs.
| Method | Backbone | Resolution | First edit (s) ↓ | Inversion (s) ↓ | Cached edit (s) ↓ | Peak VRAM (GiB) ↓ |
|---|---|---|---|---|---|---|
| SPECTRA | SD2.1 | 512 × 512 | 96.78 | 80.97 | 15.76 | 4.51 |
| Vanilla | SD2.1 | 512 × 512 | 82.71 | 80.26 | 2.45 | 3.12 |
| PnP | SD2.1 | 512 × 512 | 107.69 | 96.53 | 11.19 | 4.51 |
| SPECTRA | SD3-Medium | 1024 × 1024 | 36.96 | — | — | 18.78 |
| Vanilla | SD3-Medium | 1024 × 1024 | 11.35 | — | — | 18.76 |
| FIA-Edit | SD3.5-Medium | 1024 × 1024 | 41.34 | — | — | 20.09 |
On Color40, relative to Vanilla, SPECTRA adds 14.07 s and 1.39 GiB on SD2.1, and 25.61 s and 0.02 GiB on SD3-Medium. On SD2.1, SPECTRA completes the first edit 10.91 s faster than PnP at the same peak VRAM, primarily due to shorter inversion. At 1024 × 1024, SD3-Medium SPECTRA completes the first edit 4.38 s faster than FIA-Edit and uses 1.31 GiB less peak VRAM.
Table 16. Computational cost on PIE-Bench Style-9. Timings and per-case peak VRAM are averaged over 80 cases under the same profiling protocol. AdaIN and DCRCI are optional appearance components. The AdaIN-only setting disables DCRCI and its iterative VAE encoding and decoding.
| Method | Backbone | Appearance setting | Resolution | First edit (s) ↓ | Inversion (s) ↓ | Cached edit (s) ↓ | Peak VRAM (GiB) ↓ |
|---|---|---|---|---|---|---|---|
| SPECTRA | SD2.1 | AdaIN + DCRCI | 512 × 512 | 110.57 | 82.31 | 28.30 | 4.52 |
| SPECTRA | SD2.1 | AdaIN only | 512 × 512 | 97.57 | 80.16 | 17.45 | 4.52 |
| Vanilla | SD2.1 | — | 512 × 512 | 83.69 | 81.26 | 2.43 | 3.12 |
| PnP | SD2.1 | — | 512 × 512 | 107.53 | 96.50 | 11.04 | 4.51 |
| SPECTRA | SD3-Medium | AdaIN + DCRCI | 1024 × 1024 | 55.73 | — | — | 19.49 |
| SPECTRA | SD3-Medium | AdaIN only | 1024 × 1024 | 37.78 | — | — | 18.78 |
| Vanilla | SD3-Medium | — | 1024 × 1024 | 11.86 | — | — | 18.76 |
| FIA-Edit | SD3.5-Medium | Native | 1024 × 1024 | 42.36 | — | — | 20.09 |
On Style-9, disabling DCRCI while retaining AdaIN reduces first-edit latency by 13.00 s on SD2.1 and 17.95 s on SD3-Medium. With AdaIN alone, SPECTRA completes the first edit faster than PnP on SD2.1 and faster than resolution-matched FIA-Edit on SD3-Medium. The recommended balanced Style-9 setting in Table 5 of Section 4.9 also disables DCRCI and achieves better edge preservation than FIA-Edit at comparable target-prompt alignment.
The “Computational cost” paragraph in Section 4.9 summarizes the within-backbone overhead. Appendix K.1 reports both computational-cost tables and the profiling protocol.
Preservation–editing trade-off. Table 5 in Section 4.9, also discussed in our response to Reviewer cBud, motivates our recommended balanced settings for Color40 and Style-9. Both improve CLIP-T over default SPECTRA through configuration changes while retaining its core approach of selectively preserving source information during editing.
Blinded human evaluation. We compared the recommended Style-9 setting with FIA-Edit because the two settings have nearly identical mean DINO over the full 80-case benchmark (Table 5). Eighteen participants evaluated 20 uniformly sampled PIE-Bench Style-9 cases, yielding 360 paired judgments. Participants saw the source image, the exact target instruction, and two anonymous outputs in randomized left–right order. They selected either output, both, or neither based on which edits clearly applied the requested style while preserving the source image’s shapes, poses, and layout.
Table 19. Blinded human evaluation on Style-9.
Each selected output counted as a success for its method. The intervals use 10,000 two-way bootstrap resamples over cases and participants, keeping the methods paired. The observed joint success rate is 6.4 percentage points higher for balanced SPECTRA, with a confidence interval that spans zero. Preservation equivalence on the 20 surveyed cases remains open.
Section 4.9 (“Preservation–Editing Trade-off”) summarizes the human result. Appendix M (“Blinded Human Evaluation on Style-9”) gives the survey protocol and reports the success rates and paired difference in Table 19.