acceptodds
Under review as a conference paper at ICLR 2027

The Accent of Feature Drift: Where Reconstruction Error Is Not Noise

Abstract

Reconstruction trajectories provide a readout of how diffusion architectures organize internal features. We study feature drift, the per-layer discrepancy between a reconstruction trajectory and an independently sampled forward reference of the same image, and define the architecture accent as its min–max normalized depth profile. Under a specified readout protocol, the accent is reproducible across tested variants within trained lineages and varies across architectures. Ten model instances spanning five backbone families exhibit two dominant forms, interior-localized and terminal-ramp, with a terminal-spike boundary in two further lineages. The dominant classes are recovered in leave-one-out evaluation within this registry. These forms have different measurement interpretations: terminal ramps largely track the reference features' norm schedule and lose their ramp label under relative L2, whereas the tested interior-localized profiles retain peaks before the final block under norm-free metrics. Controlled interventions show that drift magnitude does not uniquely determine reconstruction quality, and that site sensitivity in training-free correction differs across architectures. Random-initialization controls and retraining under two objectives further indicate that the observed organization develops with training. Together, these results show how reconstruction trajectories can be used to read out architectural organization and identify candidate sites for training-free correction.

Primary area:
generative models
Keywords:
Diffusion Models, DDIM Inversion, Feature Drift, Image Editing, Architecture Accent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2ZxjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies layer-wise feature discrepancies between diffusion reconstruction trajectories and independently sampled forward references. It defines the normalized depth profile as an “architecture accent” and identifies recurring profile shapes across diffusion backbones. The experiments examine stability across model variants and measurement settings, dependence on feature norms, responses to architectural interventions, and emergence during training. The paper also explores using these profiles to select feature-correction sites. The main evaluation uses 104 COCO images, with an additional 121 editing examples drawn from those images.

Critical strengths

The paper includes substantial controls across architectures, model variants, reference seeds, measurement conventions, and inversion frameworks. In particular, the norm-free analyses and exact-inversion controls help clarify what the proposed discrepancy measures. The authors also acknowledge several important limitations, including norm dependence, architecture-specific intervention effects, and the restricted interpretation of the editing results. This makes the observations more informative than an uncontrolled collection of feature visualizations.

Critical weaknesses

1. The additional diagnostic value of the proposed accent is not sufficiently established.

The paper demonstrates reproducible profiles, but it is less clear what useful information these profiles provide beyond simpler feature statistics. Section 3.1 and Appendix C.9 show that the terminal-ramp morphology is largely carried by the feature-norm schedule and does not retain its ramp label under norm-free metrics. Appendix F further reports that drift and reference norm select identical top-five layers in six of seven measured models. The interior-localized profiles surviving normalization are interesting, but the paper still needs to establish what the proposed measurement enables us to predict or do beyond inspecting feature norms.

The correction experiments make this concern concrete. Table 3 reports the following PSNR improvements under matched layer count, injection strength, and reconstruction steps:

Site-selection strategyλ = 0.7λ = 0.3
Accent / drift top-five+3.478+3.415
Reference-norm ranks 6–10+3.568+3.432
Evenly spaced layers+3.418+3.238
Random layers, mean over ten seeds+3.441+3.217

Accent selection trails the norm-based alternative at both strengths. At λ = 0.7, it falls within the random-selection range, with five of ten random sets outperforming it. Separately, latent correction without accent-based site selection achieves +3.67 dB, compared with approximately +3.48 dB for accent-based feature correction. The authors acknowledge that most of the gain comes from correction itself.

The HunyuanDiT result demonstrates sensitivity to the chosen site, but both compared sites are drift-derived, and the selection/evaluation overlap remains unresolved for that model. It therefore does not establish an advantage over simple alternative selection rules. Overall, the observations support reproducibility more clearly than they establish the accent’s incremental diagnostic or practical value.

2. The downstream evaluation is too limited to establish usefulness for reconstruction and editing.

The main experiments use a fixed set of 104 COCO images, and the 121 editing examples also come from this set. The paper discusses PIE-Bench and EditBench but does not report systematic evaluation on them. Its comparisons across many backbones establish the breadth of the measurement study; they do not establish whether accent-guided correction improves representative reconstruction or editing pipelines under comparable conditions. The exact-inversion experiments are useful controls for the measured phenomenon, but they do not resolve this downstream question.

More importantly, Appendix F explicitly states that semantic agreement of the edits is not measured. The reported 85% LPIPS reduction therefore establishes improved fidelity to the source image, without showing that the requested edits are successfully performed. Stronger anchoring could preserve the original image while weakening the edit. Without evaluating both edit success and content preservation, the practical interpretation of this result remains unclear.

Together, these limitations leave a substantial gap between identifying a stable measurement pattern and demonstrating useful guidance for existing methods.

Questions for authors
  1. What specific predictive or diagnostic capability does the accent provide beyond a reference-feature-norm profile? Which results establish this additional capability?
  2. Given the matched-budget results in Table 3, what evidence supports using accent-based site selection over norm-based selection or correction without site selection? Is there an independently evaluated case where the advantage is clear?
  3. Is there evidence that accent-guided correction improves semantic editing success as well as source-image fidelity, relative to representative editing methods on a standard benchmark?

These questions concern the interpretation and significance of the current results, and their answers could materially affect my assessment.

2 comments

On Q1, the accent analysis provides complementary quality-diagnostic and representation-level information. On 100 held-out SD 2.1 images, a post-hoc feature ablation gives the following results:

Feature setPSNR MAE, dBR2LPIPS MAE
Reference-norm profile1.906+0.4450.091
Both norm profiles1.767+0.5210.061
Both norm profiles plus mean drift1.578+0.6170.040
Increment from mean drift over the two-norm baselinePaired error reduction [95% CI]Relative MAE reduction
PSNR0.189 dB [0.058, 0.325]10.7%
LPIPS0.021 [0.014, 0.028]34.0%

The quality-prediction gains above are attributable to mean drift. Separately, decomposing squared drift provides depth-resolved, representation-level diagnostics:

BackboneDominant componentQuantitative result
SD 1.5DirectionDominates at every layer; median share 99.34%
Z-ImageAmplitudeDominates at every layer; median direction share 1.7%

These architecture-dependent composition patterns are not available from a reference-norm profile alone.

On Q2, disjoint evaluation confirms substantial fidelity gains from accent-guided correction.

ModelEvaluation imagesCorrection conditionΔPSNR, dB [95% CI]
SD 2.1150 unusedDrift top-5+3.676 [3.290, 4.071]
SD 2.1150 unusedReference-norm top-5+3.675 [3.290, 4.071]
SD 2.1150 unusedEven spacing+4.020 [3.614, 4.442]
SD 2.1150 unusedLatent correction, no site selection+4.354 [3.930, 4.798]
HunyuanDiT50 held-outDrift top-5+2.99 [2.34, 3.67]
HunyuanDiT50 held-outEven spacing+9.41 [8.46, 10.36]
HunyuanDiT50 held-outMean over five random five-layer sets+8.24

We froze site lists on 104 calibration images and evaluated on separate images. Drift-selected correction improved PSNR by +3.676 dB on SD 2.1 and +2.99 dB on HunyuanDiT, with both 95% CIs excluding zero. All reported correction conditions improved source fidelity on both models. These results independently reproduce substantial correction benefits under a disjoint selection–evaluation protocol. Latent correction is a separate comparison. One HunyuanDiT evaluation image was previously used for an implementation check.

On Q3, we evaluate target-prompt alignment and source preservation jointly. Rather than inferring editing success from fidelity alone, we measure CLIP image-text alignment against the target prompt (CLIP_T) alongside PSNR, on 121 controlled edit pairs.

armlambdaCLIP_Tdelta CLIP_T [95% CI]PSNR (dB)delta PSNR [95% CI]
No correction-0.2467-18.41-
Late-window drift50.10.2367-0.0100 [-0.0137, -0.0065]18.90+0.495 [+0.454, +0.539]
Drift50.10.2198-0.0270 [-0.0348, -0.0192]21.75+3.338 [+3.068, +3.610]
Uniform50.10.2258-0.0209 [-0.0281, -0.0136]20.66+2.252 [+2.082, +2.424]
Norm50.10.2221-0.0247 [-0.0323, -0.0171]21.48+3.073 [+2.834, +3.327]
Drift50.70.1988-0.0479 [-0.0572, -0.0386]25.88+7.473 [+6.863, +8.063]
Latent0.70.1972-0.0495 [-0.0590, -0.0401]26.72+8.312 [+7.642, +9.009]

All reported correction settings increased source-image PSNR. Among the corrected arms, late-window drift5 (λ = 0.1) retained the highest target-prompt alignment (CLIP_T = 0.2367), with the smallest alignment change (−0.0100), while increasing PSNR by 0.495 dB. The complete grid maps the fidelity–editing trade-off and identifies a low-strength operating point that best preserves target alignment among the tested corrections.

Supplement to Q3 — Standard-benchmark evaluation

As a supplement to our previous response on Q3, we have completed an evaluation on a 120-edit sample of PIE-Bench++, measuring target-prompt alignment and source-image preservation jointly. The sample contains 30 edits each of object change, object addition, object deletion, and attribute change, drawn with a fixed seed. We invert with the source prompt and reconstruct with the target prompt, applying correction at five layers with λ = 0.1 over the last 20 of 50 steps.

Alignment is the cosine between the image-direction and text-direction vectors, cos(CLIP(edited) − CLIP(source), CLIP(target) − CLIP(source)); preservation is measured by whole-image PSNR and LPIPS against the source image. The mirror copy of PIE-Bench++ does not ship usable masks, so preservation is measured on the whole image rather than inside and outside the edit region.

ArmΔCLIP-Dir [95% CI]ΔPSNR, dB [95% CI]LPIPS reduction
Drift5−0.0113 [−0.0172, −0.0060]+0.522 [0.484, 0.560]+0.0367
Norm5−0.0101 [−0.0151, −0.0054]+0.499 [0.462, 0.537]+0.0350
Uniform5−0.0137 [−0.0200, −0.0080]+0.713 [0.668, 0.757]+0.0521
Random5, three fixed sets−0.0080 to −0.0047+0.209 to +0.457+0.015 to +0.032

LPIPS reduction is measured against the no-correction baseline, so larger values are better. Drift5 closely matches Norm5 in mean PSNR gain (+0.522 versus +0.499 dB) and achieves higher per-edit PSNR on 114 of 120 edits. Drift5 also gives higher per-edit PSNR than each fixed random set on 101 to 119 of 120 edits, with mean paired advantages of +0.06 to +0.31 dB and a +0.174 dB advantage over the three-set mean. The PSNR ordering Uniform5 > Drift5 > Norm5 > the random-set mean appears in all four edit types. Uniform5 yields a larger preservation gain (+0.713 dB) at a larger alignment cost (−0.0137 against −0.0113 for Drift5).

Together with the previously reported 121-pair grid, every reported correction setting increased source-image PSNR. The standard-benchmark sample further demonstrates that accent-guided correction provides consistent source-preservation gains over fixed random site selection, while matching norm-based selection and trading a smaller alignment cost for a smaller preservation gain than even spacing.

Reviewer_PFAcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes architecture accent, a normalized depth profile of feature drift measured along a diffusion reconstruction trajectory. For each hooked layer, the method compares a reconstructed feature with a feature from an independently sampled forward reference of the same image at matched timesteps. The resulting per layer discrepancies are averaged over images and min max normalized across depth. The paper studies whether this profile is stable within trained model lineages, distinct across diffusion architectures, and useful for choosing sites for training free correction. Under a fixed readout protocol, the authors measure model instances from several backbone families and identify two main profile types, called interior localized and terminal ramp, together with a terminal spike boundary type. They report stable peak locations across lineage variants and protocol perturbations, and use leave one out prediction to recover the profile class within their measured registry. Additional analyses examine relative distance metrics, readouts beyond the final block, local component interventions, random initialization controls, retraining under different objectives, and feature correction site selection. The results suggest that terminal ramps are largely carried by feature norm schedules, while interior localized profiles retain peaks under norm free metrics. Peak skip interventions in Stable Diffusion 1.5 and SDXL have opposite effects on reconstruction quality, indicating architecture dependent pathway sensitivity. In the correction experiments, accent guided site selection performs similarly to several simple alternatives on Stable Diffusion 1.5, while HunyuanDiT exhibits stronger site sensitivity.

Critical strengths
  1. The paper defines a concrete and reproducible measurement protocol for studying internal feature organization in diffusion reconstruction. The use of an independently sampled forward reference, matched timesteps, multiple readout conventions, and protocol perturbations makes the object of study more precise than an informal claim about feature dynamics.

  2. The authors include several informative controls that challenge simple explanations. Extending the readout beyond the final block, using relative metrics, testing random initialization, retraining under different objectives, and comparing interventions across Stable Diffusion 1.5 and SDXL provide useful evidence that the observed profiles are not explained by a single trivial artifact.

Critical weaknesses
  1. The central architecture dependent interpretation is based on a small and highly selected registry. Ten dominant instances from five families are used for the main morphology and leave one out analyses, with several variants drawn from related lineages. The classification rule is defined from peak position and monotonicity, so perfect leave one out recovery within this registry does not establish generalization to unseen architectures or a naturally occurring taxonomy. The reported classes may also conflate architecture with feature scale, parameterization, training data, and checkpoint family.

  2. The practical correction claim is not yet strongly supported. On Stable Diffusion 1.5, accent guided five layer selection is close to even spacing and random selection, and it trails norm adjacent selection in the reported settings. The stronger HunyuanDiT site sensitivity is evaluated on a small intervention study with an overlap caveat. Thus the accent is currently a descriptive diagnostic more convincingly than a generally useful method for choosing correction sites. Several pathway interventions also change feature content, scale, and downstream computation simultaneously, limiting causal interpretation.

Questions for authors
  1. Can the authors evaluate the accent on a genuinely held out set of architecture families and checkpoints that were not used to define the morphology rule or its threshold? Please include a preregistered prediction task for unseen families, report confidence intervals for classification accuracy, and compare against baselines using parameter count, feature norm schedules, block count, training objective, and family labels.

  2. How robust are the morphology assignments to choices that are currently fixed by the canonical protocol? Please provide sensitivity analyses over the image set, number and placement of timesteps, inversion solver, hook granularity, pooling rule, precision, and reference noise coupling. In particular, clarify whether the same profile class is recovered when the reference noise is coupled or partially shared with the reconstruction trajectory.

  3. Since terminal ramp profiles lose their ramp label under relative L2 and are strongly correlated with feature norm schedules, what remains of the architecture specific claim after explicitly regressing out norm growth? Please report a quantitative comparison of the residual profile across architectures and test whether residual profiles predict topology, training objective, or reconstruction quality beyond a norm schedule baseline.

  4. Can the correction site experiments be expanded with a fully disjoint selection and evaluation protocol? The current HunyuanDiT analysis acknowledges overlap between site selection and evaluation images. Please select sites on one image set or calibration protocol and evaluate on a separate set, with confidence intervals and matched compute, and compare against learned, gradient based, norm adjacent, and random site selectors.

  5. What is the mechanism connecting a drift peak to reconstruction quality or information pathways? The skip interventions establish that some sites are consequential, but they do not isolate mediation because they simultaneously alter feature content and downstream computations. Can the authors add intervention controls that preserve feature norms and downstream activation statistics, or use causal mediation or patching analyses to test whether the peak itself predicts the effect of a correction?

1 comment

Thank you for recognizing the protocol and controls. We added independent correction evaluations, held-out quality prediction, reference-noise coupling, and activation-value-preserving interventions.

Q1 — Generalization. The frozen rule recovers every class under leave-one-out evaluation without retuning:

EvaluationResult
Frozen morphology rule10/10
Family-label baselineAt most 9/10

Q2 — Protocol robustness.

ControlResult
Image half-splitsSpearman ≥0.9987; same peak in 20/20 splits
Reference seedsSame peak on four models; maximum D_s 0.0017
Three exact-inversion armsNo peak movement
Precision, solver, element-count renormalizationAssignments preserved

On 50 held-out SD 2.1 images, independent, partially coupled (c=0.5), and fully coupled (c=1) references all recover up_blocks.2.resnets.0 and interior-localized on 50/50 images. S0 uses stochastic DDIM (eta=0.7) with independent references; S50/S100 share its exact reconstruction trajectory. D0 instead uses deterministic DDIM (eta=0).

ArmSpearman vs S0Max abs [95% CI]Mean abs
D0+0.99740.0556 [0.0495, 0.0632]0.0101
S50+0.99980.0113 [0.0104, 0.0123]0.0015
S100+0.99970.0216 [0.0204, 0.0228]0.0028

Differences use per-image min–max-normalized profiles (unitless 0–1 scale); max abs averages each image's worst-layer deviation. Solver change gives 2.6× the worst-layer and 3.6× the mean deviation of full coupling.

Q3 — Information beyond norm schedules.

Profile inputLeave-one-out class recovery
Absolute drift6/6
Reference-norm schedule6/6
Norm-regressed residual4/6

The residual retains partial class information, while mean drift carries the quality-prediction gains. A post-hoc ablation on 100 held-out SD 2.1 images gives:

PredictorPSNR MAE, dBLPIPS MAE
Both norm profiles1.7670.061
Plus mean drift1.5780.040

Mean drift reduces the two-norm baseline's PSNR MAE by 10.7% and LPIPS MAE by 34.0%. Depth decomposition separately shows direction dominance throughout SD 1.5 (median share 99.34%) and amplitude dominance throughout Z-Image (median direction share 1.7%), information unavailable from reference norms alone.

Q4 — Independent correction evaluation. Site lists frozen on 104 calibration images were evaluated on disjoint images:

Model (n)ConditionΔPSNR, dB [95% CI]
SD 2.1 (150)Drift top-5+3.676 [3.290, 4.071]
SD 2.1 (150)Reference-norm top-5+3.675 [3.290, 4.071]
SD 2.1 (150)Even spacing+4.020 [3.614, 4.442]
SD 2.1 (150)Latent, no site selection+4.354 [3.930, 4.798]
HunyuanDiT (50)Drift/norm top-5, same layers+2.99 [2.34, 3.67]
HunyuanDiT (50)Even spacing+9.41 [8.46, 10.36]
HunyuanDiT (50)Mean of five random sets+8.24

Feature arms match layer count, strength, and reconstruction settings. Every SD 2.1 condition improves PSNR, SSIM, and LPIPS on every image. One HunyuanDiT image was used for an implementation check, as disclosed in the run record.

Q5 — Intervention controls. On 50 held-out SD 1.5 images, the controls localize effects to specific sites and stages:

ConditionControlΔPSNR, dB [95% CI]
Cut A, both passesZero bundle to up_blocks[2]+3.16 [2.42, 3.94]
Shuffle A, both passesPreserve each channel's activation-value multiset+3.39 [2.66, 4.17]
Adjacent bundleZero adjacent skip bundle+2.14 [1.49, 2.80]
Cut A, reconstruction onlyReconstruction-stage ablation−6.55 [−7.33, −5.83]

Cut A and Shuffle A agree within ±0.5 dB: the improvement persists with each channel's activation-value multiset fixed. The reconstruction-only control shows strong stage dependence.

Reviewer_XqgKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors introduce the notion of an architecture accent, defined as the min–max normalized feature-drift profile across network depth (the layer-wise discrepancy between features obtained from diffusion reconstruction trajectories and independently sampled forward references). They investigate whether these profiles are reproducible within model lineages, whether they differ across architectures, and whether they can inform the selection of intervention sites for reconstruction correction, finding architecture-dependent effects and showing that drift magnitude alone does not predict reconstruction quality.

Critical strengths

-I found the finding that layer-wise feature discrepancies exhibit reproducible patterns across related models potentially interesting. Characterizing these patterns across different diffusion architectures provides a useful empirical perspective on their internal representations.

  • Also, re: the experiments involving skip connections and correction-site selection: The observation that interventions can increase or decrease drift while improving reconstruction quality is interesting
Critical weaknesses

-I found it genuinely difficult to get past the framing of the paper, particularly the repeated suggestion that reconstruction error is being treated as “noise.” The title, Where Reconstruction Error Is Not Noise, appears to suggest that existing work assumes reconstruction errors are meaningless or unstructured noise. However, I do not see evidence that this is an assumption held by the literature.

-The authors show that different models exhibit different profiles, but I think it remains unclear what architectural properties can actually be inferred from these profiles beyond the empirical classification into different shapes.

Questions for authors

-re:The title, Where Reconstruction Error Is Not Noise, appears to suggest that existing work assumes reconstruction errors are meaningless or unstructured noise. However, I do not see evidence that this is an assumption held by the literature. Can authors please elaborate?

-Also, the intervention experiments are useful, but I think the mechanistic conclusions should be interpreted cautiously. For example, removing a skip connection simultaneously changes the information available to downstream layers and the resulting computation. The observed changes in drift and reconstruction quality therefore do not isolate the mechanism responsible for the improvement, right?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.