- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies which components of a frozen teacher representation help diffusion-model alignment. Latent-Aware Purification (LAP) subtracts a component predictable from clean latent patches before using the residual as an alignment target. The paper considers constant, linear, and nonlinear predictors, develops a conditional-information interpretation, and proposes a target-concentration statistic, C0, as a diagnostic. Fixed-teacher interventions, experiments with multiple teachers and alignment recipes, tokenizer variations, larger models, and aerial-image datasets support the central finding: making teacher features more directly reconstructive need not improve generation, while removing predictable components can substantially help some teachers.
Critical strengths
- The strongest contribution is the fixed-teacher experimental analysis. The predictable-only DINOv2 target yields FID 56.71, worse than the no-alignment baseline at 53.92, while the residual target reaches 41.47, close to raw-teacher alignment at 41.75 (Figure 2 and Table 21). Separately, the auxiliary-prediction experiment reports velocity-probe R-squared increasing from 0.115 to 0.534 (Table 5), while generation FID worsens from 41.75 to 46.79 (Table 21). These interventions meaningfully challenge a simple reconstruction-based explanation. The appendices further examine centering, spatial effects, and fixed versus redrawn predictable targets, substantially reducing easy alternative explanations.
- LAP is simple and produces large improvements where the proposed diagnosis predicts a problem. In Table 1, MAE alignment improves from 53.05 to 42.30 FID with LAP-N; the SiT-L/2, batch-256 MAE experiment in Table 17 improves from 15.39 to 12.06. Compositions with several alignment recipes and tokenizer settings broaden the evidence. Reporting weak or negative results for already effective teachers, transfer to RESISC45 and AID in Table 4, and limitations of the information-theoretic interpretation makes the overall argument more credible.
Critical weaknesses
- The practical generality is weaker than an unqualified reading of the headline suggests. Large gains are concentrated in high-concentration targets such as MAE, while DINOv2 improvements are small and sometimes negative. For example, the block-4 experiments in Table 18 give 36.28 versus 36.79 for raw versus purified DINOv2 REPA, and 27.05 versus 27.23 for REG. Table 2 also selects the best LAP variant per setting; moreover, its iREPA uses a depth-matched projector and its VA comparison isolates the alignment schedule rather than the full Stable Velocity objective. These are disclosed, but the main presentation should distinguish composability in these controlled implementations from superiority over each complete published method.
- C0 and the conditional-information argument have a more limited explanatory scope than the core intervention results. The 13-encoder study is encouraging, but encoders share families and a common training recipe. In the external iREPA analysis for SiT-XL/2 at 100K steps, the partial rank correlation with improvement after controlling for raw FID is -0.13, so C0 is not yet a generally calibrated predictor of benefit. Likewise, the conditional bound may become uninformative when the full clean latent grid is highly informative, and raw targets also admit a conditional bound. The theory motivates purification but does not establish that the removed component is a causally distinct nuisance factor.
Questions for authors
- Clarification: Can the main tables explicitly identify the adapted iREPA/VA implementations and distinguish best-per-setting LAP selection from a fixed selection rule? Do the conclusions remain similar when the variant chosen in the default setting is carried unchanged to the block-4 and larger-model experiments? This would determine how broadly the practical recommendation should be stated.
- Analysis request: Does the C0-benefit relationship retain its predictive ordering when an entire encoder family is held out, rather than another size of a family already present in the sweep? Given the external partial-correlation result, evidence that the diagnostic transfers to a held-out family or alignment recipe would materially strengthen the claim that it can guide teacher selection.
- Can you evaluate LAP on a text-to-image setting such as CC12M, LAION, or COCO with a text encoder (e.g., T5, CLIP, or Gemma) and a DiT/SiT/PixArt-style backbone?
3 comments
Dear Reviewer GDr4,
Thank you so much for your prompt reply and positive feedback! We've revised the manuscript following your comments and updated the PDF accordingly. The revision was marked in blue. Please check out and let us know if there's any other comments. We would greatly appreciate your consideration of a higher score.
Thank you again for your time and we are always happy to address any questions you may have!
All my problems are solved. And the response is consistent with the updated manuscript. So I raise my score to 4.
Thank you very much to the author for revising some of the expressions and adding some experiments, including the results on the text-to-image public MS-COCO dataset. However, I don't see the revised manuscript at the moment. I will decide whether to raise score based on whether the revised manuscript is consistent with the response.