- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces multi-layer feature fusion into RAE, leveraging intermediate-layer features of DINOv2 to supplement the final-layer representations. The method adopts positive and negative routing weights while preserving the original final-layer representations, and then trains the fusion module and decoder in stages. In experiments, the reconstruction rFID decreased from 0.570 to 0.290; after incorporating AutoGuidance, the class-conditional generation FID dropped from 1.740 to 1.650, and the text-to-image GenEval increased from 0.560 to 0.600.
Critical strengths
- The reconstruction performance is significantly improved. Improvements are observed in rFID, PSNR, LPIPS and SSIM. Fine details such as digits and human faces in the demonstrations are clearer, while the number and dimension of tokens to be predicted by the generative model are not increased.
- The trade-off between reconstruction and generation echoes the findings of VAVAE. Experiments including joint training and encoder fine-tuning show that better reconstruction does not necessarily facilitate generation. Experiments on preserving final-layer representations and staged training provide concrete evidence for this observation.
Critical weaknesses
- Lack of direct comparison with existing multi-layer RAE methods. RAEv2[1] has improved reconstruction using simple multi-layer summation.
- Limited generation gains. With AutoGuidance enabled, the FID decreased from 1.740 to 1.650; without guidance, it instead rose from 2.160 to 2.678.
[1] Singh, Jaskirat, et al. “Improved Baselines with Representation Autoencoders.” Advances in Neural Information Processing Systems. 2026.
Questions for authors
- Under the same settings, what performance can simple multi-layer summation achieve? How much additional gain does the method proposed in this paper bring compared with RAEv2?
- Why does the generation performance degrade without guidance after multi-layer fusion? When comparing different guidance strengths under the same guidance training budget, does the proposed method still retain advantages?