- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces manifold-faithful traversal, enabling stable traversal of autoencoder latent spaces. It constrains latent perturbations to remain close to the learned data manifold. The authors propose a two-phase training framework in which a reconstructive autoencoder is first learned, followed by training the decode–encode to project perturbed latent codes toward the manifold using the nearest clean latent code. Experiments on synthetic data and Colored MNIST show that the method reduces drift under both random and target-directed traversal.
Critical strengths
Using a separate decode–encode mapping and the nearest sample point for correction is an intuitive and reasonable approach. The authors also provide corresponding theoretical analysis and validate the method on two simple and intuitive datasets.
Critical weaknesses
My main concern is the relevance of the problem studied in the context of current AI research. The self-supervised learning capability of traditional autoencoder architectures has received relatively limited attention in recent years, partly due to their limited generalization ability. This may limit the practical impact of the proposed work on the current research community. This concern is also reflected in the relatively limited number of closely related recent publications at major conferences. Moreover, the idea of introducing an additional decode–encode mapping for correction is not particularly uncommon, and using similar sample points for such correction is also a relatively straightforward idea.
Questions for authors
- Could the authors further clarify the importance and relevance of this research in the context of current AI research?
- Could the authors provide a more comprehensive review of related work that adds additional components to autoencoders, such as decode–encode mappings, and clarify the novelty of the proposed approach relative to these existing methods?
3 comments
Thank you very much for the authors' update. I acknowledge that I am not an expert in this research area. However, conducting experiments only on Swiss Roll and Colored MNIST gives me the impression that this work may have limited impact on addressing frontier research problems.
Based on the authors' response, I believe that the proposed method has considerable potential to contribute to several important and emerging research problems. Combined with theoretical guarantees, this could make for a promising paper. Therefore, conducting experiments demonstrating the generalization capability of the proposed method on frontier problems would make the work much more convincing.
Alternatively, could the authors provide some references showing that evaluating methods only on Swiss roll and MNIST remains a common practice in this research community, e.g., ICLR, ICML?
Response to Reviewer 9NYr
We thank the reviewer for the follow-up, and for seeing potential in the method and its theory.
Is evaluation on synthetic data and MNIST common practice? Yes, for work whose central claim is about the geometry of the data manifold. Recent examples at ICLR and ICML, in autoencoders and in flow matching, are evaluated at the same scale as ours.
- Zhan et al. (ICLR 2026), Bi-Lipschitz autoencoders with an injectivity guarantee, evaluated on the swiss roll, dSprites and rotated MNIST.
- Diepeveen et al. (ICML 2025), score-based pullback Riemannian geometry and Riemannian autoencoders, evaluated on synthetic manifolds, synthetic image manifolds and MNIST.
- Chen & Lipman (ICLR 2024, oral), flow matching on general geometries, evaluated on Earth-science datasets of at most 13k points, meshes and maze-shaped manifolds.
- Nazari et al. (ICML 2023), geometric autoencoders, evaluated on MNIST, Fashion-MNIST and single-cell data.
The reason these works and ours rely on such datasets was stated by Berthelot et al. (ICLR 2019), the closest prior work on decoding between codes. Whether a decoded point lies on the data manifold can only be measured when the manifold is known. They built a synthetic dataset and evaluated interpolation quantitatively there. Our evaluation follows the same logic. The swiss roll is where the distance to the true manifold is exact, and Colored MNIST shows that the effect transfers to images.
What this setting lets us measure. Because the manifold is known, we can check Definition 1 exactly at every frame of a walk, not only at its end. We used this in a controlled comparison, now in App. C.4, where both objectives are fine-tuned from the same model and only the training target and the noise scale differ (10 seeds, three latent widths). A walk counts as a success only if it reaches the target and every decoded frame stays within a tolerance of the true manifold. At , ours succeeds in 80.8% of walks against 40.5% for denoising (gap 40.3 points, 95% CI 32.7 to 47.1), in all 10 seeds and at every latent width and step size we tested (Table 4). Together with Prop. 2 and its numerical check (Table 3), this isolates the effect of the target, which is only possible when the manifold is known.
Frontier problems. We agree that scaling up is the important next step, and a natural one. The works above show how this tends to happen. Flow matching on general geometries was introduced on small manifolds (Chen & Lipman, ICLR 2024) and extended to protein backbone generation in the same year (Bose et al., ICLR 2024). Our objective is in a similar position. It applies wherever a model is trained to return perturbed points to the data, including the autoencoders behind latent diffusion and image tokenizers (general response, point 4), and Sec. 6 gives an initial recipe for larger scales, with a memory bank that supplies the reference codes and a learned direction for traversal. We could not run this reliably within the discussion period, and the paper names it as the main direction for future work.
We hope these references show that our evaluation follows the standard of this line of work, and that the theory and the controlled results make the contribution clear. If so, we would be grateful if the reviewer would consider raising the score.
Response to Reviewer 9NYr
We thank the reviewer for finding the approach intuitive and reasonable, and for the positive assessment of soundness and presentation.
Q1. Relevance. Autoencoders remain central to generative modeling. Latent diffusion and flow models generate in the latent space of a pretrained autoencoder (Rombach et al., 2022; Peebles & Xie, 2023; Esser et al., 2024), so their samples land on latents the decoder never saw during reconstruction training. The autoencoder in these systems is itself an active research topic at major venues. Recent work studies how its latent space supports generation (Yao et al., CVPR 2025), regularizes its geometry (Kouzelis et al., ICML 2025; Skorokhodov et al., ICML 2025), compresses it further (Chen et al., ICLR 2025), and trains it jointly with the generator (Leng et al., ICCV 2025). Sparse autoencoders are a standard tool for interpreting language models (Huben et al., ICLR 2024), and the dynamics of iterated encoding and decoding, the setting closest to ours, was an oral at ICLR 2026 (Fumero et al.). The specific question of decoding off the embedded codes goes back to Berthelot et al. (2019), who showed that standard objectives do not make decoded interpolations realistic. Our method addresses this directly, and can also be read as latent augmentation, where a model trained with Lproj provides small latent steps whose decodes stay near the data manifold (Eq. 8, Table 1). We added this motivation to the introduction.
Q2. Novelty relative to relevant work. We agree decode-encode consistency terms are common, as discussed in Sec. 7, they differ from ours in the target, and we now make this explicit. They differ from ours in what goes into the cycle and what it is pulled toward.
Every other loss in the table either sends the cycle back to where it started or matches the distributions without a pointwise target. Their optimum is the identity on the prior or the latent space, or an average of codes that sits off a curved manifold (denoising). Ours is the only one that sends a perturbed code to the data manifold rather than back to its own origin. The change is small, from the original code to the nearest real code, and it is what turns the optimum into a projection with a convergence rate rather than a conditional mean (revised Prop. 2). Empirically, walks trained with this target stay at a bounded distance from the true manifold, while the latent-cycle objective, the closest decode-encode alternative, keeps drifting (Table 1, distance 2.96 against 41.6 after 40 steps).
Any model trained to return perturbed points to the data, such as autoencoders behind latent diffusion models, faces the same choice of target, and Prop. 2 shows which one lands on the data manifold. We see this simple change with a provable effect as a main contribution of the paper, and the revised Sec. 4.1 states it explicitly.
We hope the added context shows that the question matters for how autoencoders are used today, and that the nearest-code target is a precise change with provable consequences rather than a generic decode-encode term. If so, we would be grateful if the reviewer would consider raising the score.