acceptodds
Withdrawn from ICLR 2027

Facial Stylization as Geodesic Navigation on Diffusion Manifolds

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Facial stylization asks for a delicate transformation: the image should look unmistakably artistic, yet the person should remain unmistakably the same. Existing training-free and instruction-driven methods often fail to expose such control, either drifting away from identity or producing only weak stylization. We introduce Stylus, a training-free framework that treats facial style transfer as navigation on the curved latent manifold of a pretrained diffusion model. Rather than interpolating in flat Euclidean space, Stylus refines the interpolated content–style point by lowering a ratio-weighted discrete geodesic energy under the pullback metric, made tractable by a rank-one power-iteration approximation of the Jacobian. This principle drives two complementary controls: content–style latent interpolation for smooth semantic fusion, and self-attention query optimization for spatially adaptive style injection. The resulting dual-control mechanism provides continuous, fine-grained control over stylization strength while preserving identity-critical facial structure, all without additional training. Experiments show that Stylus achieves the strongest identity and content preservation at competitive stylization strength.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Facial Stylization, Geodesic Optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_8ZzpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes Stylus, a training-free algorithm specifically designed for facial image stylization. Given an identity reference image and a style reference image, the method generates a stylized facial image that follows the style of the latter while preserving the identity of the former as faithfully as possible. Similar to prior work, Stylus obtains the target image by interpolating between the identity and style references in the latent space. Its key difference is the introduction of additional correction terms after interpolation to further improve the resulting image.

Critical strengths
  1. Introducing the concept of low-dimensional manifolds into facial stylization is an interesting and potentially insightful idea.
  2. The experimental evaluation is relatively comprehensive, and most of the paper's claims are supported by empirical evidence.
Critical weaknesses
  1. The overall contribution appears somewhat limited. First, the manifold hypothesis has already been extensively explored in the context of diffusion models. Applying it to stylization is a reasonable direction, but it is difficult to regard this extension alone as a major contribution. Second, the core method largely follows the existing paradigm of interpolating between identity and style reference images in the latent space. The main difference is the addition of correction terms after interpolation, which are designed to better align the interpolation process with the manifold hypothesis. The methodological novelty therefore appears limited. Finally, the task is restricted to facial stylization, leaving the generalizability of the proposed insights and method unclear. Moreover, the method does not appear to exploit any face-specific prior, and it is therefore unclear why the authors focus exclusively on faces rather than exploring stylization in a broader domain.

  2. Limited quality of some visual results. Some qualitative results are less convincing, particularly for cartoon-style transfer, where the generated images do not appear to match the target style well.

  3. Poor presentation. The paper contains several noticeable formatting issues, such as the overlap between Fig. 2 and Table 1. In addition, the Limitations section appears to be incorrectly titled, as it does not actually discuss any limitations. The main text also contains a large number of equations, some of which lack sufficient explanation, making the paper harder to follow. I believe the authors could improve readability by reducing the number of equations and providing clearer explanations for the essential ones. Although presentation quality should not be the primary criterion for evaluating a paper's overall quality, the poor readability and several obvious figure-related errors nevertheless raise concerns about the overall quality of the paper.

Questions for authors

I hope the authors can clarify the paper's core contributions during the rebuttal. If my current understanding does not accurately reflect the intended contributions, and the authors can articulate them more clearly and convincingly, I would be willing to reconsider my score.

Reviewer_rwB3OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents Stylus, a training-free framework for controllable facial stylization based on Riemannian geometry. The method formulates content and style fusion as a geodesic-guided energy minimization problem on the diffusion latent manifold. It introduces two complementary mechanisms: latent-space interpolation for content–style fusion and self-attention query optimization for adaptive style injection. To make the optimization computationally tractable, the framework employs a rank-one Jacobian approximation using power iteration. Experiments demonstrate strong content and identity preservation while maintaining competitive stylization quality.

Critical strengths

The paper establishes a meaningful connection between style transfer and Riemannian geometry, providing a mathematically grounded framework for controllable stylization. The geodesic-guided refinement offers a systematic alternative to conventional linear interpolation, enabling smooth and semantically consistent style transitions. Also, the framework requires no additional training and can be applied across multiple Stable Diffusion versions. The experimental results demonstrate strong performance compared to state-of-the-art methods, particularly in content and facial identity preservation, while maintaining competitive style alignment.

Critical weaknesses

Although the proposed method achieves strong quantitative results, the qualitative comparisons do not consistently demonstrate clear improvements in visual quality over existing state-of-the-art methods. In several examples, the generated results appear relatively conservative in their stylization, preserving the original facial structure and appearance but not fully capturing the distinctive artistic characteristics of the reference images. This raises concerns about whether the improvements in identity preservation come at the expense of stylization strength and artistic fidelity.

Questions for authors

The paper evaluates identity preservation using ArcFace[Deng et al., 2019], which is primarily trained on real facial images and may not reliably capture identity across artistic styles. Could the authors provide additional quantitative evaluations using stylization-aware facial identity recognition models, such as StyleID [Yun et al., 2026] or Stylized-Face [Peng et al., 2025], to further validate the claimed improvements in facial identity preservation?

[Minor] How would it perform compared to GPT-Image-2 (or 2.5) or Nano-banana-2?

What is the figure at the beginning of the Appendix?

  • ArcFace: Additive Angular Margin Loss for Deep Face Recognition. CVPR 2019
  • StyleID: A Perception-Aware Dataset and Metric for Stylization-Agnostic Facial Identity Recognition. SIGGRAPH 2026
  • Stylized-Face: A Million-Level Stylized Face Dataset for Face Recognition. ICCV 2025
Reviewer_4jkxOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces Stylus, a training-free framework for controllable facial stylization using pretrained diffusion models. The method is motivated by the observation that conventional content-style interpolation treats diffusion representations as a Euclidean space, which may lead to suboptimal interpolation between content preservation and style transfer. Stylus instead formulates interpolation using the local geometry induced by the diffusion model, starting from a linear content-style interpolation and refining it by minimizing a discrete geodesic energy under a pullback metric. To make this optimization tractable, the method approximates the Jacobian-based update using power iteration and its leading singular direction.

The proposed geometric refinement is applied at two levels: first to the content and style latent representations to control their overall mixture, and second to content and style self-attention queries to provide finer control over style injection during generation. The method requires no additional training and exposes parameters for controlling the content-style trade-off.

The authors evaluate Stylus on facial stylization using WikiArt and MetFace style references and compare against conventional style-transfer, diffusion-based, and recent image-editing baselines. The reported results show strong content and identity preservation while maintaining competitive style alignment.

Critical strengths
  1. The geometric motivation of performing content-style interpolation on the diffusion-induced manifold is intuitive and well motivated compared with directly assuming a Euclidean latent space.
  2. The proposed framework is conceptually coherent, applying the same geometric refinement principle to both latent interpolation and self-attention query interpolation.
  3. The method is training-free.
  4. The experimental results demonstrate strong identity and content preservation while maintaining competitive style alignment.
  5. The paper provides reasonably extensive analysis, including component ablations, hyperparameter studies, Jacobian approximation analysis, computational cost, backbone generalization, and a user study.
Critical weaknesses
  1. The methodological presentation is difficult to follow. While the high-level motivation of replacing Euclidean interpolation with geometry-aware refinement is intuitive, the details of the proposed method are difficult to understand from the current presentation. A clearer step-by-step description connecting the equations to the implementation would substantially improve the paper. Also, more clarity on the theory and the exact algorithm that has been proposed would help.

  2. The choice of base generative models is outdated. The main experiments rely on Stable Diffusion 1.x/2.x models, with the additional backbone experiments considering SD v1.5, v2.0, and v2.1. Given the rapid progress in image generation and editing models, it is unclear whether the proposed geometric refinement remains beneficial with stronger, more recent architectures. Although FLUX.2-dev and Qwen-Image-Edit are included as editing baselines, the proposed method itself is still evaluated on the older Stable Diffusion architecture.

  3. The experimental comparison is not sufficiently clear regarding backbone and setting consistency. It is difficult to determine the exact backbone used in Table 1 from the main experimental section. The paper states that it uses SD1.4, SD1.5, SD2.0, and SD2.1, while the appendix reports comparisons across several SD versions, but the precise configuration corresponding to the main Table 1 should be stated explicitly. Moreover, the compared methods span feed-forward style-transfer models, diffusion methods, and DiT-based editing models, making it important to clearly explain which methods use the same backbone, resolution, inversion procedure, conditioning information, and auxiliary components.

  4. Page 7 has formatting issues.

  5. The empirical evidence for the geometric interpretation could be stronger? The paper provides a path-curvature analysis showing that the refined trajectory departs from linear interpolation while remaining smoother than a randomly perturbed trajectory. However, it is also noted that this does not demonstrate intrinsic manifold curvature or prove the geometric assumptions. Given that the geometric interpretation is the central motivation of the paper, stronger empirical evidence connecting the proposed geometric formulation specifically to the observed performance gains would strengthen the contribution.

Questions for authors
  1. More clarity on the method, experimental settings would help.
  2. How would this work with SOTA architectures?

Rating justification: The method including theory and the algorithm are hard to understand, experiments are outdated, the link between the implemented algorithm, observed trajectory and motivation/ theory for algorithm seem weak. Improving the paper would make it more useful to the community.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.