acceptodds
Under review as a conference paper at ICLR 2027

Dragometry: 3D Geometric Control for Drag-Based Facial Image Editing

Abstract

Point-based drag editing provides an intuitive interface for image manipulation, but existing diffusion-based methods typically operate on learned intermediate representations in which geometric information is encoded only implicitly. Geometry-conditioned diffusion models offer a different setting: explicit structured geometry is already exposed as part of the generative condition. We therefore propose DRAGOMETRY, a point-based editing framework that performs dragging directly in the geometry condition space rather than through learned feature correspondence. We instantiate the framework on MMDM with FLAME for facial editing. User specified 2D handles are associated with persistent 3D surface points, allowing drag constraints to be resolved through different geometric variables. For local deformation, we optimize FLAME expression and jaw parameters and introduce Jacobian-based parameter weighting to favor parameters relevant to the requested motion while suppressing unwanted deformation elsewhere. For viewpoint edits, we instead optimize camera parameters while preserving the underlying facial geometry. We further introduce DragFace, a benchmark containing both facial deformation and rotation edits, together with Geometry Mean Distance, which evaluates drag accuracy using explicit geometric correspondence. Experiments show that our method achieves more accurate geometric point control, more effective facial deformation, and substantially larger coherent viewpoint changes than existing diffusion-based drag editing methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
point-based drag editing, geometry-conditioned diffusion model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kC2qOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Dragometry, a point-based facial image editing framework that projects 2D user drags onto a 3D FLAME mesh to perform either local deformation or viewpoint rotation. The updated 3DMM parameters and camera poses are then fed as explicit conditions into a pretrained Morphable Multi-view Diffusion Model to synthesize the edited face image. The core technical focus of this explicit mesh manipulation lies in a Jacobian-based parameter weighting strategy designed to localize deformations and suppress unintended changes in distant facial regions. Additionally, the authors introduce DragFace, an FFHQ-based evaluation benchmark, alongside a Geometry Mean Distance metric. Compared to purely latent- or feature-based 2D drag editing baselines, the proposed framework demonstrates noticeably more robust and coherent results.

Methodologically, since image synthesis relies entirely on an off-the-shelf, frozen pretrained diffusion model, the core of this work is essentially solving for 3D facial mesh deformation or rigid rotation from 2D handle constraints (i.e., screen-space inverse kinematics). From this perspective, the paper requires much more thorough experimentation on the 3D deformation optimization itself—including comparisons against classic 3D/blendshape manipulation methods, as well as comprehensive ablation studies and hyperparameter sensitivity analyses. From the perspective of facial image editing, while comparing against generic 2D drag-based methods fits the paper's narrative, it is essential within the face editing domain to also benchmark against 3DMM- or ControlNet-based portrait editing approaches. Without such comparisons, the necessity and specific advantages of formulating facial manipulation as a point-based dragging problem over existing conditional face editing paradigms remain unsubstantiated.

Critical strengths
  1. clearly illustrates the unreliability of implicit diffusion feature correspondence (e.g., DIFT) under large facial deformations or rotations (Fig. 2). Shifting the drag operation from feature space to an explicit 3D geometry condition space is intuitive and geometrically sound.

  2. point out that standard DIFT-based Mean Distance (MD) can be misled by global face translation or feature mismatching (Fig. 4). The proposed GeoMD, which measures 3D surface correspondence while discounting global translation, is a more reliable metric for facial dragging.

  3. The paper is well-written and easy to follow. The authors also provide an honest failure analysis in Appendix D (Figs. 11 & 12).

Critical weaknesses

Since both the front-end 3D reconstruction and the back-end image synthesis rely entirely on off-the-shelf pretrained models, the proposed method essentially reduces to optimizing 3D FLAME mesh deformation or camera pose from sparse 2D screen-space constraints.

In computer graphics, this is fundamentally classic screen-space inverse kinematics (IK) or direct manipulation of blendshapes (e.g., Lewis & Anjyo, 2010; Seo et al., 2011; Cetinaslan et al., 2015). Performing simple gradient descent over a low-dimensional linear parameter space offers relatively limited algorithmic novelty for ICLR.

Moreover, confining the deformation optimization strictly to a 66-dimensional FLAME subspace (65 expression PCA coefficients + 1 jaw parameter) inherently caps the method's expressive ceiling—it fundamentally cannot handle fine-grained, out-of-subspace local edits (e.g., asymmetric cheek puffing, tongue movement, or dragging non-FLAME regions like hair and glasses).

In addition, given that the core contribution lies in overcoming global basis entanglement for localized 3D facial mesh manipulation, the paper completely lacks discussion and experimental comparisons with established graphics methods on direct blendshape manipulation and localized facial rigs (e.g., Tena et al., 2011; Neumann et al., 2013), as well as any robustness evaluation against initial 3D reconstruction errors.

[Lewis & Anjyo, 2010] John P. Lewis and Ken-ichi Anjyo. "Direct Manipulation Blendshapes."

[Seo et al., 2011] Jaewoo Seo, Geoffrey Irving, John P. Lewis, and Junyong Noh. "Compression and Direct Manipulation of Complex Blendshape Models."

[Cetinaslan et al., 2015] Ozan Cetinaslan, Verónica Orvalho, and John P. Lewis. "Sketch-Based Controllers for Blendshape Facial Animation."

[Tena et al., 2011] J. Rafael Tena, Fernando De la Torre, and Iain Matthews. "Interactive Region-Based Linear 3D Face Models."

[Neumann et al., 2013] Thomas Neumann, Kiran Varanasi, Stephan Wenger, Markus Wacker, Marcus Magnor, and Christian Theobalt. "Sparse Localized Deformation Components."

More disscusions are in Questions part.

Questions for authors
  1. As the primary algorithmic design of the paper, the Jacobian-based parameter weighting is under-evaluated (supported only by a two-row Table 3)

  2. Given that Eq. (3) already includes a vertex preservation loss to penalize distant vertex displacements, why are its first-order gradients insufficient to prevent global deformation without introducing a heuristic Jacobian pre-filtering step with multiple hardcoded hyperparameters (e.g., K=24,τu=20K=24, \tau_u=20)? Furthermore, why does the ablation study (Table 3) replace the Image Fidelity (IF) metric from Table 1 with Identity Similarity, rather than directly measuring the 3D Vertex-to-Vertex (V2V) error on the unedited mesh region to explicitly validate deformation locality?

  3. Comparing a framework built on a domain-specific multi-view 3D portrait model (MMDM) against generic 2D drag baselines without 3D facial priors is inherently asymmetric (especially for viewpoint rotation); can the authors provide comparisons against recent 3D-aware drag methods (e.g., FlowDrag, DragIn3D) as well as running feature-based dragging directly on MMDM? More importantly, within the facial editing domain, what is the specific necessity and advantage of a point-based drag interface compared to existing ControlNet- or 3DMM-conditioned portrait editing methods?

  4. Were the 410 handle-target pairs in DragFace manually annotated by human users, or synthetically generated by perturbing FLAME parameters and re-projecting (which would introduce severe subspace bias in favor of your method)?

13 comments
AuthorsOpenReview ↗

We thank reviewer kC2q for their careful evaluation and constructive feedback, particularly for recognizing the advantages of explicit 3D geometric control and the effectiveness of our editing results. However, we respectfully note that several criticisms appear to be factual misunderstandings of our method, its technical contributions, and the scope of our evaluation. We clarify these distinctions and address the reviewer’s specific concerns below.

  1. "... the proposed method essentially reduces to optimizing 3D FLAME mesh deformation or camera pose from sparse 2D screen-space constraints. In computer graphics, this is fundamentally classic screen-space inverse kinematics (IK) or direct manipulation of blendshapes (e.g., Lewis & Anjyo, 2010; Seo et al., 2011; Cetinaslan et al., 2015). Performing simple gradient descent over a low-dimensional linear parameter space offers relatively limited algorithmic novelty for ICLR."

We respectfully disagree with the reviewer's characterization of DRAGOMETRY's contribution reducing to the classical 3D mesh manipulation method. This overlooks the central objective of our work: using explicit 3D geometry to control facial image generation by guiding the latent representations of a pretrained diffusion model.

First, performing drag-based editing at inference time using a pretrained generative model is an established paradigm. All four baseline models—DragDiffusion, DragNoise, GoodDrag, and DragLoRA—build upon pretrained Stable Diffusion backbones. Their contributions lie in how they manipulate the generative process at inference time to satisfy user-specified dragging constraints, whether through latent optimization, intermediate feature manipulation, or image-specific model adaptation. DRAGOMETRY follows the same general paradigm but introduces a different control mechanism: it optimizes explicit FLAME parameters to construct geometric conditions that guide MMDM toward the desired facial image. Therefore, using an existing generative backbone does not reduce our contribution to conventional mesh manipulation. Our contribution lies in how sparse 2D dragging constraints are translated into effective control of face image generation.

Second, using pretrained models to guide the generation of another pretrained diffusion model is an established research direction in computer vision. For example, Universal Guidance (Bansal et al., ICLR 2024) uses off-the-shelf pretrained networks, including face-recognition models, to guide pretrained diffusion generation. FreeDoM (Yu et al., ICCV 2023) similarly employs pretrained networks, such as face detectors, to construct guidance signals without additional training. Other representative works include classifier guidance (Dhariwal and Nichol, NeurIPS 2021), Diffusion Self-Guidance (Epstein et al., NeurIPS 2023), and Plug-and-Play Diffusion Features (Tumanyan et al., CVPR 2023). Although their specific mechanisms differ, these studies demonstrate that developing new control mechanisms for pretrained generative models is itself a meaningful research contribution.

Third, DRAGOMETRY differs fundamentally in its objective from the classical graphics methods cited by the reviewer. Lewis and Anjyo (2010), Seo et al. (2011), and Cetinaslan et al. (2015) investigate direct manipulation of 3D blendshape models or facial animation rigs, where the intended outputs are modified 3D geometry or animations. In contrast, DRAGOMETRY does not aim to output an edited 3D mesh. Its intermediate output consists of optimized 3DMM parameters, while its final output is an edited facial image generated by a pretrained diffusion model.

Moreover, in DRAGOMETRY, FLAME parameters provide explicit geometric conditions that guide MMDM's denoising process. Different geometric conditions influence the evolution of the diffusion latents and consequently the generated facial appearance. Our method therefore optimizes FLAME parameters not simply to produce a desired 3D deformation, but to guide a pretrained diffusion model toward the facial image specified by sparse 2D dragging constraints. This distinguishes our objective from classical blendshape manipulation, where the geometric deformation itself is the intended output.

We acknowledge that the cited graphics methods provide relevant foundations for our geometric optimization component. We revised the Related Work section to incorporate the suggested references and clarify how they differ from DRAGOMETRY. Please see the newly added text in blue (Lines 845-856) in the uploaded PDF. Nevertheless, reducing DRAGOMETRY to classical screen-space inverse kinematics does not accurately characterize the complete framework or its research objective. We hope these clarifications resolve the misunderstanding regarding our technical contribution.

  1. "Moreover, confining the deformation optimization strictly to a 66-dimensional FLAME subspace (65 expression PCA coefficients + 1 jaw parameter) inherently caps the method's expressive ceiling—it fundamentally cannot handle fine-grained, out-of-subspace local edits (e.g., asymmetric cheek puffing, tongue movement, or dragging non-FLAME regions like hair and glasses)."

We agree that the expressive capacity of FLAME imposes limitations on achievable deformations. However, manipulating arbitrary image regions, such as hair or glasses, is outside the scope of our claimed contributions.

As explicitly described in the Introduction and illustrated in Fig. 1, DRAGOMETRY focuses on two tasks: facial-expression deformation and viewpoint rotation. Consequently, editing hair, glasses, or other regions not represented by FLAME is not an intended capability of our current formulation.

We acknowledge that certain fine-grained facial deformations, such as tongue movement or highly localized asymmetric expressions, may also fall outside FLAME's representational capacity. This limitation arises from the chosen geometric representation and is already acknowledged in our Conclusion (Sec. 6).

  1. "In addition, given that the core contribution lies in overcoming global basis entanglement for localized 3D facial mesh manipulation, the paper completely lacks discussion and experimental comparisons with established graphics methods on direct blendshape manipulation and localized facial rigs (e.g., Tena et al., 2011; Neumann et al., 2013), as well as any robustness evaluation against initial 3D reconstruction errors."

We agree that these classical graphics methods are relevant to the geometric optimization component of DRAGOMETRY and have incorporated them into the revised Related Work. However, fair direct experimental comparisons are nontrivial because these methods operate on different geometric representations and address different output objectives.

Tena et al. (2011) introduce region-based 3D facial models for interactive mesh manipulation and animation, while Neumann et al. (2013) extract localized deformation components from animated 3D mesh sequences. In contrast, DRAGOMETRY requires updated FLAME parameters as conditioning inputs to a pretrained diffusion model, with the final objective of generating an edited facial image rather than an edited 3D mesh. Neither method directly produces these FLAME conditioning parameters. Integrating them into our pipeline would require additional model adaptation or fitting their edited geometry back into the FLAME parameter space, making a direct comparison under identical conditions nontrivial.

Alternatively, these methods could be evaluated as independent baselines. However, they do not directly support the sparse 2D handle–target interface used in DragFace, instead operating on 3D facial geometry or deformation components. A fair comparison would require reconstructing 3D geometry from the source image, mapping 2D dragging constraints to corresponding 3D deformation targets, and synthesizing the edited geometry back into a 2D facial image while preserving the source appearance. These additional steps are nontrivial and would require substantial methodological adaptations beyond the original methods, rather than a direct application of existing baselines.

Regarding initial reconstruction errors, our work focuses on facial image editing rather than 3D reconstruction accuracy. We employ an existing pretrained FLAME fitting model to obtain the initial geometric conditions and evaluate the resulting edits using the same benchmark and evaluation protocol. DragFace contains real FFHQ images without ground-truth 3D meshes; therefore, absolute 3D reconstruction errors cannot be directly quantified. We acknowledge that inaccurate initial fitting can affect downstream editing, but improving or independently benchmarking the pretrained reconstruction model is outside the primary scope of this work.

Reviewer_gwXDOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a novel and practical framework for point-based facial image editing. Instead of manipulating implicit 2D diffusion features like prior drag-based methods, the proposed approach directly optimizes explicit 3D template variables (FLAME parameters) based on user dragging instructions. To achieve localized control and disentangle the highly correlated expression parameters, the authors introduce a Jacobian-based weighting strategy and regularization losses to isolate the relevant parameters while suppressing unwanted deformations in unrelated facial regions. Finally, the optimized geometry conditions, alongside the original image as a reference, are fed into a pretrained Morphable Multi-view Diffusion Model (MMDM) to generate the edited image. The paper is well-written, the methodology is intuitive, and the approach appears straightforward to reproduce.

Critical strengths
  • The core premise—that editing within an explicit 3D template space is fundamentally more robust and precise for geometric manipulation than tracking implicit diffusion features—is highly convincing. The proposed algorithm elegantly and perfectly realizes this goal.

  • The method demonstrates strong performance across various drag-based editing scenarios. The qualitative results presented in the figures are compelling, showing natural and precise edits that indicate high potential for practical downstream applications.

Critical weaknesses
  • The paper omits comparisons with closely related prior works that are highly relevant to disentangled geometry/appearance editing and 3D-aware dragging:

(1) DeepFaceEditing (Deep Face Generation and Editing with Disentangled Geometry and Appearance Control): While published a few years ago, it achieves excellent results in geometry-based editing (e.g., closing eyelids). Although it might lack the robust 6DoF control of the proposed method, a direct comparison is necessary to properly contextualize the authors' advancements over established 3D-aware face editing baselines.

(2) ObjectMorpher (3D-Aware Image Editing via Deformable 3DGS Models): This is another highly relevant work that tackles image editing in 3D space via 2D-to-3D lifting, ARAP-based interactive local deformation (dragging), and a 3D-to-2D refinement diffusion model. Discussing or comparing with this work would significantly strengthen the literature review.

  • The current Jacobian-based weighting strategy, while empirically effective, feels somewhat heuristic and mathematically "rough" in its design. From an optimization perspective, the goal is to optimize a variable in an N-dimensional space while satisfying specific constraints (preserving unrelated regions). Instead of the current masking/weighting approach, a more principled method would be to analyze the principal directions of the editing loss and the constraints via the Jacobian matrices. Specifically, one could project the editing gradient onto the tangent space of the constraint manifold—for example, by removing the component of the constraint's Jacobian vector from the editing vector. This would theoretically allow the optimization to proceed smoothly within the valid subspace without the need for hard thresholds.
Questions for authors
  • Could you provide comparisons—or discuss the specific trade-offs—with established 3D-aware editing baselines like DeepFaceEditing and ObjectMorpher?

  • The hard threshold (τu\tau_u) approach is heuristic. Did you experiment with more mathematically principled constrained optimization (e.g., projecting the editing gradient onto the tangent space of the unwanted-region constraints)? If so, what were the practical bottlenecks?

4 comments
AuthorsOpenReview ↗

We thank reviewer gwXD for their review and comments. We are glad that you found this paper easy to follow and could learn something from our work! We would like to address your concerns below.

  1. "The paper omits comparisons with closely related prior works that are highly relevant to disentangled geometry/appearance editing and 3D-aware dragging, ..." "Could you provide comparisons—or discuss the specific trade-offs—with established 3D-aware editing baselines like DeepFaceEditing and ObjectMorpher?"

We agree that DeepFaceEditing and ObjectMorpher are relevant 3D-aware editing methods. However, their editing interfaces and underlying optimization formulations differ from DRAGOMETRY, making direct comparisons under the same sparse 2D dragging protocol nontrivial.

  • DeepFaceEditing (Chen et al., 2021) disentangles facial geometry and appearance to enable sketch- and reference-based face editing, including expression manipulation. However, it requires richer geometric editing inputs (e.g., sketches) rather than sparse handle–target point pairs. Its richer inputs enable detailed facial shape control but are not directly interchangeable with sparse handle–target pairs. DRAGOMETRY instead optimizes explicit FLAME conditions from sparse 2D dragging constraints to guide facial image generation using a pretrained diffusion model. All four baselines use the same sparse 2D point-based dragging edit.
  • ObjectMorpher (Xie et al., CVPR 2026) reconstructs objects into a deformable 3DGS representation and performs ARAP-based manipulation through interactive 3D control points, followed by diffusion-based image refinement. Although its general 3D representation supports object-level geometric editing, its interface requires 3D manipulation targets, whereas DragFace specifies only 2D handle–target positions. Adapting ObjectMorpher for a fair comparison would require inferring 3D targets from 2D displacements, which would effectively be a new method building on ObjectMorpher.

So, these approaches offer complementary geometric control capabilities, but neither method provides this 2D point-based control formulation evaluated in DragFace.

We revised the Related Work section to incorporate the suggested references and clarify how they differ from DRAGOMETRY. Please see the newly added text in blue (Lines 838–840 and Line 845-856) in the uploaded PDF.

  1. "The current Jacobian-based weighting strategy, while empirically effective, feels somewhat heuristic and mathematically "rough" in its design. ..." "The hard threshold (τμ\tau_{\mu}) approach is heuristic. Did you experiment with more mathematically principled constrained optimization (e.g., projecting the editing gradient onto the tangent space of the unwanted-region constraints)? If so, what were the practical bottlenecks?"

We agree that projecting the editing gradient onto the null space of the preservation constraints is a mathematically principled alternative. However, strict null-space projection can be overly restrictive for FLAME, whose expression parameters are spatially entangled.

Consider an extreme but illustrative case: FLAME has 65 editable expression parameters, while preserving unrelated facial regions may introduce thousands of vertex-level constraints. If the preservation Jacobian has full column rank (65), its null space becomes zero-dimensional, and the projected editing gradient is necessarily zero. Consequently, optimization cannot proceed, even when some expression parameters could achieve the desired handle movement with only minor unwanted deformation. This illustrates why strictly enforcing preservation constraints may be less suitable than allowing a controlled trade-off between dragging accuracy and deformation locality.

Our Jacobian-based weighting instead adopts a soft trade-off between dragging accuracy and deformation locality. We acknowledge that the hard threshold τμ\tau_{\mu} is a heuristic design choice. However, the underlying Jacobian-based criterion is geometrically motivated, as it measures how individual FLAME parameters contribute to desired handle movement and unwanted deformation in unrelated facial regions. Our method favors parameters that effectively move the handles while suppressing those that cause substantial deformation in unrelated facial regions. The vertex-preservation loss further discourages unwanted changes without imposing strict equality constraints.

However, we agree that empirically characterizing this trade-off would be insightful, and are conducting additional experiments, examining Jacobian weighting and different vertex-preservation strengths, and will add an ablation comparing our weighting strategy against the proposed null-space gradient projection. We will evaluate geometric accuracy, identity preservation, and unwanted facial deformation to investigate the effects of increasingly restrictive preservation constraints.

AuthorsOpenReview ↗

Additional Ablation Studies on Jacobian-Based Weighting

  1. Comparison with exact Jacobian null-space projection

We thank reviewer gwXD for suggesting this comparison. We investigate exact Jacobian null-space projection as an alternative to our Jacobian-based parameter weighting strategy. This approach restricts parameter updates to directions that produce zero first-order displacement in unrelated facial regions.

Given the 66-dimensional FLAME deformation parameters q∈R66q \in \mathbb{R}^{66}, we construct the preservation Jacobian Jp∈R3Np×66J_p \in \mathbb{R}^{3N_p \times 66}, which measures the 3D displacement of NpN_p preserved vertices with respect to the deformation parameters. The preserved vertices are selected outside the local neighborhoods of the handle-associated triangles.

For an original optimization direction g=−∇qLgeog = -\nabla_q \mathcal{L}_{\mathrm{geo}}, exact null-space projection computes:

$ g_{\mathrm{proj}} = (I-J_p^\dagger J_p)g, $ where Jp†J_p^\dagger is the pseudoinverse. This projection restricts parameter updates to the null space of JpJ_p, ensuring zero first-order displacement of the preserved vertices.

We evaluated this formulation across all 390 deformation cases in DragFace. In every case, the preservation Jacobian has full column rank, rank⁡(Jp)=66\operatorname{rank}(J_p)=66. Consequently, its null space contains only the zero vector:

$ \operatorname{Null}(J_p)=\{0\}, \qquad I-J_p^\dagger J_p=0. $

Therefore, gproj=0g_{\mathrm{proj}}=0, regardless of the original optimization direction. Exact null-space projection completely suppresses the parameter updates and cannot realize the requested facial deformation under these strict preservation constraints.

In contrast, our Jacobian-based weighting strategy explicitly distinguishes FLAME parameter dimensions according to their contributions to the requested motion and unwanted deformation. Due to the spatial entanglement of FLAME/3DMM parameters, a parameter that moves the requested handle may also affect unrelated facial regions. Consequently, requiring zero displacement of all preserved vertices can eliminate parameter-update directions that are also necessary for the requested facial deformation.Instead, we use Jacobian responses to first identify the parameters most relevant to the requested handle motion, then suppress those that cause substantial deformation in unrelated regions. Parameters with limited unwanted influence are retained and weighted for optimization, rather than requiring their effects on preserved vertices to be exactly zero. This dimension-wise selection and weighting avoids the overconstrained behavior of exact null-space projection while balancing the requested motion and deformation preservation.

We have added this ablation to the experimental results section of the revised PDF, with the mathematical formulation and implementation details provided in the appendix.

AuthorsOpenReview ↗

Additional Ablation Studies on Jacobian-Based Weighting

  1. Sensitivity analysis of the vertex-preservation loss weight

We further investigate whether the benefits of Jacobian-based weighting can be achieved simply by increasing the vertex-preservation loss weight. We evaluate four multipliers of the default weight (0.25×, 0.5×, 1.0×, and 2.0×), with and without Jacobian weighting, on the same 30 randomly selected deformation cases.

The results reveal three observations:

  • Similar trends and trade-off: Both variants show a similar trend as the preservation weight increases: identity similarity generally improves, while Geometry MD slightly increases. This indicates a trade-off between identity preservation and drag accuracy
  • Complementary mechanisms: The similar trends, together with the additional identity-preservation benefit from Jacobian weighting, suggest that the two mechanisms are complementary. Jacobian weighting suppresses parameters that cause substantial deformation in unrelated facial regions while retaining those that strongly support the requested handle motion with relatively limited unwanted effects. However, these retained parameters may still introduce smaller displacements in unrelated regions during optimization. The vertex-preservation loss further penalizes such displacements, providing complementary control over unwanted deformation.
  • More favorable preservation–accuracy balance: The upper-left region of the trade-off plot corresponds to higher identity similarity and lower Geometry MD. Across all tested weights, Jacobian weighting achieves higher identity similarity with comparable drag accuracy.

These results support the effectiveness of Jacobian weighting beyond simply adjusting the vertex-preservation loss strength within the tested range. We have added this sensitivity analysis and the corresponding trade-off plot to the experimental section of the revised PDF.

Reviewer_L5EkOpenReview ↗
Rating
2
Confidence
3(+1)
Informativeness
2
Presentation
3
Soundness
1(−1)
Read review
Paper summary

The paper introduces a method for point based geometry editing in photos of human faces. they do so by conditioning face generation to a 3DMM FLAME shape. To apply deformations they first fit 3DMM on input image, run an optimization to find a 3DMM shape that fits desired vertex transformations and use that as conditioning for the diffusion model. The paper proposes a Jacobian-based parameter weighting strategy that reduces geometry entanglement leading to local transformations. They further introduce a benchmark for evaluating genotry changes in human faces and is teh basis of thei evaluation.

Critical strengths
  • The method is sound and makes sense. Jacobian-based parameter weighting is helpful with direct application in finding parameters that encourage disentanglement of resulting geometry.
Critical weaknesses
  • A 3DMM can pose limitations in the domain of characters it can represent, solutions based on image diffusion models w/o 3DMMs can generalize further due to less contstraints in their conditioning mechanism. More general approaches not based on 3DMM conditioning dont suffer from thjis limitation. All 4 methods, DragDiffusion (Shi et al., 2024), DragNoise (Liu et al., 2024), GoodDrag (Zhang et al., 2025), and DragLoRA (Xia et al., 2025) with which the paper compares are general diffusion methods not trained explicitly on faces so it is not surprising that they are outperformed. For a fair comparison these methods should be finetuned to the same data as the proposed method.

  • Qualitative comparison Fig 5, main diffs are between open and closed mouth and eyes which the baselines could very weel not have seen in their training data.

  • The main technical contribution of the paper is on the quality of the conditioning signal for diffusion model image generation. The Jacobian weighting strategy which prodiuces good disentanglement for teh 3D geometry, other than that it uses a model already trained on producing images of people. Given that the Jacobian strategy is not a novel idea the contribution of this work is limited.

  • For related work I identified some missing literature. These deal with shape manipulation in morphable models, the latest two operate directly on 3D vertex geometry space similar to the conditioning inputs of DRAGOMETRY:

Questions for authors
  • have you evaluated the data baselines were trained with to ensure all experiments actually test these models in domain? My concern is that the baselines may have never seen the transformations they were tested in such as eyes and mouths open->close and vice versa. It is critical to know what data were used to train MMDM vs the baselines.
  • How many points can be used silmuntaneously in your method?
11 comments
AuthorsOpenReview ↗

Thank you very much for your review and comments. We are glad that you found this paper easy to follow and could learn something from our work! We believe several of your comments contain misinterpretations of our contributions. We would like to clarify several points below.

  1. "A 3DMM can pose limitations in the domain of characters it can represent, solutions based on image diffusion models w/o 3DMMs can generalize further due to less contstraints in their conditioning mechanism. More general approaches not based on 3DMM conditioning dont suffer from thjis limitation. All 4 methods, DragDiffusion (Shi et al., 2024), DragNoise (Liu et al., 2024), GoodDrag (Zhang et al., 2025), and DragLoRA (Xia et al., 2025) with which the paper compares are general diffusion methods not trained explicitly on faces so it is not surprising that they are outperformed. For a fair comparison these methods should be finetuned to the same data as the proposed method."

We agree that 3DMM constrains the range of representable deformations, but disagree that general-purpose diffusion models necessarily lack facial priors or that the reported differences can be attributed only to face-specific training.

First, DRAGOMETRY deliberately uses a structured 3D facial representation to establish reliable surface correspondence and geometrically meaningful editing controls. This benefits facial deformation and viewpoint manipulation but necessarily restricts the editing space to transformations representable by FLAME. General-purpose diffusion methods support a broader range of objects and editing operations. We acknowledge this trade-off and do not claim that DRAGOMETRY offers more general image-editing capabilities.

Second, although the four baselines use general-purpose pretrained diffusion models, their backbones were trained on datasets containing facial images. Specifically, Stable Diffusion was trained on subsets of LAION-5B and AlDahoul et al. (AI-generated faces influence gender stereotypes and racial homogenization, 2025) explicitly identified 88,714 facial images in a sampled high-resolution subset of LAION-5B. Moreover, these methods have previously been evaluated on human-head editing tasks, including DragBench (DragDiffusion, Shi et al., CVPR 2024). Therefore, the observed performance differences cannot simply be explained by an absence of facial information in their generative models.

We evaluate all baselines in their intended configurations, and none of the evaluated methods is trained or fine-tuned on DragFace. Replacing the baselines' backbones with MMDM is not a straightforward fine-tuning operation: conventional feature-based dragging does not account for the explicit FLAME condition. Keeping that condition fixed can conflict with feature manipulation, while changing it makes conventional diffusion-feature tracking unreliable, as demonstrated in Fig. 2.

  1. "Qualitative comparison Fig 5, main diffs are between open and closed mouth and eyes which the baselines could very weel not have seen in their training data."

We disagree with the premise that the pretrained Stable Diffusion backbones used by the baselines lack the facial priors necessary to represent open- and closed-mouth or eye configurations. This capability has already been demonstrated independently in the literature.

As discussed in our response to Point 1, the four baselines—DragDiffusion, DragNoise, GoodDrag, and DragLoRA—use pretrained Stable Diffusion backbones with learned facial appearance priors and have previously been evaluated on human-head editing tasks of DragBench (DragDiffusion, Shi et al., CVPR 2024).

More directly, independent studies demonstrate that vanilla Stable Diffusion models can generate facial expressions without face-specific fine-tuning:

ReferenceEvidence
F-Bench (ICCV 2025)Supplementary Fig. 1: pretrained SD 1.5/2.1 generating crossed eyes and smiles with visible gums
ITI-GEN (ICCV 2023)Fig. 1 and Table 1: vanilla SD generating human portraits and representing smiling expressions

These independent results provide specific evidence that pretrained Stable Diffusion models can represent diverse eye and mouth configurations, including crossed eyes and smiles with visible gums, without face-specific fine-tuning. Although they do not verify the exact samples from DragFace, they directly challenge the assumption that the baselines' backbones lack the facial appearance priors required for such configurations. However, the ability to generate these facial images does not necessarily guarantee successful drag-based transformations between open/closed mouth/eyes.

  1. "The main technical contribution of the paper is on the quality of the conditioning signal for diffusion model image generation. The Jacobian weighting strategy which prodiuces good disentanglement for teh 3D geometry, other than that it uses a model already trained on producing images of people. Given that the Jacobian strategy is not a novel idea the contribution of this work is limited."

We disagree that DRAGOMETRY's technical contribution is limited to Jacobian weighting or that utilizing a pretrained diffusion model diminishes the significance of the proposed method.

Developing effective guidance and control mechanisms for pretrained diffusion models is an established and important research direction. For example, classifier guidance (Dhariwal and Nichol, NeurIPS 2021) uses gradients from a pretrained classifier to guide diffusion generation, while ControlNet (Zhang et al., ICCV 2023) introduces spatial conditioning for pretrained diffusion models. Universal Guidance (Bansal et al., ICLR 2024) uses off-the-shelf pretrained networks, including face-recognition models, to guide pretrained diffusion generation. FreeDoM (Yu et al., ICCV 2023) similarly employs pretrained networks, such as face detectors, to construct guidance signals without additional training. These works demonstrate that enabling new forms of control over pretrained generative models constitutes a meaningful technical contribution.

Similarly, although MMDM already supports FLAME conditioning, it does not provide a mechanism to infer the desired geometric conditions from sparse user-specified dragging constraints. DRAGOMETRY addresses this problem by establishing persistent 3D surface correspondence and optimizing the explicit FLAME conditions to guide the generation of the desired facial image. This formulation also avoids unreliable diffusion-feature correspondence (Fig. 2) and distinguishes facial deformation from viewpoint rotation through separate geometric parameter spaces.

We also disagree that employing an established Jacobian technique necessarily limits the technical contribution. Unlike previous facial manipulation methods that use Jacobians to directly compute parameter updates (Çetinaslan et al., Transposition Based Blendshape Direct Manipulation 2017, Stabilized Blendshape Editing Using Localized Jacobian Transpose Descent 2020), our method introduces a new Jacobian-guided selection strategy to address spatially entangled FLAME expression parameters. Specifically, we use two Jacobians to evaluate each expression dimension's contribution to the desired drag direction and its influence on unwanted facial regions, respectively. By jointly considering these criteria, we select dimensions that support the intended movement while limiting unrelated deformation. Unlike conventional Jacobian-based parameter subset selection, our design is tailored to spatially constrained facial editing, with the selected dimensions subsequently optimized to guide diffusion-based image generation.

Therefore, our contributions extend beyond reusing pretrained components or existing Jacobian techniques: they include a geometry-condition-space dragging formulation, persistent geometric correspondence, and Jacobian-guided locality-aware optimization, together with the DragFace benchmark and Geometry Mean Distance metric.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Final Remarks

Dear ACs and Reviewers,

We appreciate the constructive discussions throughout the rebuttal period. We have addressed the concerns raised by all three reviewers and completed substantial additional experiments, including comparisons with additional drag-editing methods such as FlowDrag, face-adapted Stable Diffusion baselines, Jacobian null-space projection, and preservation-weight sensitivity experiments. The results and corresponding clarifications are provided in our rebuttals and revised manuscript.

We would like to emphasize two remaining points for the Area Chairs:

Central Contribution: We identify two challenges in applying conventional feature-based dragging to geometry-conditioned diffusion models. This motivates our central research question: How can sparse 2D point-based dragging be achieved in pretrained geometry-conditioned diffusion models by exploiting their explicit 3D geometric information? DRAGOMETRY addresses this question by introducing, to the best of our knowledge, the first working framework for this setting. Our research domain is point-based drag-diffusion image editing, and our contribution should be contextualized against prior methods in this domain, which we have extensively compared against.

Novelty and Scope: Unlike the reviewer concluded, our central contribution is not developing the optimal solution to constrained 3D parameter inference or demonstrating that our Jacobian-based weighting outperforms existing local-control methods from computer graphics. Rather, it is establishing a working framework that makes point-based dragging feasible in geometry-conditioned diffusion models. Even if alternative graphics-based techniques could improve the optimization, integrating them into this setting would require additional methodological contributions. We respectfully disagree that demonstrating superiority over these alternative solvers is necessary to establish the novelty of our proposed framework.

We respectfully ask the Area Chairs and reviewers to consider the paper's central contribution, the additional experimental evidence, and our detailed responses when making their assessment.

We thank the reviewers and Area Chairs again for their time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.